Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Vladimir_M comments on What is bunk? - Less Wrong

20 [deleted] 08 May 2010 06:06PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (101)

You are viewing a single comment's thread.

Comment author: Vladimir_M 08 May 2010 08:29:25PM *  22 points [-]

SarahC:

A key thing to consider is the role of the "mainstream." When a claim is out of the mainstream, are you justified in moving it closer to the bunk file?

An important point here is that the intellectual standards of the academic mainstream differ greatly between various fields. Thus, depending on the area we're talking about, the fact that a view is out of the mainstream may imply that it's bunk with near-certainty, but it may also tell us nothing if the mainstream standards in the area are especially bad.

From my own observations of research literature in various fields and the way academia operates, I have concluded that healthy areas where the mainstream employs very high intellectual standards of rigor, honesty, and judicious open-mindedness are normally characterized by two conditions:

(1) There is lots of low-hanging fruit available, in the sense of research goals that are both interesting and doable, so that there are clear paths to quality work, which makes it unnecessary to invent bullshit instead.

(2) There are no incentives to invent bullshit for political or ideological reasons.

As soon as either of these conditions doesn't hold in an academic area, the mainstream will become infested with worthless bullshit work to at least some degree. For example, condition (2) is true for theoretical physics, but in many of its subfields, condition (1) no longer holds. Thus we get things like the Bogdanoff affair and the string theory wars -- regardless of who (if anyone) is right in these controversies, it's obvious that some bullshit work has infiltrated the mainstream. Nevertheless, the scenario where condition (1) doesn't hold, but (2) does is relatively benign, and such areas are typically still basically sound despite the partial infestation.

The real trouble starts when condition (2) doesn't hold. Even if (1) still holds, the field will be in a hopeless confusion where it's hardly possible to separate bullshit from quality work. For example, in the fields that involve human sociobiology and behavioral genetics, particularly those that touch on the IQ controversies, there are tons of interesting study ideas waiting to be done. Yet, because of the ideological pressures and prejudices -- both individual and institutional -- bullshit work multiplies without end. (Again, regardless of whom you support in these controversies, it's logically impossible that at least one side isn't bullshitting.) Thus, on the whole, condition (2) is even more critical than (1).

When neither (1) nor (2) holds in some academic field, it tends to become almost pure bullshit. Macroeconomics is the prime example.

Comment author: Vladimir_M 08 May 2010 08:52:48PM *  10 points [-]

SarahC:

There are three camps I have in mind, who are outside the academic mainstream, but not obviously (to me) dismissed as bunk: global warming skeptics, Austrian economists, and singularitarians.

So, to apply my above criteria to these cases:

  • Climate science is politicized to an extreme degree and plagued by vast methodological difficulties. (Just think about the difficulty of measuring global annual average temperature with 0.1C accuracy even in the present, let alone reconstructing it far into the past.) Thus, I'd expect a very high level of bullshit infestation in its mainstream, so critics scorned by the mainstream should definitely not be dismissed out of hand.

  • Ditto for mainstream vs. Austrian macroeconomics; in fact, even more so. If you look at the blogs of prominent macroeconomists, you'll see lots of ideologically motivated mutual scorn and abuse even within the respectable mainstream. Austrians basically call bullshit on the entire mainstream, saying that the whole idea of trying to study economic aggregates by aping physics is a fundamentally unsound cargo-cult approach, so they're hated by everyone. While Austrians have their own dubious (and sometimes obviously bunk) ideas, their criticism of the mainstream should definitely be taken into account considering its extreme level of politicization and lack of any clearly sound methodology.

  • As for singularitarians, they don't really face opposition from some concrete mainstream academic group. The problem is that their claims run afoul of the human weirdness heuristic, so it's hard to get people to consider their arguments seriously. (The attempts at sensationalist punditry by some authors associated with the idea don't help either.) But my impression is that many prominent academics in the relevant fields who have taken the time to listen to the singularity arguments take them respectfully and seriously, certainly with nothing like the scorn heaped on dissenters and outsiders in heavily politicized fields.

Comment author: gwern 15 November 2010 11:38:50PM 4 points [-]

If it's not presumptuous of me, I'd like the Bogdanov affair removed as an example. I was one of the Wikipedia administrators deeply involved in the BA edit-wars on Wikipedia, and while I originally came to it with an open mind (why I was asked to intervene), quickly there came to be not a single doubt in my mind that the brothers were complete con artists and possess only a talent for self-promotion and media manipulation.

This is unlike string theory, where there are good arguments on both sides and one could genuinely be uncertain.

Comment author: Vladimir_M 16 November 2010 06:10:41AM *  2 points [-]

However, would you agree that Bogdanoff brothers' work has been, at least at some points, approved and positively reviewed by credentialed physicists with official and reputable academic affiliations? After all, they successfully published several papers and defended their theses.

Now, it may be that after their work came under intense public scrutiny, it was shown to be unsound so convincingly that it led some of these reviewers to publicly reverse their previous judgments. However, considering that the overwhelming majority of research work never comes under any additional scrutiny beyond the basic peer review and thesis defense procedures, this still seems to me like powerful evidence that the quality of many lower-profile publications in the field could easily be as bad.

Comment author: gwern 16 November 2010 05:45:40PM *  2 points [-]

However, would you agree that Bogdanoff brothers' work has been, at least at some points, approved and positively reviewed by credentialed physicists with official and reputable academic affiliations? After all, they successfully published several papers and defended their theses.

As I recall, they didn't defend their theses, and only eventually got their degrees by a number of questionable devices like replacing a thesis with publications somewhere and forcing a shift to an entirely different field like mathematics.

EDIT: The oddities of their theses is covered in http://en.wikipedia.org/wiki/Bogdanoff_affair#Origin_of_the_affair

Comment author: multifoliaterose 20 October 2010 01:38:56AM 0 points [-]

Very articulate comment, it helped clarify my thinking on this topic; thanks.