Peer review existed in some forms well before that. The major journals like the various journals of the Royal Society often had pieces vetted by the editors rather than independent referees, but the point of independent review was not uncommon.

That is true, of course, but the sorts of editorial review that were standard before the mid-20th century were different in some very important ways. When Max Planck accepted Einsten's annus mirabilis papers, the decision was under his personal responsibility, and he was putting his own reputation on the line. This is a very different procedure from today's reliance on anonymous reviewers and impersonal committees. It's also very different from a setup where officially designated "peers" (i.e. insiders of an single exclusive group) serve as bureaucratic judges of what is valid science about some issue and what isn't. (See this old comment in which I elaborated on some important aspects of these differences.)

Also, if one looks at the RW article on peer review, one sees that they list some of the more serious problems with the system.

Notice that the "pseudoscience" article confuses peer review and replication. This clearly indicates a writer who has a very poor understanding of the present institutions of official science, but is nevertheless driven by a strong desire to mount ideological crusades on their behalf. Furthermore, the linked "peer review" article informs us that "peer review is a key part of the scientific method." This is just plain false: whether or not a given work follows the scientific method depends only on the way it's conceived and executed, and not at all on whether it's been given some kind of bureaucratic imprimatur. Robinson Crusoe would be perfectly capable of using the scientific method, limited only by his own knowledge and the resources available on the island.

Furthermore, I don't think the "peer review" article goes anywhere near a comprehensive critique of the system. It doesn't even discuss the most obvious problems that struck me the same moment I was assigned peer review work for the first time some years ago. But that's a topic for a separate long discussion.

Would you object to this sentence instead saying " If an idea has not been published in a single peer review journal, it is a warning sign that the idea may not be science?"

It depends on the field in question. In some fields (basically the hardest of the hard sciences), it is indeed a pretty good heuristic. At the other extreme, in some other fields the situation is so bad that publication in a peer-reviewed journal strongly indicates that even if the work has some valid insight, it's buried in a quagmire of bias and bad epistemology -- whereas on the other hand, perfectly valid insight is often made outside of any official accredited institutions.

In yet other cases, valid science can be done entirely unofficially by amateurs, and this isn't so rare when it comes to topics that are of little or no academic interest.

Most of the rest of the article seems ok to me, although others of your points may be valid, especially in regard to the low status nature of most of their examples (although in many locations alt med is pretty high status, especially in the American left-wing which undermines your claim somewhat.)

I didn't claim that RW falls for left-wing nonsense in general. If the left-wing nonsense in question is opposed by the mainstream academia, or if it's too far left to be embraced by the respectable left-center intellectual and media institutions, RW will also be opposed to it. (Although, again, its criticism will end up biased insofar as these institutions are biased towards the issue.)

The real problem is what happens if pseudoscience is in fact done by the official and prestigious academia and with all the official bureaucratic trappings of "science" in place. A truly correct and informative treatment of "pseudoscience" should give us some idea of how to recognize this situation. Yet the RW writers seem incapable of even conceiving such a possibility, and their entire output is driven by the desire to equate "pseudoscience" with non-academic contrarian positions. (Or, in case of the example of The Bell Curve, with work whose conclusions are unacceptable to the ideological left-center, even though they are more or less in line with the beliefs of a substantial number of academics specializing in the issues it deals with, and the work has some academic affiliation.) Now, you may disagree whether it's fair to label such bias as "ideological" -- although I think it's an appropriate description -- but I don't see on what grounds you would disagree with the diagnosis of the problem itself.

Describing it as a "heavily ideological website" when it's nothing of the sort is not how I would steel-man your argument, and may not serve your advocacy of said argument optimally.

I'm surprised that you perceive this characterization as controversial and unfair. From what I've seen, RW articles engage in scathing rhetoric of ideological warfare constantly and unabashedly, and their ideological perspective is, as far as I can tell, remarkably uniform and consistent.

The only way one could object to my characterization is if one agrees with the ideological positions of RW to such degree that one sees them as objectively correct, i.e. a matter of mere common sense and rational thinking, so that characterizing them as "ideological" would by itself be a dishonest rhetorical ploy. Now, I can accept this argument when it comes to the RW articles that sneer at, say, various physics crackpots (although even there, one wonders why the authors find it a worthwhile use of their time). However, I don't see how one could extend it to topics that are inherently heavily ideological and where we have nothing like the solid epistemological base that exists in hard sciences, like for example economics, history, politics, etc. Yet, from what I've seen, RW articles on such topics sneer and jeer with the same strength of conviction.

What are the best example articles of what you're talking about, and what's wrong with their positions? (If possible, pick something bronze or better... )

To avoid any possibility of cherry-picking, I'll take the links on top of the main page of RW. The first one is RW's article about itself, so I hope I can be pardoned if I skip that one. The next one is an article about "pseudoscience." So what does RW tell us about pseudoscience?

What it tells us is, basically, that "science" should be understood as the bureaucratic system implemented by the contemporary academia, and "pseudoscience" as any effort at finding truth that doesn't have the official imprimatur of this system. For an especially blatant example of this, consider their bizarre claim that peer review is somehow an essential part of science -- whereas in reality, as anyone with even a cursory familiarity with the history of science knows, peer review is a bureaucratic innovation that has been widely imposed only in the decades since WW2, and science had functioned perfectly well for centuries before that. Also, the writers seem to be badly confused about the difference between peer review and the truly important fundamental issue of replication. (By the way, one wonders what cognitive dissonance might be induced if they were aware that Homeopathy is a bona fide peer-reviewed journal!)

Note how the article doesn't even conceive of the possibility that pseudoscience might in fact be practiced by some branches of the official high-status academia, with peer review and all the other bureaucratic frills in place. Its authors clearly wouldn't be able to discuss meaningfully the question of how one could try to evaluate the reliability of the academic output in different areas and determine what sorts of pseudoscience might be thriving under prestigious academic titles and affiliations. On the contrary, all their examples are from low-status folkish superstitions and distant history. (Also, tellingly, the closest thing to a mainstream academic work that is included as an example of "pseudoscience" is The Bell Curve -- a book which was co-authored by a Harvard professor, but whose conclusions are ideologically unpalatable for the sort of people who write on RW.)

This is by no means the worst example -- for that, we'd need to look at articles about topics that have more direct political implications. But I think it does illustrate my points pretty well.

And by the way, what are "the intellectual institutions whose output inspires the RW writers"? I can tell you if they in fact inspired me, for example.

I think this should be clear from the above. What I have in mind is the mainstream academia and, in case of more immediate political topics, the left-centrist perspective of the mainstream media. (For the latter, a good litmus test is if you can imagine a given position being argued by a New York Times op-ed columnist.)

Pretty much any RW article I've ever seen takes the premise that the position of the mainstream academia -- and, in case of more explicitly politicized topics, the left-center of the respectable public opinion -- is correct, and any serious contrarian position can be held only by comically nutty crackpots or sinisterly malevolent extremists. Now, this isn't always a bad heuristic: it produces more or less correct conclusions on topics where the aforementioned institutions are usually reliable, such as, say, physics. But on any topic where they are significantly biased, RW ends up as a passionate defender of all their biases and falsities. And from what I've seen, the RW writers typically make no serious effort to study such topics dispassionately, but instead jump at the first opportunity to engage in ideological warfare, typically via ignorant sneering and mocking.

As an example, you can take pretty much any topic that's matter of significant ideological controversy -- economics, history, human differences, political theory, let alone everyday politics. Out of several dozen RW articles I've read, I don't think I've seen a single one that looks like a fair attempt to make sense of the topic at hand.

Of course, you may believe that the intellectual institutions whose output inspires the RW writers are in fact reliably correct about everything. But even then, it would be clear that the latter are mostly just parroting the former without much real understanding and motivated mainly by the thrill of ideological warfare.

I've browsed through a bunch of topics on RationalWiki, and it looks like a heavily ideological website. Thus, it tends to be extremely unreliable and biased on any topic that has even the slightest whiff of controversy. Anyone who makes a genuine effort to form an accurate view of the world will surely come to have at least some beliefs that will be met with scorn and sneering by the sort of people who write on RW.

He further claims to have once saved $1,200 over the price quoted on the Internet for a car he negotiated for his daughter, who was 3000 miles away at the time.

What does he mean by "price quoted on the Internet"? If it's the manufacturer's suggested retail price, then depending on the car model and various other factors, saving $1,200 over this price sounds unremarkable at best, and a badly losing proposition at worst. If it was the first price quoted by the dealer, it could be even worse -- at least where I live, dealers will often start with some ridiculous quote that's even higher that the MSRP.

From my limited experience with buying cars, as well as from theoretical considerations, this won't work because you lack the pre-commitment to buy at the price offered. Once they give you a favorable price, you can try to push it even further downwards, possibly by continuing to play the dealerships against each other. So they'll be afraid to offer anything really favorable. (The market for new cars is a confusopoly based on concealing the information about the dealers' exact profit margins for particular car models, which is surprisingly well-guarded insider knowledge. So once you know that a certain price is still profitable for them, it can only be a downward ratchet.)

The problem can be solved by making the process double-blind, i.e. by sending the message anonymously through a credible middleman, who communicates back anonymous offers from all dealers. (The identities of each party are revealed to the other only if the offer is accepted and an advance paid.) Interestingly, in Canada, someone has actually tried to commercialize this idea and opened a website that offers the service for $50 or so (unhaggle.com); I don't know if something similar exists in the U.S. or other countries. (They don't do any sort of bargaining, brokering, deal-hunting, etc. on your behalf -- just the service of double-anonymous communication, along with signaling that your interest is serious because you've paid their fee.) From my limited observations, it works pretty well.

I can't think of anything of similar quality right now.

Why do people not punish useless status-seeking behavior?

They do, whenever such behavior seems like it's bound to fail. However, when it looks like it will succeed (and thus bring high status to whoever is practicing it), an attempt to punish would mean a declaration of war against someone of high status, which is usually not a smart move.

Do they assume that swaggerers might have social clout to match their personalities and are afraid of having them as enemies?

Often yes, as explained above, but it's usually not done consciously. Most status-related behaviors are instinctive, and the conscious mind only invents rationalizations for them (which can be of many different kinds).

I don't think "parochial" is the right word here -- a more accurate term for what you're describing would be "contrarian."

In any case, insofar as there exists some coherent body of insight that can be named "Less Wrong rationality," one of its main problems is that it lacks any really useful methods for separating truth from nonsense when it comes to the output of the contemporary academia and other high-status intellectual institutions. I find this rather puzzling: on the one hand, I see people here who seem seriously interested in forming a more accurate view of the world -- but at the same time, living in a society that has vast powerful, influential, and super-high-status official intellectual institutions that deal with all imaginable topics, they show little or no interest in the question of what systematic biases and perverse incentives might be influencing their output.

Now, the point of your post seems to be that LW is good because its opinion is in line with that of these high-status institutions. (Presumably thanks to the fact that both sides have accurately converged onto the truth.) But then what exactly makes LW useful or worthwhile in any way? Are the elite universities so marginalized and powerless that they need help from a blog run by amateurs to spread the word about their output? It really seems to me that if a forum like LW is to have any point at all, it can only be in identifying correct contrarian positions. Otherwise you might as well just cut out the middleman and look at the mainstream academic output directly.

Looking again at the questions listed in this paper, I remembered a blog post by Bryan Caplan in which he proposed some skillfully thought up alternative questions that make Haidt's biases especially apparent:
http://econlog.econlib.org/archives/2010/03/do_liberals_use.html

(Here is Haidt's response, which I find rather unconvincing.)

In fact, the more I think about Haidt's questions, the more heavily biased they seem. For example, one of his "authority" questions asks for how much money you'd curse your parents in their face, and have to wait for a year to explain and apologize. Imagine if he instead asked for how much money you'd yell racial insults at a black person. Now, Haidt would presumably say that the latter falls properly under "harm," since it would be greatly emotionally hurtful to this person. But how does this same argument not apply to someone being cursed by their own child?!