It is a part of difficulty to subvert - it is difficult to arrange a scheme with positive expected utility for falsifying data. At the same time there's plenty of subtle falsifications such as discarding of negative results. And when it comes to rationality - if you have a hypothesis X that is supported by arguments A,B,C,D and is debunked by arguments E,F,G,H , you can count on rational self interested agents to put more effort into finding the first four but not the last four, as payoff for former is bigger. (The real agent's reasoning costs utility, and it is expensive to find those arguments)
Consider some issue like AI risk. If you can pick out the few reasons why AI would kill everyone, even very bad reasons that rely on some oracular stuff that is not implementable, you are set for life (and you don't even have to invent them, you can pick out of fiction and simply collect them and promote together). If you can make a few equally good reasons not to, that's pure waste of your time as far as self interest is concerned. Of course science does not trust you to put equal effort when it is clearly irrational to put equal effort, for anyone but the true angels (and then for the true angels it is also rational to try to grab as much money (which would be ill spent otherwise) as they can as easily as they can, and then donate it to charities etc, so for purpose of fact finding you can't trust even the selfless angels).
It is a part of difficulty to subvert - it is difficult to arrange a scheme with positive expected utility for falsifying data.
Given that one gets fame for "spectacular" discoveries, not at all especially in fields like biology where there are frequently lots of confounding variables that you can use to provide cover.
Today's post, Science Doesn't Trust Your Rationality was originally published on 14 May 2008. A summary (taken from the LW wiki):
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Dilemma: Science or Bayes?, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.