sam0345 comments on Follow-up on ESP study: "We don't publish replications" - Less Wrong

71 Post author: CarlShulman 12 July 2011 08:48PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (50)

You are viewing a single comment's thread. Show more comments above.

Comment author: Zed 11 July 2011 11:49:19PM *  9 points [-]

Academics know, or at least ought to know, that most new publications are either wrong or completely uninteresting. This is the logical side-effect from the publish-or-perish policy in academia and the increase of PhD students worldwide. Estimated 1.346 million papers per year are published in journals alone[1]. If humanity produced interesting papers at that rate scientific progress would go a lot quicker!

So if it's true that most publications are uninteresting and if it's true that most academics have to publish at a high rate in order to protect their career and send the right signals we don't want to punish and humiliate academics for publishing stupid ideas or badly executed experiments. And when you publish a paper that demonstrates the other party did a terrible job it does exactly that. The signal to noise ratio in academic journals wouldn't increase by much but suddenly academics can simply reach their paper quota by picking the ideas of other academics apart. You'd get an even more poisonous environment as a result!

In our current academic environment (or at least my part of it) most papers without a large number of citations are ignored. A paper without any citations is generally considered such a bad source that it's only one step up from wikipedia. You can cite it, if you must, but you better not base your research on it. So in practice I don't think it's a big deal that mistakes aren't corrected and that academics typically aren't expected to publicly admit that they were wrong. It's just not necessary.

Comment author: sam0345 13 July 2011 02:43:53AM 12 points [-]

A paper without any citations is generally considered such a bad source that it's only one step up from wikipedia. You can cite it, if you must, but you better not base your research on it. So in practice I don't think it's a big deal that mistakes aren't corrected and that academics typically aren't expected to publicly admit that they were wrong. It's just not necessary

Suppose the paper supposedly proves something that lots of people wish was true. Surely it is likely to get an immense number of citations.

For example,the paper supposedly proves that America always had strict gun control, or that the world is doomed unless government transfers trillions of dollars from group A to group B, by restricting the usage of evil substance X, where group A tends to have rather few academics, and group B tends to have rather a lot of academics.