You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Pentashagon comments on Problems with Academia and the Rising Sea - Less Wrong Discussion

28 Post author: JonahSinick 23 May 2013 07:17PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (39)

You are viewing a single comment's thread. Show more comments above.

Comment author: Pentashagon 24 May 2013 11:08:14PM 0 points [-]

Yes, two people getting a P<0.05 and both setting up the experiment are better than one.

How many false-positives get published as opposed to negatives (or rarely even false-negatives)? If the ratio is too high then you'll need more than just two positive studies. If, as claimed in the quoted article, 70% of papers in a field are not reproducible that implies finding papers at random would require about nine positive studies to reach a true P<0.05, and that's only if each paper is statistically independent from the others.

If there is financial incentive to reproduce existing studies when there is a ready-made template to copy and paste into a grant-funding paper I think the overall quality of published research could decline. At least in the current model there's a financial incentive to invent novel ideas and then test them, versus just publishing a false reproduction.