ChristianKl comments on Outside the Laboratory - Less Wrong

63 Post author: Eliezer_Yudkowsky 21 January 2007 03:46AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (336)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: ChristianKl 27 December 2012 04:23:34PM 1 point [-]

I wouldn't. Two studies opens the door to publication bias concerns and muddles the 'replication': rarely do people do a straight replication.

If you put the general significance standard at P<0.005 you will even further decrease the amount of straight replications. We need more straight replication instead of less.

A single study can wrong due to systematic bias. One researcher could engage in fraud and therefore get a P<0.005 result. He could also simply be bad at blinding his subjects properly. There are many possible ways to get a P<0.005 result by messing up the underlying science in a way that you can't see by reading a paper.

Having a second researcher reproduce the effects is vital to know that the first result is not due to some error in the experiment setup of the first study.