CarlShulman comments on Follow-up on ESP study: "We don't publish replications" - Less Wrong

71 Post author: CarlShulman 12 July 2011 08:48PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (50)

You are viewing a single comment's thread. Show more comments above.

Comment author: jsalvatier 12 July 2011 06:00:05PM *  6 points [-]

What rules of thumb do you use to 'keep this in mind'? I generally try to never put anything in my brain that just has one or two studies behind it. I've been thinking of that more as 'it's easy to make a mistake in a study' and 'maybe this author has some bias that I am unaware of', but perhaps this cuts in the opposite direction.

Comment author: CarlShulman 12 July 2011 07:23:08PM 5 points [-]

Actually, even with many studies and a meta-analysis, you can still get blindsided by publication bias. There are plenty of psi meta-analyses showing positive effects (with studies that were not pre-registered, and are probably very selected), and many more in medicine and elsewhere.