PECOS-9 comments on Open thread, July 29-August 4, 2013 - Less Wrong

3 Post author: David_Gerard 29 July 2013 10:26PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (381)

You are viewing a single comment's thread.

Comment author: PECOS-9 01 August 2013 05:31:23PM 6 points [-]

I typed up the below message before discovering that the term I was looking for is "data dredging" or "hypothesis fishing." Still decided to post below so others know.

Is there a well-known term for the kind of error that pre-registrations of scientific studies is meant to avoid? I mean the error where an experiment is designed to test something like "This drug cures the common cold," but then when the results show no effect, the researchers repeatedly do the analysis on smaller slices of the data from the experiment, until eventually they have the results "This drug cures the common cold in males aged 40-60, p<.05," when of course that result is just due to random chance (because if you do the statistical tests on 20 subsets of the data, chances are one of them will show an effect with p<.05).

It's similar to the file drawer effect, except it's within a single experiment, not many.

Comment author: DanielLC 03 August 2013 03:28:12AM 0 points [-]