gwern comments on Rationality Quotes August 2011 - Less Wrong

3 Post author: dvasya 02 August 2011 08:24PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (176)

You are viewing a single comment's thread.

Comment author: gwern 05 August 2011 05:55:09PM 22 points [-]

"Lottery tickets should not be free. In such purely random and independent events as the lottery, the probability of having a winning number depends directly on the number of tickets you have purchased. When one evaluates the outcome of a scientific work, attention must be given not only to the potential interest of the ‘significant’ outcomes but also to the number of ‘lottery tickets’ the authors have ‘bought’. Those having many have a much higher chance of ‘winning a lottery prize’ than of getting a meaningful scientific result. It would be unfair not to distinguish between significant results of well-planned, powerful, sharply focused studies, and those from ‘fishing expeditions’ with a much higher probability of catching an old truck tyre than of a really big fish."

Stan Young, 28-Jul-07 www.NISS.org; quoted in "Everything is Dangerous: A Controversy", a paper discussing epidemiology's failure to use things like the Bonferroni correction which has led to things like 80% of observed correlations failing to replicate (or only 1 out of 20 NIH randomized-trials replicating the original claim).