HungryHobo comments on Rationality Quotes April 2014 - LessWrong

8 Post author: elharo 07 April 2014 05:25PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (656)

You are viewing a single comment's thread. Show more comments above.

Comment author: JQuinton 01 April 2014 01:00:19PM 27 points [-]

Now, one basic principle in all of science is GIGO: garbage in, garbage out. This principle is particularly important in statistical meta-analysis: because if you have a bunch of methodologically poor studies, each with small sample size, and then subject them to meta-analysis, what can happen is that the systematic biases in each study — if they mostly point in the same direction — can reach statistical significance when the studies are pooled. And this possibility is particularly relevant here, because meta-analyses of homeopathy invariably find an inverse correlation between the methodological quality of the study and the observed effectiveness of homeopathy: that is, the sloppiest studies find the strongest evidence in favor of homeopathy. When one restricts attention only to methodologically sound studies — those that include adequate randomization and double-blinding, predefined outcome measures, and clear accounting for drop-outs — the meta-analyses find no statistically significant effect (whether positive or negative) of homeopathy compared to placebo.

Comment author: HungryHobo 02 April 2014 12:52:10PM 11 points [-]

A bigger danger is publication bias. collect 10 well run trials without knowing that 20 similar well run ones exist but weren't published because their findings weren't convenient and your meta-analysis ends up distorted from the outset.