ChristianKl comments on Rationality Quotes April 2014 - LessWrong

8 Post author: elharo 07 April 2014 05:25PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (656)

You are viewing a single comment's thread. Show more comments above.

Comment author: raisin 01 April 2014 04:09:29PM 7 points [-]

This principle is particularly important in statistical meta-analysis: because if you have a bunch of methodologically poor studies, each with small sample size, and then subject them to meta-analysis, what can happen is that the systematic biases in each study — if they mostly point in the same direction — can reach statistical significance when the studies are pooled.

Does anyone know how often this happens in statistical meta-analysis?

Comment author: ChristianKl 03 April 2014 01:55:43PM 2 points [-]

We can't know for certain. That's the idea of systematic biases. There no way to tell if all your trials are slanted in a specific fashion, if the biases also appears in your high quality studies.

On the other hand we have fields such as homeopathy or telephathy (Ganzfeld experiments) where there are meta-analysis that treat all studies mostly equally that find that homeopathy works and telepahty exist. On the other hand you have meta-analysis who try to filter out low quality studies who come to the conclusion that homeopathy doesn't work and telepathy doesn't exist.