Phlebas comments on Funnel plots: the study that didn't bark, or, visualizing regression to the null - Less Wrong

47 Post author: gwern 04 December 2011 11:05AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (40)

You are viewing a single comment's thread.

Comment author: [deleted] 30 November 2011 09:58:55PM *  0 points [-]

Hypothesis A: Some medical genetics researchers are unaware of the fact that gene-disease correlation studies involving only a small number of subjects or alleles are of very little value.

Hypothesis B: Some medical researchers, despite having encountered facts such as those described in gwern's article, either through dishonesty or self-delusion conduct studies involving small numbers of subjects or alleles, in order to be able to publish some apparently valuable research.

Hypothesis C: There is some other good reason for conducting studies in medical genetics using small numbers of subjects or alleles, despite the fact that the results of such studies appear to be, in many cases, so widely scattered as to be meaningless.

Comment author: gwern 30 November 2011 10:18:38PM *  0 points [-]

Have you read the linked and previous article "Why epidemiology will not correct itself"?

Comment author: [deleted] 30 November 2011 10:54:49PM 1 point [-]

I had not, and I see that you addressed my poser there.

From the abstract of a paper that you linked to:

a research finding is less likely to be true [...] when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.

This corroborates my suspicion that effect B is more important than A or C; competition for funding and prestige leads researchers to knowingly publish false positives.

Another somewhat relevant article and discussion thread is here, concerning declining standards in published research due to oversupply. I'm not sure if the writer has a good handle on the root causes of the problem though.