gwern comments on Why epidemiology will not correct itself - Less Wrong

38 Post author: gwern 11 August 2011 12:54AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (13)

You are viewing a single comment's thread. Show more comments above.

Comment author: gwern 11 August 2011 03:54:24AM *  7 points [-]

It would be nice to have what Ioannidis suggests, but what do we do in the decades (or ever) before those suggestions happen? Throwing out the correlations seems like the best idea to me - 20% of randomized trials having issues is a win in a way that 80% of results with serious issues is not.

Comment author: Andy_McKenzie 11 August 2011 04:33:46AM 3 points [-]

Certainly not all correlations are useless. This feels like I am breaking some analogue of Godwin's law, but just consider the association between cigarette smoke and some types of cancer. Generally, discounting correlations and treating them with more skepticism seem like good ideas. But "throwing out" seems needlessly harsh to me, unless for some reason you are in a hurry, in which case you should think about deferring to more expert sources anyways.

For example, this useful source http://www.informationisbeautiful.net/play/snake-oil-supplements/ (see the spreadsheet at the link) uses mostly randomized trials but also includes some studies which discuss prospective associations. I don't think the organizers should be criticized for including the correlations.

Comment author: gwern 12 August 2011 12:53:45AM 4 points [-]

This feels like I am breaking some analogue of Godwin's law, but just consider the association between cigarette smoke and some types of cancer.

It seems like everyone wants to bring up tobacco as the justification for such irresponsibility - it paid off once, so we should keep doing it... See my reply to http://news.ycombinator.com/item?id=2870962 (since they brought up tobacco before you did).