ThisSpaceAvailable comments on FiveThirtyEight (Nate Silver) rolls out new blog today, and attempts to teach people Bayes' rule. - Less Wrong

4 Post author: RomeoStevens 18 March 2014 06:37AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (10)

You are viewing a single comment's thread. Show more comments above.

Comment author: ThisSpaceAvailable 19 March 2014 12:59:34AM 0 points [-]

If you're just looking at the study, then it's quite difficult for the support ratio to be less than one. However, suppose we assume that on average, for every published study, there are 100 unpublished studies, and the one with the lowest p-value gets published. Then if a study has a p-value of .04, that particular study supports the headline. However, the fact that that study was published contradicts the headline: if the headline were true, we would expect the lowest p-value to be lower than .04.

Comment author: benkuhn 19 March 2014 05:01:50AM 0 points [-]

Yes, that's what I meant by "very rare:" there are situations where it happens, like the model that you gave, but I don't think ones that happen in real life likely to contribute a very large effect. You need really insane publication bias to get a large effect there.