Vaniver comments on Too good to be true - Less Wrong

24 Post author: PhilGoetz 11 July 2014 08:16PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (119)

You are viewing a single comment's thread. Show more comments above.

Comment author: byrnema 12 July 2014 11:58:32PM 1 point [-]

Another way of asking my question, perhaps more clearly, is: how do we know if the 60 considered studies were testing the hypothesis that there was a link or the hypothesis that there was not a link?

Comment author: Vaniver 13 July 2014 12:35:25AM 6 points [-]

how do we know if the 60 considered studies were testing the hypothesis that there was a link or the hypothesis that there was not a link?

I think the answer to this is "because they're using NHST." They say "we couldn't detect an effect at the level that random chance would give us 5% of the time, thus we are rather confident there is no effect." But that we don't see our 5% false positives suggests that something about the system is odd.

Comment author: byrnema 13 July 2014 12:58:47AM *  2 points [-]

OK, that sounds straightforward.

How does one know that the 60 studies are these? (rather then the others (e.g., that were designed to show an effect with 95% probability, but failed to do so and thus got a negative result)).