selylindi comments on The Universal Medical Journal Article Error - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (189)
Or rather, you can conclude that, if there were no effect of food dye on hyperactivity and we did this test a whole lotta times, then we'd get data like this 16% of the time, rather than beneath the 5%-of-the-time maximum cutoff you were hoping for.
It's not so easy to jump from frequentist confidence intervals to confidence for or against a hypothesis. We'd need a bunch of assumptions. I don't have access to the original article so I'll just make shit up. Specifically, if I assume that we got the 84% confidence interval from a normal distribution in which it was centrally located and two-tailed, then the corresponding minimum Bayes Factor is 0.37 for the model {mean hyperactivity = baseline} versus the model {mean hyperactivity = baseline + food dye effect}. Getting to an actual confidence level in the hypothesis requires having a prior. Since I'm too ignorant of the subject material to have an intuitive sense of the appropriate prior, I'll go with my usual here which is to charge 1 nat per parameter as a complexity penalty. And that weak complexity prior wipes out the evidence from this study.
So given these assumptions, the original article's claim...
...would be correct.