= 27d567bc2d48d9f40e9ccc20ea370b15)
Decius comments on The Universal Medical Journal Article Error - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (189)
Strictly speaking, the subgroup in question only has to be one person smaller than everybody for those two statements to be compatible.
Suppose that there is no effect on 10% of the population, and a consistent effect in 90% of the population that just barely meets the p<.05 standard when measured using that subgroup. If that measurement is make using the whole population, p>.05
95% is an arbitrarily chosen number which is a rule of thumb. Very frequently you will see people doing further investigation into things where p>0.10, or if they simply feel like there was something interesting worth monitoring. This is, of course, a major cause of publication bias, but it is not unreasonable or irrational behavior.
If the effect is really so minor it is going to be extremely difficult to measure in the first place, especially if there is background noise.
It's not a rule of thumb; it's used as the primary factor in making policy decisions incorrectly. In this specific example, the regulatory agency made the statement "There is no evidence that artificial colorings are linked to hyperactivity" based on the data that artificial colorings are linked to hyperactivity with p~.13
There are many other cases in medical where 0.05p<.5 is used as evidence against p.