= 27d567bc2d48d9f40e9ccc20ea370b15)
Cyan comments on The Universal Medical Journal Article Error - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (189)
disagree because not correct.
Phil's logical interpretation procedure would call shenanigans whether or not the statistical reasoning was correct.
The whole point of statistics is that it can tell us things logic cannot. If there is an important point to be made here, it needs to be made with a statistical analysis, not a logical one.
Logical analysis is a limiting case of statistical analysis, thus problems with logical reasoning have corresponding problems with statistical reasoning. I agree that Phil should have spelled out this distinction explicitly.
Their statistical analysis was correct, modulo their assumptions. They made their logical error in how they interpreted its conclusion.
People. Explain your downvotes of this comment. Do you think their statistical analysis was incorrect? Do you think they made no logical error?
This post makes a point that is both correct and important. A post that makes this point should be in Main.
The reception of this post indicates that the desired point is not coming through to the target audience. That matters.
This post makes a point that is both correct and important, but Phil has clearly lost much of the audience and is ticked off besides, and I don't blame him.
I think we've got two issues. The general issue of how one tests a null hypothesis and what it does and does not mean to reject the null, and the particular issue of food dyes. The general issue seems important, while the particular could provide a helpful illustration of the general.
But I would think that someone else, and probably multiple someone's, have already plowed this ground. Jaynes must have an article on this somewhere.
Anyone got a good article?
Depends on what you want. You could probably get something useful out of my http://lesswrong.com/lw/g13/against_nhst/ collection.
Thanks. Interesting, but it doesn't really get at the heart of the problem here, of mistaken interpretation of a "failure to reject" result as confirmation of the null hypothesis, thereby privileging the null. That just shouldn't happen, but often does.
I saw the Gigerenzer 2004 paper (you're talking about the Null Ritual paper, right?) earlier today, and it rang a few bells. Definitely liked the chart about the delusions surrounding p=0.01. Appalling that even the profs did so poorly.
GG has another 2004 paper with a similar theme: The Journal of Socio-Economics 33 (2004) 587–606 Mindless statistics http://people.umass.edu/~bioep740/yr2009/topics/Gigerenzer-jSoc-Econ-1994.pdf
Isn't that a major criticism of NHST, that almost all users and interpreters of it reverse the conditionality - a fallacy/confusion pointed by Cohen, Gigerenzer, and almost every paper I cited there?
I think that's a separate mistake. This paper shows Pr[data|H0] > 0.05. The standard mistake you refer to switches this to falsely conclude Pr[H0|data] > 0.05. However, neither of these is remotely indicative of H0 being true.
No it doesn't. It takes the word "all" as used in everyday language and pretends it is intended to be precisely the same as the logical "all" operator, which it of course it is not. It's the worst kind of nitpicking, the kind of "all people who have a heart attack should go to a licensed hospital" - "nuh-uh, not if the hospital is on fire / not if they are billionaires with a fully equipped medical team in their attic".
What on Earth is "important" about such a point?
Not even that. It takes the zero-article plural as used in everyday language and pretends it is intended to be precisely the same as the logical "all" operator, which it of course it is not.
But ... but ... Science?
I am looking at a claim in a scientific paper. The word "all" in such a claim is universally interpreted by doctors and scientists as being universally quantified. That is how other scientists interpret it when they cite a paper. That is how the FDA treats it when they deny you a drug or a medical procedure.
This is not everyday language. This is a claim to have rigorously proven something.
Even if you don't focus on the word "all", which you should, but I accept that you are ignorant of how scientific discourse works, it is still the fact that the paper did not provide ANY evidence that food dye does not affect behavior. You can fail an F-test for a hypothesis even with data that supports the hypothesis.
Not universally interpreted by doctors and scientists. I'm gonna go ahead and say that you have no idea what you're talking about and go off of what you think "all" should mean in 'all' the sciences, not what it defaults to in actual medical papers. Context!
No medical publications whatsoever can use the "all" quantifier without restricting the scope, implicitly or explicitly. Whenever you find an "all" quantifier without a restriction specified, that's at best a lazy omission or at worst an automatic error. What, a parasympathomimetic drug will slow down a subject's heart rate for all humans? Have you checked them all?
"Scientists" publishing in medicine don't get all excited (oooh an "all" quantifier) like you whenever they come across a claim that's unwisely worded using "all" without explicitly restricting the scope.
Bowing out, I'll leave you the last word if you want it.
Thanks; I was trying to write a comment that said the same thing, but failed to do so.