PhilGoetz comments on The Universal Medical Journal Article Error - Less Wrong

6 Post author: PhilGoetz 29 April 2014 05:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (189)

You are viewing a single comment's thread. Show more comments above.

Comment author: PhilGoetz 13 April 2013 03:06:33AM *  0 points [-]

Look, what you've written above is based on misunderstanding how an F-test works. I've already explained repeatedly why what you're saying here, which is the same thing you've said each time before, is not correct.

This study contains a failure of an F-test. Because of how the F-test is structured, failure of an F-test to prove forall X P(X), is not inductive evidence, nor evidence of any kind at all, that P(X) is false for most X.

I will try to be more polite, but you need to a) read the study, and b) learn how an F-test works, before you can talk about this. But I just don't understand why you keep making confident assertions about a study you haven't read, using a test you don't understand.

The F-test is especially tricky, because you know you're going to find some difference between the groups. What difference D would you expect to find if there is in fact no effect? That's a really hard question, and the F-test dodges it by using the arbitrary but standard 95% confidence interval to pick a higher threshold, F. Results between D and F would still support the hypothesis that there is an effect, while results below D would be evidence against that hypothesis. Not knowing what D is, we can't say whether failure of an F-test is evidence for or against a hypothesis.

Comment author: lfghjkl 13 April 2013 04:43:10PM *  1 point [-]

This study contains a failure of an F-test.

And I've repeatedly told you that you should've focused your critique on this instead of ranting about deduction. The last time I said it, you claimed the following:

There is nothing statistically wrong with the paper.

Now to answer your question:

But I just don't understand why you keep making confident assertions about a study you haven't read,

I haven't been discussing this study, I've been trying to help you understand why your critique of it has been misguided.

using a test you don't understand.

As for this claim you undoubtedly have an interesting "proof" for, I've simply avoided confusing you further with a discussion of statistics until you realized the following:

  1. All statistical conclusions are deductively wrong.

  2. A statistical study must be critiqued for it's misuse of statistics (and obviously, then you must first claim that there is something statistically wrong with the paper).