PhilGoetz comments on The Universal Medical Journal Article Error - Less Wrong

6 Post author: PhilGoetz 29 April 2014 05:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (189)

You are viewing a single comment's thread. Show more comments above.

Comment author: PhilGoetz 07 April 2013 03:45:31PM *  1 point [-]

The fact that the authors ignored potential heterogeneity in responses IS a problem for their analysis, but their result is still evidence against heterogeneous responses.

Why do you say that? Did you look at the data?

They found F values of 0.77, 2.161, and 1.103. That means they found different behavior in the two groups. But those F-values were lower than the thresholds they had computed assuming homogeneity. They therefore said "We have rejected the hypothesis", and claimed that the evidence, which interpreted in a Bayesian framework might support that hypothesis, refuted it.

Comment author: Matt_Simpson 07 April 2013 07:30:22PM 2 points [-]

I didn't look at the data. I was commenting on your assessment of what they did, which showed that you didn't know how the F test works. Your post made it seem as if all they did was run an F test that compared the average response of the control and treatment groups and found no difference.