= 27d567bc2d48d9f40e9ccc20ea370b15)
Cyan comments on Case study: abuse of frequentist statistics - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (96)
Fair enough - in that case, how would you have produced a statistical measure of how well the distributions matched? What kind of assumptions would you employ?
I have three answers for that. The first is I wouldn't have bothered -- in this field, data collection is not super-precise, so three data points per method isn't going to tell me shit. But this really avoids the question, so...
The second answer is that Bayesian analogues to this approach do exist. But we know a priori that the two methods won't generate data from the same distribution -- there's really no need to even formulate the null hypothesis. What we really care about is the accuracy and precision of the new method, so...
The third answer is given enough data, I would have set up a hierarchical Bayesian model to estimate the accuracy and precision of the new method, where accuracy is defined as "matching the old method as closely as possible".