= 27d567bc2d48d9f40e9ccc20ea370b15)
Matt_Simpson comments on The Logic of the Hypothesis Test: A Steel Man - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (37)
This doesn't address the problem that the truth isn't in your hypothesis space (which is what I thought you were criticizing me for). If your model assumes constant variance, for example, when in truth there's nonconstant variance, the truth is outside your hypothesis space. You're not even considering it as a possibility. What does considering likelihood ratios of the hypotheses in your hypothesis space do to help you out here?
Reading that thread, I think jsteinart is right - if the truth is outside of your hypothesis space, you're screwed no matter if you're a Bayesian or a frequentist (which is a much more succinct way of putting my response to you). Setting up a "everything else" hypothesis doesn't really help because you can't compute a likelihood without some assumptions that, in all probability, expose you to the problem you're trying to avoid.
Are you happier if I say that Bayes is a "thick" inductive theory and that NHST can be viewed as induction with a "thin" theory which therefore keeps you from committing yourself to as much? (I do acknowledge that others treat NHST as a "thick" theory and that this difference seems like it should result in differences in the details of actually doing hypothesis tests.)
The likelihood ratio was for comparing the hypotheses under consideration, the Null and the alternative. My point is that the likelihood of the alternative isn't taken into consideration at all. Prior to anything Bayesian, hypothesis testing moved from only modeling the likelihood of the null to also modeling the likelihood of a specified alternative, and comparing the two.
Therefore, you put an error placeholder of appropriate magnitude onto "it's out of my hypothesis space" so that unreasonable results have some systematic check.
And the difference between Bayesian and NHST isn't primarily how many assumptions you've committed too, which is enormous, but how many of those assumptions you've identified, and how you've specified them.