buybuydandavis comments on The Logic of the Hypothesis Test: A Steel Man - Less Wrong

5 Post author: Matt_Simpson 21 February 2013 06:19AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (37)

You are viewing a single comment's thread. Show more comments above.

Comment author: buybuydandavis 26 February 2013 10:08:23PM 0 points [-]

What does considering likelihood ratios of the hypotheses in your hypothesis space do to help you out here?

The likelihood ratio was for comparing the hypotheses under consideration, the Null and the alternative. My point is that the likelihood of the alternative isn't taken into consideration at all. Prior to anything Bayesian, hypothesis testing moved from only modeling the likelihood of the null to also modeling the likelihood of a specified alternative, and comparing the two.

if the truth is outside of your hypothesis space, you're screwed no matter if you're a Bayesian or a frequentist

Therefore, you put an error placeholder of appropriate magnitude onto "it's out of my hypothesis space" so that unreasonable results have some systematic check.

And the difference between Bayesian and NHST isn't primarily how many assumptions you've committed too, which is enormous, but how many of those assumptions you've identified, and how you've specified them.