John_Thacker comments on Outside the Laboratory - Less Wrong

63 Post author: Eliezer_Yudkowsky 21 January 2007 03:46AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (336)

Sort By: Old

You are viewing a single comment's thread.

Comment author: John_Thacker 22 January 2007 08:20:21PM 1 point [-]

I consider myself a 'Bayesian wannabe' and my favorite author thereon is E. T. Jaynes.

Ah, well then I agree with you. However, I'm interested in how you reconcile your philosophical belief as a subjectivist when it comes to probability with the remainder of this post. Of course, as a mathematician, arguments based on the idea of rejecting arbitrary axioms are inherently less impressive than to some other scientists. After all, most of us believe in the Axiom of Choice for some reason like that the proofs needing it are too beautiful and must be true; this is despite the Banach-Tarski paradox and knowing that it is logically independent of the other axioms of Zermelo-Fraenkel set theory.

it is demonstrably too high

Hmm. I lean towards agreeing that it may be too high, but at the same time there would be problems introduced from a lower standard as well. In particular, one such silly problem is that from testing many relationships at the same time, and one then inevitably finding that (from random chance) one is "significant," another thing that many scientists are not aware of, particularly when doing demographic studies. I shudder at the idea of ridiculous demographic data dredging and multiple comparisons being even more widespread.

That said, I, being largely a Bayesian, question the entire concept of null hypotheses. If you are truly "vehemently denying that the posterior probability following an experiment should depend on whether Alice decided ahead of time to conduct 12 trials or decided to conduct trials until 3 successes were achieved," then you must logically reject the entire concept of point hypothesis testing, not merely believe that it's arbitrary or too high, and favor something like Bayes factor.

Of course, it's hard for any of us to be completely consistent in our statistical tests or even understand them all or understand all the completely arbitrary axioms that go into our reasoning.