Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

GreedyAlgorithm comments on Avoiding Your Belief's Real Weak Points - Less Wrong

50 Post author: Eliezer_Yudkowsky 05 October 2007 01:59AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (205)

Sort By: Old

You are viewing a single comment's thread.

Comment author: GreedyAlgorithm 05 October 2007 07:38:00AM 1 point [-]

Ha, this just happened to me. Luckily it wasn't too painful because I knew the weakness existed, I avoided it, and then reading E. T. Jaynes' "Probability Theory: The Logic of Science" gave me a different and much better belief to patch up my old one. Also, thanks for that recommendation. A lot.

For a while I had been what I called a Bayesian because I thought the frequentist position was incoherent and the Bayesian position elegant. But I couldn't resolve to my satisfaction the problem of scale parameters. I read that there was a prior that was invariant with respect to them but something kept bothering me.

It turns out that my intuition of probability was still "there is a magic number I call probability inherent in objects and what they might do". So when I saw the question "What is the probability that a glass has water:wine in a ratio of 1.5:1 or less, given that it has water:wine in a ratio between 1:1 and 2:1?" I was still thinking something along the lines of "Well, consider all possible glasses of watered wine, and maybe weight them in some way, and I'll get a probability..."

Jaynes has convinced me that the right way to think about probability is plausibility of situations given states of knowledge. There's nothing wrong with insisting that a prior be set up for any given problem; it's incoherent to set up a problem _without_ looking at the priors. They aren't just useful, they're necessary, and anyone who says it's cheating to push the difficulty of an inductive reasoning problem onto the difficulty of determining real-world priors can be dismissed.

If only I'd asked around about this problem before, maybe I would have discovered meta-Jaynes earlier! Speaking of that, why haven't I seen his stuff or things building on it before? I feel like saying that 99% of people miss its importance says more about my importance assignment than their seeming apathy.