You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Cyan comments on The Joys of Conjugate Priors - Less Wrong Discussion

41 Post author: TCB 21 May 2011 02:41AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (24)

You are viewing a single comment's thread. Show more comments above.

Comment author: Cyan 21 May 2011 11:31:51PM *  2 points [-]

The improper alpha = beta = 0 prior, sometimes known as Haldane's prior, is derived using an invariance argument in Jaynes's 1968 paper Prior Probabilities. I actually don't trust that argument -- I find the critiques of it here compelling.

Jeffreys priors are derived from a different invariance argument; Wikipedia has a pretty good article on the subject.

I have mostly used the uniform prior myself in the past, although I think in the future I'll be using the Jeffreys prior as a default for the binomial likelihood. But the maximum entropy argument for the uniform prior is flawed: differential entropy is not an extension of discrete Shannon entropy to continuous distributions. The correct generalization is to relative entropy. Since the measure is arbitrary, the maximum entropy argument is missing an essential component.