jsteinhardt comments on (Subjective Bayesianism vs. Frequentism) VS. Formalism - Less Wrong

27 Post author: potato 26 November 2011 05:05AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (106)

You are viewing a single comment's thread. Show more comments above.

Comment author: jsteinhardt 29 November 2011 05:22:23AM 0 points [-]

I don't understand. Based on reading through the passages you referenced in PtLoS, maximum entropy is a way of choosing a distribution out of a family of distributions (which, by the way, is a frequentist technique, not a Bayesian one). Solomonoff induction is a choice of prior. I don't really understand in what sense these are related to each other, or in what sense Maxent generates priors at all.

Incidentally I'm surprised that there appears to be so much disagreement about this, given that LW is basically a forum brought into existence on the strength of Yudkowsky's abilities as a thinker, writer and populariser, and he clearly holds frequentism in contempt.

I've always felt that the frequentists that Eliezer argues against are straw men. As I said earlier, I've never met a frequentist who is guilty of the accusations that you keep making, although I have met Bayesians whose philosophy interfered with their ability to do good statistical modeling / inference. Have you actually run into the people who you seem to be arguing against? If not, then I think you should restrict yourself to arguing against opinions that people are actually trying to support, although I also think that whether or not some very foolish people happen to be frequentists is irrelevant to the discussion (something Eliezer himself discussed in the "Reversed Stupidity is not Intelligence" post).

Comment author: nshepperd 29 November 2011 07:25:23AM *  2 points [-]

If you know nothing about a variable except that it's in the interval [a, b] your probability distribution must be from the class of distributions where p(x) = 0 for x outside of [a, b]. You pick the distribution of maximal entropy from this class as your prior, to encode ignorance of everything except that x ∈ [a,b].

That is one way Maxent may generate a prior, anyway.

Comment author: Manfred 29 November 2011 05:47:32AM 1 point [-]

a way of choosing a distribution out of a family of distributions (which, by the way, is a frequentist technique, not a Bayesian one).

We can call dibs on things now? Ooh, I call dibs on approximating a slowly varying function as a constant!