jsteinhardt comments on (Subjective Bayesianism vs. Frequentism) VS. Formalism - Less Wrong

27 Post author: potato 26 November 2011 05:05AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (106)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 26 November 2011 04:42:24PM *  4 points [-]

Bayesian techniques, by choosing a specific prior (such as a Gaussian prior), are making an assumption that will hurt them in an extreme cases or when the data is not drawn from the prior. The tradeoff is that frequentist methods tend to be much more conservative as a result (requiring more data to come to the same conclusion).

Bayesian methods with uninformative (possibly improper) priors agree with frequentist methods whenever the latter make sense.

Comment author: jsteinhardt 27 November 2011 12:11:48AM 1 point [-]

Are you referring to the result that every non-dominated decision procedure is either a Bayesian procedure or a limit of Bayesian procedures? If so, one could imagine a frequentist procedure that is strictly dominated by other procedures, but where finding the dominating procedures is computationally infeasible. Alternately, a procedure could be non-dominated, and thus Bayesian for the right choice of prior, but the correct choice of prior could be difficult to find (the only proof I know of the "non-dominated => Bayesian" result is non-constructive).