CronoDAS comments on The trouble with Bayes (draft) - Less Wrong

10 Post author: snarles 19 October 2015 08:50PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (58)

You are viewing a single comment's thread.

Comment author: CronoDAS 20 October 2015 05:43:56PM 1 point [-]

You're violating Jaynes's Infinity Commandment:

Never introduce an infinity into a probability problem except as the limit of finite processes!

Hence we need a prior over joint distributions of (X, Y). And yes, I do mean a prior distribution over probability distributions: we are saying that (X, Y) has some unknown joint distribution, which we treat as being drawn at random from a large collection of distributions. This is therefore a non-parametric Bayes approach: the term non-parametric means that the number of the parameters in the model is not finite.

Comment author: IlyaShpitser 20 October 2015 05:51:37PM *  7 points [-]

Non-parametric methods are limits of finite processes. Or, more precisely, they are rules that work for any finite data set you have. Think about using histograms to approximate a density empirically, for any dataset we have a finite number of bins, but the number of parameters depends on the size of the data. That's basically what "non-parametric" means.


Please keep your religious language out of my statistics, thank you.

Comment author: snarles 20 October 2015 06:43:57PM 2 points [-]

It is worth noting that the issue of non-consistency is just as troublesome in the finite setting. In fact, in one of Wasserman's examples he uses a finite (but large) space for X.