I don't know much about Bayesian Inference, but I am familiar with the well-posedness problem the paper seems to allude to. The authors seem to claim that in the continuous limit the inference problem is ill-posed, specifically, that the solution's behavior does not change continuously with the initial conditions. "Chaotic" is the corresponding popular meme. If true, it means that the continuous version of the Aumann's agreement theorem is unstable: a tiny difference in priors may result in a complete disagreement. Which is very interesting and has direct applications to the FAI research.
EDIT: the relevant quote (emphasis mine):
This brittleness persists beyond the discretization of continuous systems and suggests that Bayesian inference is generically ill-posed in the sense of Hadamard when applied to such systems: if closeness is defined in terms of the total variation metric or the matching of a finite system of moments, then (1) two practitioners who use arbitrarily close models and observe the same (possibly arbitrarily large amount of) data may reach diametrically opposite conclusions; and (2) any given prior and model can be slightly perturbed to achieve any desired posterior conclusions.
That can't be right. Regardless of priors, they'd agree on evidence ratios. If their priors are similar, then their evidence ratios are near unity, so their posteriors must also be similar.
I recently ran across this post, which gives a lighter discussion of a recent paper on Bayesian inference ("On the Brittleness of Bayesian Inference"). I don't understand it, but I'd like to, and it seems like the sort of paper other people here might enjoy discussing.
I am not a statistician, and this summary is based on the blog post (I haven't had time to read the paper yet) so please discount my summary accordingly: It looks like the paper focuses on the effects of priors and underlying models on the posterior distribution. Given a continuous distribution (or a discrete approximation of one) to be estimated from finite observations (of sufficiently high precision), and finite priors, the range of posterior estimates is the same as the range of the distribution to be estimated. Given models that are arbitrarily close (I'm not familiar with the total variance metric, but the impression I had was that, for finite accuracy, they produce the same observations with arbitrarily similar probability), you can have posterior estimates that are arbitrarily distant (within the range of the distribution to be estimated) given the same information. My impression is that implicitly relying on arbitrary precision of a prior can give updates that are diametrically opposed to the ones you'd get with different, but arbitrarily similar priors.
First, of course, I want to know if my summary's accurate, misses the point, or wrong.
Second, I'd be interested in hearing discussions of the paper in general and whether it might have any immediate impact on practical applications.
Some other areas of discussion that would be of interest to me: I'm also not entirely sure what 'sufficiently high precision' would be. I also have only a vague idea of the circumstances where you'd be implicitly relying on the arbitrary precision of a prior. I'm also just generally interest in hearing what people more experienced/intelligent than I am might have to say here.