Eliezer_Yudkowsky comments on Are calibration and rational decisions mutually exclusive? (Part two) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (15)
The classic answer is that your confidence intervals are liable to occasionally tell you that mass is a negative number, when a large error occurs. Is this interval allowing only negative masses, 90% likely to be correct? No, even if you used an experimental method that a priori was 90% likely to yield an interval covering the correct answer. In other words, using the confidence interval as the posterior probability and plugging it into the expected-utility decision function doesn't make sense. Frequentists think that ignoring this problem means it goes away.
They don't.
I don't mean the negative-answer problem. I mean "the confidence interval simply is not the posterior probability full stop" problem.
Well, sure. But whither calibration?
I already gave Cyan that classic answer, complete with a link to Jaynes, in this very comment thread. :-) But it doesn't settle the problem completely for me. It feels like finger-pointing. Yes, frequentists have lower quality answers; but why isn't the average calibration of a billion Bayesians in any way related to that 90% number that they all use?