Eliezer_Yudkowsky comments on Are calibration and rational decisions mutually exclusive? (Part two) - Less Wrong

6 Post author: Cyan 24 July 2009 12:49AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (15)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 25 July 2009 03:19:07PM *  2 points [-]

The classic answer is that your confidence intervals are liable to occasionally tell you that mass is a negative number, when a large error occurs. Is this interval allowing only negative masses, 90% likely to be correct? No, even if you used an experimental method that a priori was 90% likely to yield an interval covering the correct answer. In other words, using the confidence interval as the posterior probability and plugging it into the expected-utility decision function doesn't make sense. Frequentists think that ignoring this problem means it goes away.

Comment author: Cyan 25 July 2009 05:38:08PM 2 points [-]

Frequentists think that ignoring this problem means it goes away.

They don't.

Comment author: Eliezer_Yudkowsky 26 July 2009 05:33:01AM 1 point [-]

I don't mean the negative-answer problem. I mean "the confidence interval simply is not the posterior probability full stop" problem.

Comment author: Cyan 26 July 2009 02:16:10PM 0 points [-]

Well, sure. But whither calibration?

Comment author: cousin_it 25 July 2009 03:27:37PM *  2 points [-]

I already gave Cyan that classic answer, complete with a link to Jaynes, in this very comment thread. :-) But it doesn't settle the problem completely for me. It feels like finger-pointing. Yes, frequentists have lower quality answers; but why isn't the average calibration of a billion Bayesians in any way related to that 90% number that they all use?