= 27d567bc2d48d9f40e9ccc20ea370b15)
DanielLC comments on Quote on Nate Silver, and how to think about probabilities - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (24)
I think the bigger question is what exactly it means for the probability estimate to be wrong. The best I can figure is that it's not whatever it's called where exactly x% of the predictions that you're x% sure of are correct. In that case, Romney winning is evidence that it's more extreme than it should be, and Obama winning is evidence that it's less extreme. Whether or not it's evidence that it's wrong depends on whether you thought it was more likely to be more or less extreme beforehand.
Usually probability prediction quality are broken into calibration and discrimination.
If you buy the Bayesian argument (e.g. in Jaynes) that there is a single correct Pr(A|I) where I is your state of information and A is any proposition in question, then p, an estimate of Pr(A|I), is wrong if and only if p != Pr(A|I). In practice, we virtually never know Pr(A|I), so we can't make this check. But as far as a conceptual understanding goes, that's it - if, as I said, you buy the argument.
In practice, we check the things gwern mentioned.