= 27d567bc2d48d9f40e9ccc20ea370b15)
thomblake comments on Raising the forecasting waterline (part 1) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (108)
Talking about increments of 5% runs counter to my intuitions regarding good thinking about probability estimates. For most purposes, the difference between 90% and 95% is significantly larger than the difference between 50% and 55%. Think in logs.
Yes, near the extremes it makes a difference - but we're using a Brier scoring rule, averaged over all days a forecast is open. That makes thinking in logs less important - 99% isn't much worse than 100% on errors. I'll discuss that in pt.2 under 'loss function'.
Hooray!
It depends on whether you're using probabilities epistemically or instrumentally. Changing the probability of A from 90% to 95% doesn't affect your expected utility any more than changing it from 50% to 55%.
The change in expected utility given constant decisions is the same for any 5% change in probability regardless of where the baseline is for the change. However, that "given constant decisions" criterion may be less likely to hold for a change from 90-95% than it is for a change from 50-55%. If you have to choose whether to risk a negative consequence of not-A in exchange for some benefit, for example, then it matters whether the expected negative utility of not-A just fell by a tenth or by half.
Yeah, that's why I said "For most purposes".