katydee comments on Right for the Wrong Reasons - LessWrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (66)
It's very difficult for humans to actually isolate such cases, but in principle yes. After all, one in twenty predictions that you make with 95% confidence should turn out to be wrong. Just as lucking into accuracy doesn't mean you're right, lucking into inaccuracy doesn't mean you're wrong.
Should you be punished if your predictions with 95% confidence come true 97% of a large number of trials? If not, someone is still skimming.
Yes. Underconfidence is as much an error as overconfidence is, albeit a less common one.
Of course. After all, that means that your predictions with 5% confidence came true in only 3% of trials.