wedrifid comments on Right for the Wrong Reasons - LessWrong

14 Post author: katydee 24 January 2013 12:02AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (66)

You are viewing a single comment's thread.

Comment author: wedrifid 24 January 2013 09:59:56AM *  4 points [-]

In order to make better predictions, we must cast out those predictions that are right for the wrong reasons. While it may be tempting to award such efforts partial credit, this flies against the spirit of the truth.

Are you going to reward me for being wrong for the right reasons? If not I want to know who is skimming 'credit' off the top.

Comment author: katydee 24 January 2013 10:45:32AM 2 points [-]

Are you going to reward me for being wrong for the right reasons?

It's very difficult for humans to actually isolate such cases, but in principle yes. After all, one in twenty predictions that you make with 95% confidence should turn out to be wrong. Just as lucking into accuracy doesn't mean you're right, lucking into inaccuracy doesn't mean you're wrong.

Comment author: Decius 30 January 2013 12:37:13AM 0 points [-]

Should you be punished if your predictions with 95% confidence come true 97% of a large number of trials? If not, someone is still skimming.

Comment author: katydee 30 January 2013 02:54:48AM *  2 points [-]

Yes. Underconfidence is as much an error as overconfidence is, albeit a less common one.

Comment author: nshepperd 30 January 2013 12:39:42AM 1 point [-]

Of course. After all, that means that your predictions with 5% confidence came true in only 3% of trials.

Comment author: epigeios 07 February 2013 03:22:39AM -1 points [-]

You're the one skimming credit off the top.

My interpretation of this point is that the person doing the rewarding and punishing is the person doing the predicting.

This hints at the deeper problem, too: that the subconscious reinforcement of these predictions is causing them to continue. the most common reward is that a wrong prediction seems right. For most people, that is a reward in and of itself.

So the real question is: are you going to reward yourself for being wrong for the right reasons? how about being right for the wrong reasons?