= 27d567bc2d48d9f40e9ccc20ea370b15)
Decius comments on Fake Explanations - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (81)
That would mean a large value would be added when going from "guess" to "almost guess", which would mean that it would be beneficial for a student to lie and claim to almost guess when he's really completely guessing.
Suppose the student thinks that there is a 10% chance that he is right, and the reward structure is +5/-1 for confidence interval 1.
In fact, make the reward structure:(right/wrong) 1/0, 6/-1, 10/-3, 13/-6, 15/-10, 16/-15
That puts the breakpoints at roughly even intervals, keeps the math easy, and with a little bit of clarifying exactly where the breakpoints are, doesn't reward someone who accurately determines their accuracy and then lies about it.
I sat down late last night trying to prove that this couldn't work and instead proved that it could. If I did this correctly, in order for it to work, with the confidences increasing from 0 to 1,
left side confidence <= (difference in Y)/(difference in X + difference in Y)
right side confidence >= (difference in Y)/(difference in X + difference in Y).
Differences in X are 5, 4, 3, 2, 1 and differences in Y are 1, 2, 3, 4, 5 leading to values of 1/6 through 5/6; as 0 < 1/6 < 1/5 < 2/6 < 2/5 < 3/6 < 3/5 < 4/6 < 4/5 < 5/6 < 1 this is immune to lying within a single interval (and also turns out to be so for multiple intervals).
So, what are the downsides of making this a grading standard? The biggest one I see is that it would be unfair except in classes that have as prerequisites an outstanding score in a class that covers credence calibration.