Benja comments on Shut Up And Guess - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (105)
Ok, so you're saying the total score the student gets is
f1(q_i*) + Sum_(i /= i*) f0(q_i)? I didn't understand that from your original post, sorry.So does "(if) he score for a wrong answer was 0 (...) the only proper score function is the log" mean that if there are more than two options, log is the only proper score function that depends only on the probability assigned to the correct outcome, not on the way the rest of the probability mass is distributed among the other options? Or am I still misunderstanding?
Yes, if there are two or more options and the score function depends only on the probability assigned to the correct outcome, then the only proper function is log. You can see that with the equation I gave
f0' (x) = (k - x.f1' (x))/(1-x)
for f0 = 0, it means x.f1'(x) = -k thus f1(x) = -k ln(x) + c (necessary condition)
Then you have to check that -k ln(x) + c indeed works for some k and c, that is left as an exercise for the reader ^^