= 27d567bc2d48d9f40e9ccc20ea370b15)
Nanashi comments on Calibration Test with database of 150,000+ questions - LessWrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (31)
Got it. I'll make them color coded and farther apart.
I'll write some better instructions as well.
What would help most is: "Pick an answer. How confident are you that your answer is correct?"
Then, make sure that when the user clicks the 'show answer' button, make sure that neither of the two new buttons are in the same place.
ALSO, it would be nice if the calibration curve showed the credible interval for each bin, so I can tell at a glance that my getting 1/1 right at 30% and 0/1 right at 60% isn't actually that big a hit to my calibration.
And if the second graph was stacked so that I don't have this giant red bar at 100%, which just looks odd. If it was red behind/on-top-of green, that would make the most sense (if stacked on top, you will obviously need to take the difference to maintain the sense of the graph).
Do you intend to curate out questions that are impossible/require additional clarifications like Alex would have given in advance or people would have worked out from the easy ones?