falenas108 comments on Calibration Test with database of 150,000+ questions - LessWrong

37 Post author: Nanashi 14 March 2015 11:22AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (31)

You are viewing a single comment's thread.

Comment author: owencb 13 March 2015 08:21:32PM 7 points [-]

Thanks for providing this!

I have a worry about using trivia questions for calibration: there's a substantial selection effect in the construction of trivia questions, so you're much more likely to get an obscure question pointing to a well-known answer than happens by chance. The effect may be to calibrate people for trivia questions in a way that transfers poorly to other questions.