whateverfor comments on 2014 Survey Results - Less Wrong

87 Post author: Yvain 05 January 2015 07:36PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (279)

You are viewing a single comment's thread. Show more comments above.

Comment author: whateverfor 04 January 2015 10:49:59PM 0 points [-]

OK, so all that makes sense and seems basically correct, but I don't see how you get from there to being able to map confidence for persons across a question the same way you can for questions across a person.

Adopting that terminology, I'm saying for a typical Less Wrong user, they likely have a similar understanding-the-question module. This module will be right most of the time and wrong some of the time, so they correctly apply the outside view error afterwards on each of their estimates. Since the understanding-the-question module is similar for each person, though, the actual errors aren't evenly distributed across questions, so they will underestimate on "easy" questions and overestimate on "hard" ones, if easy and hard are determined afterwards by percentage that get the answer correct.

Comment author: Vaniver 05 January 2015 12:21:49AM 0 points [-]

Since the understanding-the-question module is similar for each person, though, the actual errors aren't evenly distributed across questions, so they will underestimate on "easy" questions and overestimate on "hard" ones, if easy and hard are determined afterwards by percentage that get the answer correct.

That seems reasonable to me, yes, as an easy way for a question to be 'hard' is if most answerers interpret it differently from the questioner.