Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

satt comments on Assessing Kurzweil: the results - Less Wrong

42 Post author: Stuart_Armstrong 16 January 2013 04:51PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (59)

You are viewing a single comment's thread.

Comment author: satt 15 January 2013 06:31:55PM *  1 point [-]

I've been looking forward to this. Looking at the raw data now to get an idea of the inter-rater agreement. The two columns of Youtopia ratings agree fairly well on the 33 predictions where they overlap, and the 9 LW raters seem to disagree more, but that's only my first impression. (Maybe it's just that inter-rater variation is more obvious for predictions with ≥3 ratings.) Thanks again to all the assessors for putting in the legwork.

Comment author: Stuart_Armstrong 15 January 2013 06:38:45PM *  2 points [-]

There are more than two youtopia raters. Different people, at different times, completed the assessments individually (and sometimes did "second opinions" if someone had already done that one before them). I think they were 5 assessors in total.

Comment author: satt 15 January 2013 07:13:36PM 1 point [-]

Oops. Fixed.