JGWeissman comments on 3 Levels of Rationality Verification - Less Wrong

43 Post author: Eliezer_Yudkowsky 15 March 2009 05:19PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (182)

You are viewing a single comment's thread. Show more comments above.

Comment author: JGWeissman 01 April 2009 06:36:37AM 1 point [-]

Perhaps we can still test for this systematic optimism, while filtering for the noise I objected to, by instead of asking a "yes" or "no" question, asking for the probability that the student is in the top 50%. Treat the sum of these probabilities as the count of "yes" answers in the original version. Then a rational student should be able to account for his ignorance of other students in his answer.

Comment author: jschulter 21 January 2011 06:12:01AM 0 points [-]

This is even easier to game: assuming the school has any merit, any individual you ask should have good incentive to simply say "50%" guaranteeing a perfect score. The very first time you used the test it might be okay, but only if nobody knew that the school's reputation was at stake.