Eliezer_Yudkowsky comments on 3 Levels of Rationality Verification - Less Wrong

43 Post author: Eliezer_Yudkowsky 15 March 2009 05:19PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (182)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 01 April 2009 06:02:32AM 3 points [-]

Also, this doesn't consider the possibility that students can be good rationalists, but don't interact with enough of the other students to make a good assessment of their relative strengths.

Good rationalists, taken as a group, shouldn't be systematically optimistic.

Comment author: pjeby 01 April 2009 02:54:58PM *  8 points [-]

Good rationalists, taken as a group, shouldn't be systematically optimistic.

They should be if they want to win in practice, as opposed to just getting theoretically-correct answers. See, e.g., the studies referenced in Seligman's "Learned Optimism", that show optimists consistently out-perform pessimists (i.e., realists) in a wide variety of fields and endeavors.

(Of course, Seligman's definition of optimism may be different from yours.)

Comment author: JGWeissman 01 April 2009 06:36:37AM 1 point [-]

Perhaps we can still test for this systematic optimism, while filtering for the noise I objected to, by instead of asking a "yes" or "no" question, asking for the probability that the student is in the top 50%. Treat the sum of these probabilities as the count of "yes" answers in the original version. Then a rational student should be able to account for his ignorance of other students in his answer.

Comment author: jschulter 21 January 2011 06:12:01AM 0 points [-]

This is even easier to game: assuming the school has any merit, any individual you ask should have good incentive to simply say "50%" guaranteeing a perfect score. The very first time you used the test it might be okay, but only if nobody knew that the school's reputation was at stake.