ciphergoth comments on 3 Levels of Rationality Verification - Less Wrong

43 Post author: Eliezer_Yudkowsky 15 March 2009 05:19PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (182)

You are viewing a single comment's thread. Show more comments above.

Comment author: Kaj_Sotala 15 March 2009 06:00:26PM *  4 points [-]

Hmm. Some off the top of my head:

  • Look for studies that have recognized a certain bias, then use that information to come up with reasoning problems where the participants have to reach the correct answer without falling prey to the biases. Somewhat vulnerable to people studying to beat the test, though can potentially be defeated by creatively combining several different biases and applying them into new situations. Downside: coming up with lots of different scenarios where one may fall victim to biases is a lot of work. Perhaps come up with suitable computer games where success depends on avoiding biased behavior, and the scenarios can be automatically generated?
  • Calibration tests. These could be auto-generated, drawing on a far wider field of information than the current ones.
  • As the above two, but subjects are forced to write down their reasoning. This may be more helpful in making them reflect more on their reasoning, than for actual verification - somebody's train of thought can be very hard to interpet, since they'll never write down everything that influenced their decision.
Comment author: ciphergoth 15 March 2009 10:02:19PM *  1 point [-]

Somewhat vulnerable to people studying to beat the test

If the test is, say, a battery of experiments already performed that demonstrate the existence of various well-known cognitive biases, most people could not study to beat the test without improving your rationality to a significant extent if they tried.