AnnaSalamon comments on Ask LW: What questions to test in our rationality questionnaire? - Less Wrong

15 Post author: AnnaSalamon 29 March 2009 12:03PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (45)

You are viewing a single comment's thread. Show more comments above.

Comment author: AnnaSalamon 29 March 2009 06:24:30PM 1 point [-]

If you want to test for rationality, ask questions that require rationality to get the right answer.

Any suggestions? That's basically the idea with section D (the heuristics and biases type questions, that have correct answers) and (with more interpretive ambiguity, because it is less obvious which beliefs are correct) with section E (questions about current beliefs).

Comment author: kybernetikos 04 August 2011 07:25:11PM 1 point [-]

Set up questions that require you assume something odd in the preamble, and then conclude with something unpalatable (and quite possibly false). This tests to see if people can apply rationality even when it goes against their emotional involvement and current beliefs. As well as checking that they reach the conclusion demanded (logic), also give them an opportunity as part of a later question to flag up the premise that they feel caused the odd conclusion.

Something bayesian - like the medical test questions where the incidence in the general population is really low, but that specific one has been done so much loads of people know it. Maybe take some stats from newspaper reports and see if appropriate conclusions can be drawn.

"When was the last time you changed your mind about something you believed?" tests peoples ability to apply their rationality.