The Mistake Script

12 jimrandomh 09 March 2009 05:35PM

Here on Less Wrong, we have hopefully developed our ability to spot mistaken arguments. Suppose you're reading an article and you encounter a fallacy. What do you do? Consider the following script:

  1. Reread the argument to determine whether it's really an error. (If not, resume reading.)
  2. Verify that the error is relevant to the point of the article. (If not, resume reading.)
  3. Decide whether the remainder of the article is worth reading despite the error. Resume reading or don't.

This script seems intuitively correct, and many people follow a close approximation of it. However, following this script is very bad, because the judgement in step (3) is tainted: you are more likely to continue reading the article if you agree with its conclusion than if you don't. If you disagreed with the article, then you were also more likely to have spotted the mistake in the first place. These two biases can cause you to unknowingly avoid reading anything you disagree with, which makes you strongly resist changing your beliefs. Long articles almost always include some bad arguments, even when their conclusion is correct. We can greatly improve this script with an explicit countermeasure:

  1. Reread the argument to determine whether it's really an error. (If not, resume reading.)
  2. Verify that the error is relevant to the point of the article. (If not, resume reading.)
  3. Decide whether you agree with the article's conclusion. If you are sure you do, stop reading. If you aren't sure what the conclusion is or aren't sure you agree with it, continue.
  4. Decide whether the remainder of the article is worth reading despite the error. Resume reading or don't.

This extra step protects us from confirmation bias and the "echo chamber" effect. We might try adding more steps, to reduce bias even further:

continue reading »