Even if you do want to integrate contrary evidence, it can be hard to do so quickly enough to continue a normal conversation, especially if the evidence is quite unexpected.
For example, suppose I have come to believe that in war X, the victory of the Reds against the Greens was always likely to happen. That is, from the first skirmish between the two sides, one could (with only the information about the two sides available at that point in time) confidently bet on the Reds winning. And the Reds did actually win.
If I know nothing of heuristics and biases, and someone counters my assertion about war X by mentioning the Hindsight Bias, then it may take me quite a long time to integrate this new evidence into my model of the world. I will need to think about the epistemic weight of heuristics and biases information, and ask how closely the conditions of tests of bias resemble my own. If my belief about war X depends in part on evidence about what professional historians believe, then I will have to consider the potentially thorny question of how much professional historians are subject to Hindsight Bias. Of course, my subjective probability in the inevitability of Red victory will go down, but the important question of how much it goes down cannot be answered so easily.
So, what we should often do in face-to-face discussion when we get new evidence is say, "please give me some time to integrate that new evidence into my model". This would be taken by many as a concession of defeat, just as would saying "After conditionalization on your evidence I have lowered by credence in P", and so will be hard for people to do in practice.
I talked about a style of reasoning in which not a single contrary argument is allowed, with the result that every non-supporting observation has to be argued away. Here I suggest that when people encounter a contrary argument, they prevent themselves from downshifting their confidence by rehearsing already-known support.
Suppose the country of Freedonia is debating whether its neighbor, Sylvania, is responsible for a recent rash of meteor strikes on its cities. There are several pieces of evidence suggesting this: the meteors struck cities close to the Sylvanian border; there was unusual activity in the Sylvanian stock markets before the strikes; and the Sylvanian ambassador Trentino was heard muttering about “heavenly vengeance.”
Someone comes to you and says: “I don’t think Sylvania is responsible for the meteor strikes. They have trade with us of billions of dinars annually.” “Well,” you reply, “the meteors struck cities close to Sylvania, there was suspicious activity in their stock market, and their ambassador spoke of heavenly vengeance afterward.” Since these three arguments outweigh the first, you keep your belief that Sylvania is responsible—you believe rather than disbelieve, qualitatively. Clearly, the balance of evidence weighs against Sylvania.
Then another comes to you and says: “I don’t think Sylvania is responsible for the meteor strikes. Directing an asteroid strike is really hard. Sylvania doesn’t even have a space program.” You reply, “But the meteors struck cities close to Sylvania, and their investors knew it, and the ambassador came right out and admitted it!” Again, these three arguments outweigh the first (by three arguments against one argument), so you keep your belief that Sylvania is responsible.
Indeed, your convictions are strengthened. On two separate occasions now, you have evaluated the balance of evidence, and both times the balance was tilted against Sylvania by a ratio of 3 to 1.
You encounter further arguments by the pro-Sylvania traitors—again, and again, and a hundred times again—but each time the new argument is handily defeated by 3 to 1. And on every occasion, you feel yourself becoming more confident that Sylvania was indeed responsible, shifting your prior according to the felt balance of evidence.
The problem, of course, is that by rehearsing arguments you already knew, you are double-counting the evidence. This would be a grave sin even if you double-counted all the evidence. (Imagine a scientist who does an experiment with 50 subjects and fails to obtain statistically significant results, so the scientist counts all the data twice.)
But to selectively double-count only some evidence is sheer farce. I remember seeing a cartoon as a child, where a villain was dividing up loot using the following algorithm: “One for you, one for me. One for you, one-two for me. One for you, one-two-three for me.”
As I emphasized in the last essay, even if a cherished belief is true, a rationalist may sometimes need to downshift the probability while integrating all the evidence. Yes, the balance of support may still favor your cherished belief. But you still have to shift the probability down—yes, down—from whatever it was before you heard the contrary evidence. It does no good to rehearse supporting arguments, because you have already taken those into account.
And yet it does appear to me that when people are confronted by a new counterargument, they search for a justification not to downshift their confidence, and of course they find supporting arguments they already know. I have to keep constant vigilance not to do this myself! It feels as natural as parrying a sword-strike with a handy shield.
With the right kind of wrong reasoning, a handful of support—or even a single argument—can stand off an army of contradictions.