Followup to: Making Beliefs Pay Rent (in Anticipated Experiences)

Carl Sagan once told a parable of a man who comes to us and claims: "There is a dragon in my garage." Fascinating! We reply that we wish to see this dragon—let us set out at once for the garage! "But wait," the claimant says to us, "it is an invisible dragon."

Now as Sagan points out, this doesn't make the hypothesis unfalsifiable. Perhaps we go to the claimant's garage, and although we see no dragon, we hear heavy breathing from no visible source; footprints mysteriously appear on the ground; and instruments show that something in the garage is consuming oxygen and breathing out carbon dioxide.

But now suppose that we say to the claimant, "Okay, we'll visit the garage and see if we can hear heavy breathing," and the claimant quickly says no, it's an inaudible dragon. We propose to measure carbon dioxide in the air, and the claimant says the dragon does not breathe. We propose to toss a bag of flour into the air to see if it outlines an invisible dragon, and the claimant immediately says, "The dragon is permeable to flour."

Carl Sagan used this parable to illustrate the classic moral that poor hypotheses need to do fast footwork to avoid falsification. But I tell this parable to make a different point: The claimant must have an accurate model of the situation somewhere in his mind, because he can anticipate, in advance, exactly which experimental results he'll need to excuse.

I just finished reading a history of Enron's downfall, The Smartest Guys in the Room, which hereby wins my award for "Least Appropriate Book Title".

An unsurprising feature of Enron's slow rot and abrupt collapse was that the executive players never admitted to having made a large mistake.  When catastrophe #247 grew to such an extent that it required an actual policy change, they would say "Too bad that didn't work out—it was such a good idea—how are we going to hide the problem on our balance sheet?"  As opposed to, "It now seems obvious in retrospect that it was a mistake from the beginning."  As opposed to, "I've been stupid."  There was never a watershed moment, a moment of humbling realization, of acknowledging a fundamental problem.  After the bankruptcy, Jeff Skilling, the former COO and brief CEO of Enron, declined his own lawyers' advice to take the Fifth Amendment; he testified before Congress that Enron had been a great company.

Not every change is an improvement, but every improvement is necessarily a change.  If we only admit small local errors, we will only make small local changes.  The motivation for a big change comes from acknowledging a big mistake.

What is true is already so.
Owning up to it doesn't make it worse.
Not being open about it doesn't make it go away.
And because it's true, it is what is there to be interacted with.
Anything untrue isn't there to be lived.
People can stand what is true,
for they are already enduring it.
—Eugene Gendlin

(Hat tip to Stephen Omohundro.)

Part of the Letting Go subsequence of How To Actually Change Your Mind

Next post: "The Meditation on Curiosity"

Previous post: "The Proper Use of Doubt"

Followup to:  Absence of Evidence Is Evidence of Absence.

Friedrich Spee von Langenfeld, a priest who heard the confessions of condemned witches, wrote in 1631 the Cautio Criminalis ('prudence in criminal cases') in which he bitingly described the decision tree for condemning accused witches:  If the witch had led an evil and improper life, she was guilty; if she had led a good and proper life, this too was a proof, for witches dissemble and try to appear especially virtuous. After the woman was put in prison: if she was afraid, this proved her guilt; if she was not afraid, this proved her guilt, for witches characteristically pretend innocence and wear a bold front. Or on hearing of a denunciation of witchcraft against her, she might seek flight or remain; if she ran, that proved her guilt; if she remained, the devil had detained her so she could not get away.

Spee acted as confessor to many witches; he was thus in a position to observe every branch of the accusation tree, that no matter what the accused witch said or did, it was held a proof against her.  In any individual case, you would only hear one branch of the dilemma.  It is for this reason that scientists write down their experimental predictions in advance.

But you can't have it both ways—as a matter of probability theory, not mere fairness.  The rule that "absence of evidence is evidence of absence" is a special case of a more general law, which I would name Conservation of Expected Evidence:  The expectation of the posterior probability, after viewing the evidence, must equal the prior probability.

P(H) = P(H)
P(H) = P(H,E) + P(H,~E)
P(H) = P(H|E)*P(E) + P(H|~E)*P(~E)

Therefore, for every expectation of evidence, there is an equal and opposite expectation of counterevidence.

Politics is the mind-killer.  Debate is war, arguments are soldiers.  There is the temptation to search for ways to interpret every possible experimental result to confirm your theory, like securing a citadel against every possible line of attack.  This you cannot do.  It is mathematically impossible. For every expectation of evidence, there is an equal and opposite expectation of counterevidence.

But it's okay if your cherished belief isn't perfectly defended.  If the hypothesis is that the coin comes up heads 95% of the time, then one time in twenty you will see what looks like contrary evidence.  This is okay.  It's normal.  It's even expected, so long as you've got nineteen supporting observations for every contrary one.  A probabilistic model can take a hit or two, and still survive, so long as the hits don't keep on coming in.

Yet it is widely believed, especially in the court of public opinion, that a true theory can have no failures and a false theory no successes.

Followup to:  Update Yourself Incrementally

Yesterday I talked about a style of reasoning in which not a single contrary argument is allowed, with the result that every non-supporting observation has to be argued away.  Today I suggest that when people encounter a contrary argument, they prevent themselves from downshifting their confidence by rehearsing already-known support.

Suppose the country of Freedonia is debating whether its neighbor, Sylvania, is responsible for a recent rash of meteor strikes on its cities.  There are several pieces of evidence suggesting this: the meteors struck cities close to the Sylvanian border; there was unusual activity in the Sylvanian stock markets before the strikes; and the Sylvanian ambassador Trentino was heard muttering about "heavenly vengeance".

Someone comes to you and says:  "I don't think Sylvania is responsible for the meteor strikes.  They have trade with us of billions of dinars annually."  "Well," you reply, "the meteors struck cities close to Sylvania, there was suspicious activity in their stock market, and their ambassador spoke of heavenly vengeance afterward."  Since these three arguments outweigh the first, you keep your belief that Sylvania is responsible—you believe rather than disbelieve, qualitatively. Clearly, the balance of evidence weighs against Sylvania.

Then another comes to you and says:  "I don't think Sylvania is responsible for the meteor strikes.  Directing an asteroid strike is really hard. Sylvania doesn't even have a space program."  You reply, "But the meteors struck cities close to Sylvania, and their investors knew it, and the ambassador came right out and admitted it!"  Again, these three arguments outweigh the first (by three arguments against one argument), so you keep your belief that Sylvania is responsible.

Indeed, your convictions are strengthened.  On two separate occasions now, you have evaluated the balance of evidence, and both times the balance was tilted against Sylvania by a ratio of 3-to-1.

When I was very young—I think thirteen or maybe fourteen—I thought I had found a disproof of Cantor's Diagonal Argument, a famous theorem which demonstrates that the real numbers outnumber the rational numbers.  Ah, the dreams of fame and glory that danced in my head!

My idea was that since each whole number can be decomposed into a bag of powers of 2, it was possible to map the whole numbers onto the set of subsets of whole numbers simply by writing out the binary expansion.  13, for example, 1101, would map onto {0, 2, 3}.  It took a whole week before it occurred to me that perhaps I should apply Cantor's Diagonal Argument to my clever construction, and of course it found a counterexample—the binary number ...1111, which does not correspond to any finite whole number.

So I found this counterexample, and saw that my attempted disproof was false, along with my dreams of fame and glory.

I was initially a bit disappointed.

I recently ran across this interesting article about Radical Honesty, a movement founded by a psychotherapist named Brad Blanton who suggests that we should kick our addiction to lying and just tell the complete truth all the time.  I also like this quote from the Wikipedia article on Radical Honesty:  "The significant majority of participants in the Radical Honesty workshops report dramatic changes in their lives after taking the course, though they are not always comfortable and positive."  The movement visibly suffers from having been founded by a psychotherapist - it's more about the amazing happiness that absolute truth-telling can bring to your relationships (!!) rather than such rationalist values as seeking truth by teaching yourself a habit of honesty, or not wishing to deceive others because it infringes on their autonomy.

I once suggested a notion called "Crocker's Rules", which was the mirror image of Radical Honesty - rather than telling the whole truth to other people, you would strive to always allow others to tell you the complete truth without being offended.

An oblong slip of newspaper had appeared between O'Brien's fingers. For perhaps five seconds it was within the angle of Winston's vision. It was a photograph, and there was no question of its identity. It was the photograph. It was another copy of the photograph of Jones, Aaronson, and Rutherford at the party function in New York, which he had chanced upon eleven years ago and promptly destroyed. For only an instant it was before his eyes, then it was out of sight again. But he had seen it, unquestionably he had seen it! He made a desperate, agonizing effort to wrench the top half of his body free. It was impossible to move so much as a centimetre in any direction. For the moment he had even forgotten the dial. All he wanted was to hold the photograph in his fingers again, or at least to see it.

'It exists!' he cried.

'No,' said O'Brien.

He stepped across the room.

There was a memory hole in the opposite wall. O'Brien lifted the grating. Unseen, the frail slip of paper was whirling away on the current of warm air; it was vanishing in a flash of flame. O'Brien turned away from the wall.

'Ashes,' he said. 'Not even identifiable ashes. Dust. It does not exist. It never existed.'

'But it did exist! It does exist! It exists in memory. I remember it. You remember it.'

'I do not remember it,' said O'Brien.

Winston's heart sank. That was doublethink. He had a feeling of deadly helplessness. If he could have been certain that O'Brien was lying, it would not have seemed to matter. But it was perfectly possible that O'Brien had really forgotten the photograph. And if so, then already he would have forgotten his denial of remembering it, and forgotten the act of forgetting. How could one be sure that it was simple trickery? Perhaps that lunatic dislocation in the mind could really happen: that was the thought that defeated him.

   —George Orwell, 1984

What if self-deception helps us be happy?  What if just running out and overcoming bias will make us—gasp!—unhappy?  Surely, true wisdom would be second-order rationality, choosing when to be rational.  That way you can decide which cognitive biases should govern you, to maximize your happiness.

Leaving the morality aside, I doubt such a lunatic dislocation in the mind could really happen.

Followup to:  Conjunction Fallacy

 "Merely corroborative detail, intended to give artistic verisimilitude to an otherwise bald and unconvincing narrative..."
            -- Pooh-Bah, in Gilbert and Sullivan's The Mikado

The conjunction fallacy is when humans rate the probability P(A&B) higher than the probability P(B), even though it is a theorem that P(A&B) <= P(B).  For example, in one experiment in 1981, 68% of the subjects ranked it more likely that "Reagan will provide federal support for unwed mothers and cut federal support to local governments" than that "Reagan will provide federal support for unwed mothers."

A long series of cleverly designed experiments, which weeded out alternative hypotheses and nailed down the standard interpretation, confirmed that conjunction fallacy occurs because we "substitute judgment of representativeness for judgment of probability".  By adding extra details, you can make an outcome seem more characteristic of the process that generates it.  You can make it sound more plausible that Reagan will support unwed mothers, by adding the claim that Reagan will also cut support to local governments.  The implausibility of one claim is compensated by the plausibility of the other; they "average out".

Which is to say:  Adding detail can make a scenario SOUND MORE PLAUSIBLE, even though the event necessarily BECOMES LESS PROBABLE.

If so, then, hypothetically speaking, we might find futurists spinning unconscionably plausible and detailed future histories, or find people swallowing huge packages of unsupported claims bundled with a few strong-sounding assertions at the center.