Suppose you're on a game show, and you're given a choice between three doors, One has a car, one has a goat, and one has a tiger. You pick a door, and you figure it has 1/3 chance of having the car. The host tells you to stand in front of your chosen door. He presses a button, and one of the doors you didn't choose swings open and a tiger jumps out. Should you update your previous belief that the door you chose has a 1/3 chance of having the car? Your only new evidence was that you've seen that the door you chose didn't have the tiger. But if the door had had the tiger, you wouldn't have seen that evidence, because the tiger would have killed you.
Should you update your previous belief that the door you chose has a 1/3 chance of having the car?
Yes. The door now has a 1/2 chance of having the car.
(This is assuming that the host's button means "open the door with the tiger".)
The standard view of Mutually Assured Distruction (MAD) is something like:
Occasionally people will reply with an argument like:
This is an anthropic argument, an attempt to handle the bias that comes from a link between outcomes and the number of people who can observe them. Imagine we were trying to figure out whether flipping "heads" was more likely than flipping "tails", but there was a coin demon that killed everyone if "tails" came up. Either we would see "heads" flipped, or we would see nothing at all. We're not able to sample from the "tails: everyone-dies" worlds. Even if the demon responds to tails by killing everyone only 40% of the time, we're still going to over-sample the happy-heads outcome.
Applying the anthropic principle here, however, requires that a failure of MAD really would have killed everyone. While it would have killed billions, and made major parts of the world uninhabitable, still many people would have survived. [1] How much would we have rebuilt? What would be the population now? If the cold war had gone hot and the US and USSR had fallen into wiping each other out, what would 2013 be like? Roughly, we're oversampling the no-nukes outcome by the ratio of our current population to the population there would have been in a yes-nukes outcome, and the less lopsided that ratio is the more evidence that MAD did work after all.
[1] For this wikipedia cites: The global health effects of nuclear war (1982), Long-term worldwide effects of multiple nuclear-weapons detonations (1975). Some looking online also turns up an Accelerating Future blog post. I haven't read them thoroughly, and I don't know much about the research here.
I also posted this on my blog