Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

roystgnr comments on Being Half-Rational About Pascal's Wager is Even Worse - Less Wrong

19 Post author: Eliezer_Yudkowsky 18 April 2013 05:20AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (168)

You are viewing a single comment's thread. Show more comments above.

Comment author: roystgnr 18 April 2013 04:43:37PM 14 points [-]

Strictly speaking, chain reactions do naturally occur, they're just so rare that we never found one until decades after we knew exactly what we were looking for, so Fermi certainly didn't have that evidence available.

Also, although I like your argument... wouldn't it apply as well to fire as it does to fission? In fact we do have a world filled with material that doesn't burn, material that oxidizes so rapidly that we never see the unoxidized chemical in nature, and material that burns only when concentrated enough to make an ignition self-sustaining. If forests and grasslands were as rare as uranium, would we have been justified in asserting that wildfires are likely impossible?

One reason why neither your argument nor my analogy turned out to be correct: even if one material is out of a narrow band of possible parameters, there are many other materials that could be in it. If our atmosphere was low-oxygen enough to make wood noncombustable, we might see more plants safely accumulating more volatile tissues instead. If other laws of physics made uranium too stable to use in technology, perhaps in that universe fermium would no longer be too unstable to survive in nature.

Comment author: orthonormal 18 April 2013 09:17:51PM 3 points [-]

Good counter-analogy, and awesome Wikipedia article. Thanks!

Comment author: Eliezer_Yudkowsky 18 April 2013 05:37:21PM 6 points [-]

Consider also the nature of the first heap: Purified uranium and a graphite moderator in such large quantities that the neutron multiplication factor was driven just over one. Elements which were less stable than uranium decayed earlier in Earth's history; elements more stable than this would not be suitable for fission. But the heap produced plutonium by its internal reactions, which could be purified chemically and then fizzed. All this was a difficult condition to obtain, but predictable that human intelligence would seek out such points in possibility-space selectively and create them - that humans would create exotic intermediate conditions not existing in nature, by which the remaining sorts of materials would fizz for the first time, and that such conditions indeed might be expected to exist, because among some of the materials not eliminated by 5 billion years, there would be some unstable enough to decay in 50 billion years, and these would be just-barely-non-fizzing and could be pushed along a little further by human intervention, with a wide space of possibilities for which elements you could try. Or to then simplify this conclusion: "Of course it wouldn't exist in nature! Those bombs went off a long time ago, we'll have to build a slightly different sort! We're not restricted to bombs that grow on trees." By such reasoning, if you had attended to it, you might have correctly agreed with Szilard, and been correctly skeptical of Fermi's hypothetical counterargument.

Not taking into account that engineering intelligence will be applied to overcome the first hypothetical difficulty is, indeed, a source of systematic directional pessimistic bias in long-term technological forecasts. Though in this case it was only a decade. I think if Fermi had said that things were 30 years off and Szilard had said 10, I would've been a tad more sympathetic toward Fermi because of the obvious larger reference class - though I would still be trying not to update my brain in the opposite direction from the training example.

Comment author: private_messaging 19 April 2013 09:43:39PM *  4 points [-]

because among some of the materials not eliminated by 5 billion years, there would be some unstable enough to decay in 50 billion years, and these would be just-barely-non-fizzing and could be pushed along a little further by human intervention

Except there aren't any that are not eliminated by, say, 10 billion years. And even 40 million years eliminate everything you can make a nuke out of except U235 . This is because besides fizzling, unstable nuclei undergo this highly asymmetric spontaneous fission known as alpha decay.