Confessions of the Father of the Neutron Bomb, Sam Cohen:
The one idea dearest to Teller’s heart was the H-bomb. He and a couple of his cronies applied themselves to devising various schemes on designing such a weapon. All of them turned out to be impractical and most of them unworkable. Which never slowed him down in the slightest for reasons we’ll never know nor will he...One day, Teller announced he would be giving a colloquium on his work. Since this was a pretty fascinating subject that held the potential for providing a bang a hundred or even a thousand times bigger than the A-bomb, he spoke to a pretty full house, that included me.
...The briefing was over, there were lots of questions that Teller handled with aplomb, knowing far more about the subject than most in the audience. Finally there came a question that had nothing to do with whether or not his [h-]bomb was feasible. Instead, it was whether it would destroy the world by causing uncontrollable thermonuclear reactions in the earth’s atmosphere that would cause it to burn up, plus you and me and everyone else. Teller was on his game, as he always was, and replied that he had estimated this terrible possibility and we were quite safe — by about a factor of ten. Now there aren’t too many people who rest comfortably with assurances that mankind’s fate, let alone their own, is on the safe side by a factor of ten, although there are millions of smokers, including myself, drug addicts, sex addicts, who take much greater chances than that with their lives, and they know it. Except that they know it won’t happen to them, which is why many soldiers, at least as stupid as they’re brave, get Congressional Medals of Honor.
Naturally, the next question was: “How accurate is the nuclear data you used in your calculation?” As only Edward Teller is capable of doing, he replied, with a smirk-smile going from ear to ear: “Well, it’s possible that the data might be off by a factor of ten.” Which way, he didn’t profess to know, but I suspect that much of the audience didn’t sleep too well that night. (If you’re beginning to worry that you might not be sleeping too well from now on, I have to tell you that as it turned out, after careful nuclear measurements and detailed calculations were made, we were safe by far more than a factor of ten. It simply couldn’t happen.)
Cohen then goes on to discuss in the next section how egregiously biased & unrealistic was the study RAND performed for the Air Force in order to justify the military utility of h-bombs and hence their development. After mentioning that the nuclear stockpile now reflects Oppenheimer's strategy of smaller warheads, he concludes:
There’s a profound lesson to be learned about the great H-bomb debate; that it was a farce. I’m sure (or am I?) that many scholars now understand how farcical it was, but you’d never know it from their writings. However, whether generals, admirals, congressmen, and Presidents have grasped this, I wonder. In fact, I don’t wonder too much, for as I observe the passing scene with arguments on the Stealth bomber, the MX missile, and all that nuclear business that is supposed to mean so much for our survival, I don’t think they’ve learned a thing. They still carry on in the same way, fighting over expensive nuclear weapon systems they don’t understand, to be used in a war we don’t know how to fight.
LA-602: Ignition of the Atmosphere with Nuclear Bombs, a research report from the Manhattan Project, is to the best of my knowledge the first technical analysis ever conducted of an uncertain danger of a human-caused extinction catastrophe.
Previously, Teller and Konopinski had been assigned the task of disproving a crazy suggestion by Enrico Fermi that a fission chain reaction could ignite a thermonuclear reaction in deuterium - what we now know as an H-Bomb. Teller and Konopinski found that, contrary to their initial skepticism, the hydrogen bomb appeared possible.
Good for their rationality! Even though they started with the wrong conclusion on their bottom line, they were successfully forced away from it by arguments that could only support one answer.
Still, in retrospect, I think that the advice the future would give to the past, would be: Start by sitting down and saying, "We don't know if a hydrogen bomb is possible". Then list out the evidence and arguments; then at the end weigh it.
So the hydrogen bomb was possible. Teller then suggested that a hydrogen bomb might ignite a self-sustaining thermonuclear reaction in the nitrogen of Earth's atmosphere. This also appeared extremely unlikely at a first glance, but Teller and Konopinski and Marvin investigated, and wrote LA-602...
As I understand LA-602, the authors went through the math and concluded that there were several strong reasons to believe that nitrogen fusion could not be self-sustaining in the atmosphere: it would take huge energies to start the reaction at all; the reaction would lose radiation from its surface too fast to sustain the fusion temperature; and even if the fusion reaction did grow, the Compton effect would increase radiation losses with volume(?).
And we're still here; so the math, whatever it actually says, seems to have been right.
Note that the Manhattan scientists didn't always get their math right. The Castle Bravo nuclear test on March 1, 1954 produced 15 megatons instead of the expected 4-8 megatons due to an unconsidered additional nuclear reaction that took place in lithium-7. The resulting fallout contaminated fishing boats outside the declared danger zone; at least one person seems to have died.
But the LA-602 calculations were done with very conservative assumptions, and came out with plenty of safety margin. AFAICT (I am not a physicist) a Castle Bravo type oversight could not realistically have made the atmosphere ignite anyway, and if it did, it'd have gone right out, etc.
The last time I know of when a basic physical calculation with that much safety margin, and multiple angles of argument, turned out to be wrong anyway, was when Lord Kelvin showed from multiple angles of reasoning that the Earth could not possibly be so much as a hundred million years old.
LA-602 concludes:
Decades after LA-602, another paper would be written to analyze an uncertain danger of human-created existential risk: The Review of Speculative "Disaster Scenarios" at RHIC.
The RHIC Review was written in response to suggestions that the Relativistic Heavy Ion Collider might create micro black holes or strangelets.
A B.Sc. thesis by Shameer Shah of MIT, Perception of Risk: Disaster Scenarios at Brookhaven, chronicles the story behind the RHIC Review:
The RHIC flap began when Walter Wagner wrote to Scientific American, speculating that the Brookhaven collider might create a "mini black hole". A reply letter by Frank Wilczek of the Institute for Advanced Studies labeled the mini-black-hole scenario as impossible, but also introduced a new possibility, negatively charged strangelets, which would convert normal matter into more strange matter. Wilczek considered this possibility slightly more plausible.
Then the media picked up the story.
Shameer Shah interviewed (on Nov 22, 2002) Robert Jaffe, Director of MIT's Center for Theoretical Physics, a pioneer in the theory of strange matter, and primary author of the RHIC Review.
According to Jaffe, even before the investigative committe was convened, "No scientist who understood the physics thought that this experiment posed the slightest threat to anybody." Then why have the committee in the first place? "It was an attempt to take seriously the fears of science that they don't understand." Wilczek was asked to serve on the committee "to pay the wages of his sin, since he's the one that started all this with his letter."
Between LA-602 and the RHIC Review there is quite a difference of presentation.
I mean, just look at the names:
See a difference?
LA-602 began life as a classified report, written by scientists for scientists. You're assumed to be familiar with the meaning of terms like Bremsstrahlung, which I had to look up. LA-602 does not begin by asserting any conclusions; the report walks through the calculations - at several points clearly labeling theoretical extrapolations and unexplored possibilities as such - and finally concludes that radiation losses make self-sustaining nitrogen fusion impossible-according-to-the-math, even under the most conservative assumptions.
The RHIC Review presents a nontechnical summary of its conclusions in six pages at the start, relegating the math and physics to eighteen pages of appendices.
LA-602 concluded, "There remains the distinct possibility that some other less simple mode of burning may maintain itself in the atmosphere..."
The RHIC Review concludes: "Our conclusion is that the candidate mechanisms for catastrophic scenarios at RHIC are firmly excluded by existing empirical evidence, compelling theoretical arguments, or both. Accordingly, we see no reason to delay the commissioning of RHIC on their account."
It is not obvious to my inexpert eyes that the assumptions in the RHIC Review are any more firm than those in LA-602 - they both seem very firm - but the two papers arise from rather different causes.
To put it bluntly, LA-602 was written by people curiously investigating whether a hydrogen bomb could ignite the atmosphere, and the RHIC Review is a work of public relations.
Now, it does seem - so far as I can tell - that it's pretty damned unlikely for a particle accelerator far less powerful than random cosmic rays to destroy Earth and/or the Universe.
But I don't feel any more certain of that after reading the RHIC Review than before I read it. I am not a physicist; but if I was a physicist, and I read a predigested paper like the RHIC Review instead of doing the whole analysis myself from scratch, I would be fundamentally trusting the rationality of the paper's authors. Even if I checked the math, I would still be trusting that the equations I saw were the right equations to check. I would be trusting that someone sat down, and looked for unpleasant contrary arguments with an open mind, and really honestly didn't find anything.
When I contrast LA-602 to the RHIC Review, well...
Don't get me wrong: I don't feel the smallest particle of real fear about particle accelerators. The basic cosmic-ray argument seems pretty convincing. Nature seems to have arranged for the calculations in this case to have some pretty large error margins. I, myself, am not going to worry about risks we can actually calculate to be tiny, when there are incalculable large-looking existential risks to soak up my concern.
But there is something else that I do worry about: The primary stake on the table with things like RHIC, is that it is going to get scientists into the habit of treating existential risk as a public relations issue, where ignorant technophobes say the risk exists, and the job of scientists is to interface with the public and explain to them that it does not.
Everyone knew, before the RHIC report was written, what answer it was supposed to produce. That is a very grave matter. Analysis is what you get when physicists sit down together and say, "Let us be curious," and walk through all the arguments they can think of, recording them as they go, and finally weigh them up and reach a conclusion. If this does not happen, no analysis has taken place.
The general rule of thumb I sometimes use, is that - because the expected utility of thought arises from the utility of what is being reasoned about - a single error in analyzing an existential risk, even if it "doesn't seem like it ought to change the conclusion", is worth at least one human life.
The RHIC Review is not written to the standard of care that would be appropriate if, after the RHIC Review was written, some higher authority went through the paper; and if a single argument in it was wrong, anywhere, whether or not it changed the conclusion, a hostage got shot. That's how to think about analyzing existential risks. That way, for each and every element of the analysis, you can find it in yourself to be a little uncertain about that element, even if it doesn't seem "like it could possibly change the conclusion"; uncertainty invokes curiosity.
The RHIC Review was produced by authors who were already sure that the RHIC couldn't destroy the Earth, the problem-at-hand was explaining this to the public. If the authors decided just by eyeballing the problem that the RHIC couldn't destroy the Earth, then the only actual analysis that took place was conducted in 5 seconds. Yes, it's a lopsided issue, but it seems that as a general matter of policy, any existential risk at all deserves a longer and truly curious analysis than that.
Though I don't really blame the RHIC Review's authors. No one ever told them that there was such a thing as existential risk, or that it raised the standards of analysis beyond what was usual in a scientific paper, or that rational analysis requires placing yourself into a state of genuine uncertainty about each individual element's exact value...
And the much greater reason I don't blame them, is that between the 1940s and today, society has developed a "Gotcha!" attitude toward risk.
You can't admit a single particle of uncertain danger if you want your science's funding to survive. These days you are not allowed to end by saying, "There remains the distinct possibility..." Because there is no debate you can have about tradeoffs between scientific progress and risk. If you get to the point where you're having a debate about tradeoffs, you've lost the debate. That's how the world stands, nowadays.
So no one can do serious analysis of existential risks anymore, because just by asking the question, you're threatening the funding of your whole field.
The number one lesson I take from this whole issue is that where human-caused uncertain existential dangers are concerned, the only way to get a real, serious, rational, fair, evenhanded assessment of the risks, in our modern environment,
Is if the whole project is classified, the paper is written for scientists without translation, and the public won't get to see the report for another fifty years.
This is the lesson of LA-602: Ignition of the Atmosphere with Nuclear Bombs and the Review of Speculative "Disaster Scenarios" at RHIC. Read them and weep.