More (#5) from The Making of the Atomic Bomb:
With fifty-three people aboard including the concert violinist the Hydro sailed on time. Forty-five minutes into the crossing, Haukelid’s charge of plastic explosive blew the hull. The captain felt the explosion rather than heard it, and though Tinnsjö is landlocked he thought they might have been torpedoed. The bow swamped first as Haukelid had intended; while the passengers and crew struggled to release the lifeboats, the freight cars with their thirty-nine drums of heavy water — 162 gallons mixed with 800 gallons of dross — broke loose, rolled overboard and sank like stones. Of passengers and crew twenty-six drowned. The concert violinist slipped high and dry into a lifeboat; when his violin case floated by, someone was kind enough to fish it out for him. Kurt Diebner of German Army Ordnance counted the full effect on German fission research of the Vemork bombing and the sinking of the Hydro in a postwar interview:
"When one considers that right up to the end of the war, in 1945, there was virtually no increase in our heavy-water stocks in Germany... it will be seen that it was the elimination of German heavy-water production in Norway that was the main factor in our failure to achieve a self-sustaining atomic reactor before the war ended.
The race to the bomb, such as it was, ended for Germany on a mountain lake in Norway on a cold Sunday morning in February 1944.
One open question in AI risk strategy is: Can we trust the world's elite decision-makers (hereafter "elites") to navigate the creation of human-level AI (and beyond) just fine, without the kinds of special efforts that e.g. Bostrom and Yudkowsky think are needed?
Some reasons for concern include:
But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):
The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)
Personally, I am not very comforted by this argument because:
Obviously, there's a lot more for me to spell out here, and some of it may be unclear. The reason I'm posting these thoughts in such a rough state is so that MIRI can get some help on our research into this question.
In particular, I'd like to know: