From Richard Rhodes' The Making of the Atomic Bomb:
[Experimental physicist Francis William Aston wrote:] "If we were able to transmute [hydrogen] into [helium] nearly 1 percent of the mass would be annihilated. [Because e=mc^2, as Einstein recently proved,] the quantity of energy liberated would be prodigious. Thus to change the hydrogen in a glass of water into helium would release enough energy to drive the Queen Mary across the Atlantic and back at full speed."
Aston goes on in this lecture, delivered in 1936, to speculate about the consequences of that energy release... "There are those about us who say that such research should be stopped by law, alleging that man’s destructive powers are already large enough. So, no doubt, the more elderly and ape-like of our prehistoric ancestors objected to the innovation of cooked food and pointed out the grave dangers attending the use of the newly discovered agency, fire. Personally I think there is no doubt that sub-atomic energy is available all around us, and that one day man will release and control its almost infinite power. We cannot prevent him from doing so and can only hope that he will not use it exclusively in blowing up his next door neighbor."
More (#5) from The Making of the Atomic Bomb:
...With fifty-three people aboard including the concert violinist the Hydro sailed on time. Forty-five minutes into the crossing, Haukelid’s charge of plastic explosive blew the hull. The captain felt the explosion rather than heard it, and though Tinnsjö is landlocked he thought they might have been torpedoed. The bow swamped first as Haukelid had intended; while the passengers and crew struggled to release the lifeboats, the freight cars with their thirty-nine drums of heavy water — 162 gallons mixed with 800 g
One open question in AI risk strategy is: Can we trust the world's elite decision-makers (hereafter "elites") to navigate the creation of human-level AI (and beyond) just fine, without the kinds of special efforts that e.g. Bostrom and Yudkowsky think are needed?
Some reasons for concern include:
But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):
The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)
Personally, I am not very comforted by this argument because:
Obviously, there's a lot more for me to spell out here, and some of it may be unclear. The reason I'm posting these thoughts in such a rough state is so that MIRI can get some help on our research into this question.
In particular, I'd like to know: