More from Command and Control:
Agnew thought that sort of lock would solve many of the custody problems at NATO. A coded switch, installed in every nuclear weapon, would block the crucial arming circuits. It would make a clear distinction between the physical possession of a weapon and the ability to use one. It would become a form of remote control. And the power to exert that control, to prohibit or allow a nuclear detonation, would remain with whoever had the code.
Agnew brought an early version of the electromechanical locking system to Washington, D.C., for a closed-door hearing of the joint committee... To unlock a nuclear weapon, a two-man custodial team would attach a cable to it from the decoder. Then they’d turn the knobs on the decoder to enter a four-digit code. It was a “split-knowledge” code— each custodian would be given only two of the four numbers. Once the correct code was entered, the switch inside the weapon would take anywhere from thirty seconds to two and a half minutes to unlock, as its little gears, cams, and cam followers whirred and spun... everyone in the hearing room agreed that it was absolutely essential for national security.
The American military, however, vehemently opposed putting any locks on nuclear weapons. The Army, the Navy, the Air Force, the Marines, the Joint Chiefs of Staff, General Power at SAC, General Norstad at NATO — all of them agreed that locks were a bad idea. The always/never dilemma lay at the heart of military’s thinking. “No single device can be expected to increase both safety and readiness,” the Joint Chiefs of Staff argued. And readiness was considered more important: the nuclear weapons in Europe were “adequately safe, within the limits of the operational requirements imposed on them.”
...After reading the joint committee’s report, President Kennedy halted the dispersal of nuclear weapons among America’s NATO allies. Studies on weapon safety and command and control were commissioned. At Sandia, the development of coded, electromechanical locks was begun on a crash basis. Known at first as “Prescribed Action Links,” the locks were given a new name, one that sounded less restrictive, in the hopes of appeasing the military. “Permissive Action Links” sounded more friendly, as did the acronym: PALs.
And:
Feuds between the Army, the Navy, and the Air Force continued, despite McNamara’s vow that the Pentagon would have “one defense policy, not three conflicting defense policies.” Interservice rivalries once again complicated the effort to develop a rational nuclear strategy. The Joint Chiefs of Staff had been instructed to alter the SIOP, so that President Kennedy would have a number of options during a nuclear war. Studies were under way to make that possible. But the nuclear ambitions of the Army, the Navy, and the Air Force still seemed incompatible— and, at times, incomprehensible.
One open question in AI risk strategy is: Can we trust the world's elite decision-makers (hereafter "elites") to navigate the creation of human-level AI (and beyond) just fine, without the kinds of special efforts that e.g. Bostrom and Yudkowsky think are needed?
Some reasons for concern include:
But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):
The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)
Personally, I am not very comforted by this argument because:
Obviously, there's a lot more for me to spell out here, and some of it may be unclear. The reason I'm posting these thoughts in such a rough state is so that MIRI can get some help on our research into this question.
In particular, I'd like to know: