Will the world's elites navigate the creation of AI just fine?

lukeprog

One open question in AI risk strategy is: Can we trust the world's elite decision-makers (hereafter "elites") to navigate the creation of human-level AI (and beyond) just fine, without the kinds of special efforts that e.g. Bostrom and Yudkowsky think are needed?

Some reasons for concern include:

Otherwise smart people say unreasonable things about AI safety.
Many people who believed AI was around the corner didn't take safety very seriously.
Elites have failed to navigate many important issues wisely (2008 financial crisis, climate change, Iraq War, etc.), for a variety of reasons.
AI may arrive rather suddenly, leaving little time for preparation.

But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):

If AI is preceded by visible signals, elites are likely to take safety measures. Effective measures were taken to address asteroid risk. Large resources are devoted to mitigating climate change risks. Personal and tribal selfishness align with AI risk-reduction in a way they may not align on climate change. Availability of information is increasing over time.
AI is likely to be preceded by visible signals. Conceptual insights often take years of incremental tweaking. In vision, speech, games, compression, robotics, and other fields, performance curves are mostly smooth. "Human-level performance at X" benchmarks influence perceptions and should be more exhaustive and come more rapidly as AI approaches. Recursive self-improvement capabilities could be charted, and are likely to be AI-complete. If AI succeeds, it will likely succeed for reasons comprehensible by the AI researchers of the time.
Therefore, safety measures will likely be taken.
If safety measures are taken, then elites will navigate the creation of AI just fine. Corporate and government leaders can use simple heuristics (e.g. Nobel prizes) to access the upper end of expert opinion. AI designs with easily tailored tendency to act may be the easiest to build. The use of early AIs to solve AI safety problems creates an attractor for "safe, powerful AI." Arms races not insurmountable.

The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)

Personally, I am not very comforted by this argument because:

Elites often fail to take effective action despite plenty of warning.
I think there's a >10% chance AI will not be preceded by visible signals.
I think the elites' safety measures will likely be insufficient.

Obviously, there's a lot more for me to spell out here, and some of it may be unclear. The reason I'm posting these thoughts in such a rough state is so that MIRI can get some help on our research into this question.

In particular, I'd like to know:

Which historical events are analogous to AI risk in some important ways? Possibilities include: nuclear weapons, climate change, recombinant DNA, nanotechnology, chloroflourocarbons, asteroids, cyberterrorism, Spanish flu, the 2008 financial crisis, and large wars.
What are some good resources (e.g. books) for investigating the relevance of these analogies to AI risk (for the purposes of illuminating elites' likely response to AI risk)?
What are some good studies on elites' decision-making abilities in general?
Has the increasing availability of information in the past century noticeably improved elite decision-making?

Some reasons for concern include:

Otherwise smart people say unreasonable things about AI safety.
Many people who believed AI was around the corner didn't take safety very seriously.
Elites have failed to navigate many important issues wisely (2008 financial crisis, climate change, Iraq War, etc.), for a variety of reasons.
AI may arrive rather suddenly, leaving little time for preparation.

But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):

If AI is preceded by visible signals, elites are likely to take safety measures. Effective measures were taken to address asteroid risk. Large resources are devoted to mitigating climate change risks. Personal and tribal selfishness align with AI risk-reduction in a way they may not align on climate change. Availability of information is increasing over time.
AI is likely to be preceded by visible signals. Conceptual insights often take years of incremental tweaking. In vision, speech, games, compression, robotics, and other fields, performance curves are mostly smooth. "Human-level performance at X" benchmarks influence perceptions and should be more exhaustive and come more rapidly as AI approaches. Recursive self-improvement capabilities could be charted, and are likely to be AI-complete. If AI succeeds, it will likely succeed for reasons comprehensible by the AI researchers of the time.
Therefore, safety measures will likely be taken.
If safety measures are taken, then elites will navigate the creation of AI just fine. Corporate and government leaders can use simple heuristics (e.g. Nobel prizes) to access the upper end of expert opinion. AI designs with easily tailored tendency to act may be the easiest to build. The use of early AIs to solve AI safety problems creates an attractor for "safe, powerful AI." Arms races not insurmountable.

The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)

Personally, I am not very comforted by this argument because:

Elites often fail to take effective action despite plenty of warning.
I think there's a >10% chance AI will not be preceded by visible signals.
I think the elites' safety measures will likely be insufficient.

In particular, I'd like to know:

Which historical events are analogous to AI risk in some important ways? Possibilities include: nuclear weapons, climate change, recombinant DNA, nanotechnology, chloroflourocarbons, asteroids, cyberterrorism, Spanish flu, the 2008 financial crisis, and large wars.
What are some good resources (e.g. books) for investigating the relevance of these analogies to AI risk (for the purposes of illuminating elites' likely response to AI risk)?
What are some good studies on elites' decision-making abilities in general?
Has the increasing availability of information in the past century noticeably improved elite decision-making?

More (#1) from The Myth of the Rational Voter:

Marxist regimes — and Stalin in particular — treated biology and physics asymmetrically.

In biology, Stalin and other prominent Marxist leaders elevated the views of the quack antigeneticist Trofim Lysenko to state-supported orthodoxy, leading to the dismissal of thousands of geneticists and plant biologists. Lysenkoism hurt Soviet agriculture, and helped trigger the deadliest famine in human history during China’s Great Leap Forward.

In physics, on the other hand, leading scientists enjoyed more intellectual autonomy than any other segment of Soviet society. Internationally respected physicists ran the Soviet atomic project, not Marxist ideologues. When their rivals tried to copy Lysenko’s tactics, Stalin balked. A conference intended to start a witch hunt in Soviet physics was abruptly canceled, a decision that had to originate with Stalin. Holloway recounts a telling conversation between Beria, the political leader of the Soviet atomic project, and Kurchatov, its scientific leader: "Beria asked Kurchatov whether it was true that quantum mechanics and relativity theory were idealist, in the sense of antimaterialist. Kurchatov replied that if relativity theory and quantum mechanics were rejected, the bomb would have to be rejected too. Beria was worried by this reply, and may have asked Stalin to call off the conference."

The “Lysenkoization” of Soviet physics never came.

The best explanation for the difference is that modern physics had a practical payoff that Stalin and other Communist leaders highly valued: nuclear weapons.

And:

We encounter the price-sensitivity of irrationality whenever someone unexpectedly offers us a bet based on our professed beliefs. Suppose you insist that poverty in the Third World is sure to get worse in the next decade. A challenger immediately retorts, “Want to bet? If you’re really ‘sure,’ you won’t mind giving me ten-to-one odds.” Why are you are unlikely to accept this wager? Perhaps you never believed your own words; your statements were poetry— or lies. But it is implausible to tar all reluctance to bet with insincerity. People often believe that their assertions are true until you make them “put up or shut up.” A bet moderates their views— that is, changes their minds— whether or not they retract their words.

How does this process work? Your default is to believe what makes you feel best. But an offer to bet triggers standby rationality. Two facts then come into focus. First, being wrong endangers your net worth. Second, your belief received little scrutiny before it was adopted. Now you have to ask yourself which is worse: Financial loss in a bet, or psychological loss of self-worth? A few prefer financial loss, but most covertly rethink their views. Almost no one “bets the farm” even if — pre-wager — he felt sure.

36

Will the world's elites navigate the creation of AI just fine?

36

36

36

Will the world's elites navigate the creation of AI just fine?

36

36