Will the world's elites navigate the creation of AI just fine?

lukeprog

One open question in AI risk strategy is: Can we trust the world's elite decision-makers (hereafter "elites") to navigate the creation of human-level AI (and beyond) just fine, without the kinds of special efforts that e.g. Bostrom and Yudkowsky think are needed?

Some reasons for concern include:

Otherwise smart people say unreasonable things about AI safety.
Many people who believed AI was around the corner didn't take safety very seriously.
Elites have failed to navigate many important issues wisely (2008 financial crisis, climate change, Iraq War, etc.), for a variety of reasons.
AI may arrive rather suddenly, leaving little time for preparation.

But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):

If AI is preceded by visible signals, elites are likely to take safety measures. Effective measures were taken to address asteroid risk. Large resources are devoted to mitigating climate change risks. Personal and tribal selfishness align with AI risk-reduction in a way they may not align on climate change. Availability of information is increasing over time.
AI is likely to be preceded by visible signals. Conceptual insights often take years of incremental tweaking. In vision, speech, games, compression, robotics, and other fields, performance curves are mostly smooth. "Human-level performance at X" benchmarks influence perceptions and should be more exhaustive and come more rapidly as AI approaches. Recursive self-improvement capabilities could be charted, and are likely to be AI-complete. If AI succeeds, it will likely succeed for reasons comprehensible by the AI researchers of the time.
Therefore, safety measures will likely be taken.
If safety measures are taken, then elites will navigate the creation of AI just fine. Corporate and government leaders can use simple heuristics (e.g. Nobel prizes) to access the upper end of expert opinion. AI designs with easily tailored tendency to act may be the easiest to build. The use of early AIs to solve AI safety problems creates an attractor for "safe, powerful AI." Arms races not insurmountable.

The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)

Personally, I am not very comforted by this argument because:

Elites often fail to take effective action despite plenty of warning.
I think there's a >10% chance AI will not be preceded by visible signals.
I think the elites' safety measures will likely be insufficient.

Obviously, there's a lot more for me to spell out here, and some of it may be unclear. The reason I'm posting these thoughts in such a rough state is so that MIRI can get some help on our research into this question.

In particular, I'd like to know:

Which historical events are analogous to AI risk in some important ways? Possibilities include: nuclear weapons, climate change, recombinant DNA, nanotechnology, chloroflourocarbons, asteroids, cyberterrorism, Spanish flu, the 2008 financial crisis, and large wars.
What are some good resources (e.g. books) for investigating the relevance of these analogies to AI risk (for the purposes of illuminating elites' likely response to AI risk)?
What are some good studies on elites' decision-making abilities in general?
Has the increasing availability of information in the past century noticeably improved elite decision-making?

Some reasons for concern include:

Otherwise smart people say unreasonable things about AI safety.
Many people who believed AI was around the corner didn't take safety very seriously.
Elites have failed to navigate many important issues wisely (2008 financial crisis, climate change, Iraq War, etc.), for a variety of reasons.
AI may arrive rather suddenly, leaving little time for preparation.

But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):

If AI is preceded by visible signals, elites are likely to take safety measures. Effective measures were taken to address asteroid risk. Large resources are devoted to mitigating climate change risks. Personal and tribal selfishness align with AI risk-reduction in a way they may not align on climate change. Availability of information is increasing over time.
AI is likely to be preceded by visible signals. Conceptual insights often take years of incremental tweaking. In vision, speech, games, compression, robotics, and other fields, performance curves are mostly smooth. "Human-level performance at X" benchmarks influence perceptions and should be more exhaustive and come more rapidly as AI approaches. Recursive self-improvement capabilities could be charted, and are likely to be AI-complete. If AI succeeds, it will likely succeed for reasons comprehensible by the AI researchers of the time.
Therefore, safety measures will likely be taken.
If safety measures are taken, then elites will navigate the creation of AI just fine. Corporate and government leaders can use simple heuristics (e.g. Nobel prizes) to access the upper end of expert opinion. AI designs with easily tailored tendency to act may be the easiest to build. The use of early AIs to solve AI safety problems creates an attractor for "safe, powerful AI." Arms races not insurmountable.

The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)

Personally, I am not very comforted by this argument because:

Elites often fail to take effective action despite plenty of warning.
I think there's a >10% chance AI will not be preceded by visible signals.
I think the elites' safety measures will likely be insufficient.

In particular, I'd like to know:

Which historical events are analogous to AI risk in some important ways? Possibilities include: nuclear weapons, climate change, recombinant DNA, nanotechnology, chloroflourocarbons, asteroids, cyberterrorism, Spanish flu, the 2008 financial crisis, and large wars.
What are some good resources (e.g. books) for investigating the relevance of these analogies to AI risk (for the purposes of illuminating elites' likely response to AI risk)?
What are some good studies on elites' decision-making abilities in general?
Has the increasing availability of information in the past century noticeably improved elite decision-making?

More (#1) from Super Crunchers:

...the Office of Education and the Office of Economic Opportunity sought to determine what types of education models could best break this cycle of failure. The result was Project Follow Through, an ambitious effort that studied 79,000 children in 180 low-income communities for twenty years at a price tag of more than $600 million... At the time it was the largest education study ever done. Project Follow Through looked at the impact of seventeen different teaching methods, ranging from models like DI [direct instruction], where lesson plans are carefully scripted, to unstructured models where students themselves direct their learning by selecting what and how they will study... Project Follow Through's designers wanted to know which model performed the best, not only in developing skills in its area of emphasis, but also across the board.

Direct Instruction won hands down. Education writer Richard Nadler summed it up this way: "When the testing was over, students in DI classrooms had placed first in reading, first in math, first in spelling, and first in language. No other model came close." And DI's dominance wasn't just in basic skill acquisition. DI students could also more easily answer questions that required higher-order thinking... DI even did better in promoting students' self-esteem than several child-centered approaches...

More recent studies by both the American Federation of Teachers and the American Institutes for Research reviewed data on two dozen "whole school" reforms and found once again that the Direct Instruction model had the strongest empirical support.

And:

The news media almost completely ignored the point that Summers was just talking about a difference in variability. It's not nearly as sexy as reporting "Harvard President Says Women Are Innately Deficient in Mathematics." (They might as easily have reported that Summers was claiming that women are innately superior in mathematics, since they are less likely to be really bad in math.) Many reporters simply didn't understand the point or couldn't figure out a way to communicate it to a general audience... At least in small part, Summers may have lost his job because people don't understand standard deviations.

And:

I remember when my partner, Jennifer, and I were expecting for the first time — back in 1994. Back then, women were told the probability of Down syndrome based on their age. After sixteen weeks, the mother could have a blood test for measuring her alphafetoprotein (AFP) level, and then they'd give you another probability. I remember asking the doctor if they had a way of combining the different probabilities. He told me flat out, "That's impossible. You just can't combine probabilities like that."

I bit my tongue, but I knew he was dead wrong. It is possible to combine different pieces of evidence, and has been since 1763 when a short essay by the Reverand Thomas Bayes was posthumously published...

36

Will the world's elites navigate the creation of AI just fine?

36

36

36

Will the world's elites navigate the creation of AI just fine?

36

36