Will the world's elites navigate the creation of AI just fine?

lukeprog

One open question in AI risk strategy is: Can we trust the world's elite decision-makers (hereafter "elites") to navigate the creation of human-level AI (and beyond) just fine, without the kinds of special efforts that e.g. Bostrom and Yudkowsky think are needed?

Some reasons for concern include:

Otherwise smart people say unreasonable things about AI safety.
Many people who believed AI was around the corner didn't take safety very seriously.
Elites have failed to navigate many important issues wisely (2008 financial crisis, climate change, Iraq War, etc.), for a variety of reasons.
AI may arrive rather suddenly, leaving little time for preparation.

But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):

If AI is preceded by visible signals, elites are likely to take safety measures. Effective measures were taken to address asteroid risk. Large resources are devoted to mitigating climate change risks. Personal and tribal selfishness align with AI risk-reduction in a way they may not align on climate change. Availability of information is increasing over time.
AI is likely to be preceded by visible signals. Conceptual insights often take years of incremental tweaking. In vision, speech, games, compression, robotics, and other fields, performance curves are mostly smooth. "Human-level performance at X" benchmarks influence perceptions and should be more exhaustive and come more rapidly as AI approaches. Recursive self-improvement capabilities could be charted, and are likely to be AI-complete. If AI succeeds, it will likely succeed for reasons comprehensible by the AI researchers of the time.
Therefore, safety measures will likely be taken.
If safety measures are taken, then elites will navigate the creation of AI just fine. Corporate and government leaders can use simple heuristics (e.g. Nobel prizes) to access the upper end of expert opinion. AI designs with easily tailored tendency to act may be the easiest to build. The use of early AIs to solve AI safety problems creates an attractor for "safe, powerful AI." Arms races not insurmountable.

The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)

Personally, I am not very comforted by this argument because:

Elites often fail to take effective action despite plenty of warning.
I think there's a >10% chance AI will not be preceded by visible signals.
I think the elites' safety measures will likely be insufficient.

Obviously, there's a lot more for me to spell out here, and some of it may be unclear. The reason I'm posting these thoughts in such a rough state is so that MIRI can get some help on our research into this question.

In particular, I'd like to know:

Which historical events are analogous to AI risk in some important ways? Possibilities include: nuclear weapons, climate change, recombinant DNA, nanotechnology, chloroflourocarbons, asteroids, cyberterrorism, Spanish flu, the 2008 financial crisis, and large wars.
What are some good resources (e.g. books) for investigating the relevance of these analogies to AI risk (for the purposes of illuminating elites' likely response to AI risk)?
What are some good studies on elites' decision-making abilities in general?
Has the increasing availability of information in the past century noticeably improved elite decision-making?

Some reasons for concern include:

Otherwise smart people say unreasonable things about AI safety.
Many people who believed AI was around the corner didn't take safety very seriously.
Elites have failed to navigate many important issues wisely (2008 financial crisis, climate change, Iraq War, etc.), for a variety of reasons.
AI may arrive rather suddenly, leaving little time for preparation.

But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):

If AI is preceded by visible signals, elites are likely to take safety measures. Effective measures were taken to address asteroid risk. Large resources are devoted to mitigating climate change risks. Personal and tribal selfishness align with AI risk-reduction in a way they may not align on climate change. Availability of information is increasing over time.
AI is likely to be preceded by visible signals. Conceptual insights often take years of incremental tweaking. In vision, speech, games, compression, robotics, and other fields, performance curves are mostly smooth. "Human-level performance at X" benchmarks influence perceptions and should be more exhaustive and come more rapidly as AI approaches. Recursive self-improvement capabilities could be charted, and are likely to be AI-complete. If AI succeeds, it will likely succeed for reasons comprehensible by the AI researchers of the time.
Therefore, safety measures will likely be taken.
If safety measures are taken, then elites will navigate the creation of AI just fine. Corporate and government leaders can use simple heuristics (e.g. Nobel prizes) to access the upper end of expert opinion. AI designs with easily tailored tendency to act may be the easiest to build. The use of early AIs to solve AI safety problems creates an attractor for "safe, powerful AI." Arms races not insurmountable.

The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)

Personally, I am not very comforted by this argument because:

Elites often fail to take effective action despite plenty of warning.
I think there's a >10% chance AI will not be preceded by visible signals.
I think the elites' safety measures will likely be insufficient.

In particular, I'd like to know:

Which historical events are analogous to AI risk in some important ways? Possibilities include: nuclear weapons, climate change, recombinant DNA, nanotechnology, chloroflourocarbons, asteroids, cyberterrorism, Spanish flu, the 2008 financial crisis, and large wars.
What are some good resources (e.g. books) for investigating the relevance of these analogies to AI risk (for the purposes of illuminating elites' likely response to AI risk)?
What are some good studies on elites' decision-making abilities in general?
Has the increasing availability of information in the past century noticeably improved elite decision-making?

From Think Like a Freak:

Now we asked about the print ads. How often did they run? One executive told us, with obvious pride, that the company had bought newspaper inserts every single Sunday for the past twenty years in 250 markets across the United States.

So how could they tell whether these ads were effective? They couldn’t. With no variation whatsoever, it was impossible to know.

What if, we said, the company ran an experiment to find out? In science, the randomized control trial has been the gold standard of learning for hundreds of years—but why should scientists have all the fun? We described an experiment the company might run. They could select 40 major markets across the country and randomly divide them into two groups. In the first group, the company would keep buying newspaper ads every Sunday. In the second group, they’d go totally dark—not a single ad. After three months, it would be easy to compare merchandise sales in the two groups to see how much the print ads mattered.

“Are you crazy?” one marketing executive said. “We can’t possibly go dark in 20 markets. Our CEO would kill us.”

“Yeah,” said someone else, “it’d be like that kid in Pittsburgh.”

What kid in Pittsburgh?

They told us about a summer intern who was supposed to call in the Sunday ad buys for the Pittsburgh newspapers. For whatever reason, he botched his assignment and failed to make the calls. So for the entire summer, the company ran no newspaper ads in a large chunk of Pittsburgh. “Yeah,” one executive said, “we almost got fired for that one.”

So what happened, we asked, to the company’s Pittsburgh sales that summer?

They looked at us, then at each other—and sheepishly admitted it never occurred to them to check the data. When they went back and ran the numbers, they found something shocking: the ad blackout hadn’t affected Pittsburgh sales at all!

Now that, we said, is valuable feedback. The company may well be wasting hundreds of millions of dollars on advertising. How could the executives know for sure? That 40-market experiment would go a long way toward answering the question. And so, we asked them, are you ready to try it now?

“Are you crazy?” the marketing executive said again. “We’ll get fired if we do that!”

To this day, on every single Sunday in every single market, this company still buys newspaper advertising—even though the only real piece of feedback they ever got is that the ads don’t work.

More (#1) from Think Like a Freak:

The rules were simple. A contestant ate as many hot dogs and buns (“HDB,” officially) as he could in 12 minutes. Any HDB or portion thereof already in the eater’s mouth when the final bell rang would count toward his total as long as he swallowed it eventually. An eater could be disqualified, however, if during the contest a significant amount of HDB that had gone into his mouth came back out—known in the sport as a “reversal of fortune.” Condiments were allowed but no serious competitor would bother. Beverages were also

... (read more)

36

Will the world's elites navigate the creation of AI just fine?

36

36

36

Will the world's elites navigate the creation of AI just fine?

36

36