Will the world's elites navigate the creation of AI just fine?

lukeprog

One open question in AI risk strategy is: Can we trust the world's elite decision-makers (hereafter "elites") to navigate the creation of human-level AI (and beyond) just fine, without the kinds of special efforts that e.g. Bostrom and Yudkowsky think are needed?

Some reasons for concern include:

Otherwise smart people say unreasonable things about AI safety.
Many people who believed AI was around the corner didn't take safety very seriously.
Elites have failed to navigate many important issues wisely (2008 financial crisis, climate change, Iraq War, etc.), for a variety of reasons.
AI may arrive rather suddenly, leaving little time for preparation.

But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):

If AI is preceded by visible signals, elites are likely to take safety measures. Effective measures were taken to address asteroid risk. Large resources are devoted to mitigating climate change risks. Personal and tribal selfishness align with AI risk-reduction in a way they may not align on climate change. Availability of information is increasing over time.
AI is likely to be preceded by visible signals. Conceptual insights often take years of incremental tweaking. In vision, speech, games, compression, robotics, and other fields, performance curves are mostly smooth. "Human-level performance at X" benchmarks influence perceptions and should be more exhaustive and come more rapidly as AI approaches. Recursive self-improvement capabilities could be charted, and are likely to be AI-complete. If AI succeeds, it will likely succeed for reasons comprehensible by the AI researchers of the time.
Therefore, safety measures will likely be taken.
If safety measures are taken, then elites will navigate the creation of AI just fine. Corporate and government leaders can use simple heuristics (e.g. Nobel prizes) to access the upper end of expert opinion. AI designs with easily tailored tendency to act may be the easiest to build. The use of early AIs to solve AI safety problems creates an attractor for "safe, powerful AI." Arms races not insurmountable.

The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)

Personally, I am not very comforted by this argument because:

Elites often fail to take effective action despite plenty of warning.
I think there's a >10% chance AI will not be preceded by visible signals.
I think the elites' safety measures will likely be insufficient.

Obviously, there's a lot more for me to spell out here, and some of it may be unclear. The reason I'm posting these thoughts in such a rough state is so that MIRI can get some help on our research into this question.

In particular, I'd like to know:

Which historical events are analogous to AI risk in some important ways? Possibilities include: nuclear weapons, climate change, recombinant DNA, nanotechnology, chloroflourocarbons, asteroids, cyberterrorism, Spanish flu, the 2008 financial crisis, and large wars.
What are some good resources (e.g. books) for investigating the relevance of these analogies to AI risk (for the purposes of illuminating elites' likely response to AI risk)?
What are some good studies on elites' decision-making abilities in general?
Has the increasing availability of information in the past century noticeably improved elite decision-making?

Some reasons for concern include:

Otherwise smart people say unreasonable things about AI safety.
Many people who believed AI was around the corner didn't take safety very seriously.
Elites have failed to navigate many important issues wisely (2008 financial crisis, climate change, Iraq War, etc.), for a variety of reasons.
AI may arrive rather suddenly, leaving little time for preparation.

But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):

If AI is preceded by visible signals, elites are likely to take safety measures. Effective measures were taken to address asteroid risk. Large resources are devoted to mitigating climate change risks. Personal and tribal selfishness align with AI risk-reduction in a way they may not align on climate change. Availability of information is increasing over time.
AI is likely to be preceded by visible signals. Conceptual insights often take years of incremental tweaking. In vision, speech, games, compression, robotics, and other fields, performance curves are mostly smooth. "Human-level performance at X" benchmarks influence perceptions and should be more exhaustive and come more rapidly as AI approaches. Recursive self-improvement capabilities could be charted, and are likely to be AI-complete. If AI succeeds, it will likely succeed for reasons comprehensible by the AI researchers of the time.
Therefore, safety measures will likely be taken.
If safety measures are taken, then elites will navigate the creation of AI just fine. Corporate and government leaders can use simple heuristics (e.g. Nobel prizes) to access the upper end of expert opinion. AI designs with easily tailored tendency to act may be the easiest to build. The use of early AIs to solve AI safety problems creates an attractor for "safe, powerful AI." Arms races not insurmountable.

The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)

Personally, I am not very comforted by this argument because:

Elites often fail to take effective action despite plenty of warning.
I think there's a >10% chance AI will not be preceded by visible signals.
I think the elites' safety measures will likely be insufficient.

In particular, I'd like to know:

Which historical events are analogous to AI risk in some important ways? Possibilities include: nuclear weapons, climate change, recombinant DNA, nanotechnology, chloroflourocarbons, asteroids, cyberterrorism, Spanish flu, the 2008 financial crisis, and large wars.
What are some good resources (e.g. books) for investigating the relevance of these analogies to AI risk (for the purposes of illuminating elites' likely response to AI risk)?
What are some good studies on elites' decision-making abilities in general?
Has the increasing availability of information in the past century noticeably improved elite decision-making?

More (#2) from The Myth of the Rational Voter:

Many economists took the [self-interested voter hypothesis] for granted, but few bothered to defend it. After completing my doctorate I read more outside my discipline, and discovered that political scientists have subjected the SIVH to extensive and diverse empirical tests. Their results are impressively uniform: The SIVH fails.

Start with the easiest case: partisan identification. Both economists and the public almost automatically accept the view that poor people are liberal Democrats and rich people are conservative Republicans. The data paint a quite different picture. At least in the United States, there is only a flimsy connection between individuals’ incomes and their ideology or party. The sign fits the stereotype: As your income rises, you are more likely to be conservative and Republican. But the effect is small, and shrinks further after controlling for race. A black millionaire is more likely to be a Democrat than a white janitor. The Republicans might be the party for the rich, but they are not the party of the rich.

We see the same pattern for specific policies. The elderly are not more in favor of Social Security and Medicare than the rest of the population. Seniors strongly favor these programs, but so do the young. Contrary to the SIVH-inspired bumper sticker “If men got pregnant, abortion would be a sacrament,” men appear a little more pro-choice on abortion than women. Compared to the overall population, the unemployed are at most a little more in favor of government-guaranteed jobs, and the uninsured at most a little more supportive of national health insurance. Measures of self-interest predict little about beliefs about economic policy. Even when the stakes are life and death, political self-interest rarely surfaces: Males vulnerable to the draft support it at normal levels, and families and friends of conscripts in Vietnam were in fact more opposed to withdrawal than average.

The broken clock of the SIVH is right twice a day. It fails for party identification, Social Security, Medicare, abortion, job programs, national health insurance, Vietnam, and the draft. But it works tolerably well for a few scattered issues. You might expect to see the exceptions on big questions with a lot of money at stake, but the truth is almost the reverse. The SIVH shines brightest on the banal issue of smoking. Donald Green and Ann Gerken find that smokers and nonsmokers are ideologically and demographically similar, but smokers are a lot more opposed to restrictions and taxes on their favorite vice. Belief in “smokers’ rights” cleanly rises with daily cigarette consumption: fully 61.5% of “heavy” smokers want laxer antismoking policies, but only 13.9% of people who “never smoked” agree. If the SIVH were true, comparable patterns of belief would be everywhere. They are not.

The elderly are not more in favor of Social Security and Medicare than the rest of the population. Seniors strongly favor these programs, but so do the young. Contrary to the SIVH-inspired bumper sticker “If men got pregnant, abortion would be a sacrament,” men appear a little more pro-choice on abortion than women. Compared to the overall population, the unemployed are at most a little more in favor of government-guaranteed jobs, and the uninsured at most a little more supportive of national health insurance.

This is an absurdly narrow definition of se... (read more)

36

Will the world's elites navigate the creation of AI just fine?

36

36

36

Will the world's elites navigate the creation of AI just fine?

36

36