"Can we know what to do about AI?": An Introduction

JonahS

28 "Can we know what to do about AI?": An Introduction

9th Jul 2013

2 min read

28

I'm currently working on a research project for MIRI, and I would welcome feedback on my research as I proceed. In this post, I describe the project.

As a part of an effort to steel-man objections to MIRI's mission, MIRI Executive Director Luke Muehlhauser has asked me to develop the following objection:

"Even if AI is somewhat likely to arrive during the latter half of this century, how on earth can we know what to do about it now, so far in advance?"

In Luke's initial email to me, he wrote:

I think there are plausibly many weak arguments and historical examples suggesting that P: "it's very hard to nudge specific distant events in a positive direction through highly targeted actions or policies undertaken today." Targeted actions might have no lasting effect, or they might completely miss their mark, or they might backfire.

If P is true, this would weigh against the view that a highly targeted intervention today (e.g. Yudkowsky's Friendly AI math research) is likely to positively affect the future creation of AI, and might instead weigh in favor of the view that all we can do about AGI from this distance is to engage in broad interventions likely to improve our odds of wisely handling future crises in general — e.g. improving decision-making institutions, spreading rationality, etc.

I'm interested in abstract arguments for P, but I'm even more interested in historical data. What can we learn from seemingly analogous cases, and are those cases analogous in the relevant ways? What sorts of counterfactual history can we do to clarify our picture?

Luke and I brainstormed a list of potential historical examples of people predicting the future 10+ years out, and using the predictions to inform their actions. We came up with the following potential examples, which I've listed in chronological order by approximate year:

1896: Svante Arrhenius's prediction of anthropogenic climate change.
1935: Leo Szilard's ~1935 attempts to keep his patent of the atomic bomb secret from Germany.
1950-1980: Efforts to win the Cold War decades later, such as increasing education for gifted children.
1960: Norbert Weiner highlighting the dangers of artificial intelligence.
1972: The circle of ideas and actions around the The Limits to Growth, a book about the consequences of unchecked population growth and economic growth.
1975: The WASH-1400 reactor safety study, which attempted to assess the risks associated with nuclear reactors.
1975: The Asilomar Conference on Recombinant DNA, which set up guidelines to ensure the safety of recombinant DNA technology.
1978: China's one-child policy to reduce population growth.
1980: The Ford Foundation setting up a policy think in India that helped India recover from its 1991 financial crisis
1988: Early climate change mitigation efforts
1992+: Asteroid strike deflection efforts
???: Possible deliberate long term efforts to produce revolutionary scientific technologies.
????: Long term computer security research

In addition, we selected

The Signal and the Noise: Why So Many Predictions Fail — but Some Don't by Nate Silver
Expert Political Judgment: How Good Is It? How Can We Know? by Philip Tetlock

as background reading.

I would greatly appreciate any ideas from the Less Wrong community concerning potential historical examples and relevant background reading.

Over the coming weeks, I'll be making a series of discussion board posts on Less Wrong reporting on my findings, and linking these posts here.

Personal Blog

28

New Comment

Rendering 0/69 comments, sorted by

top scoring

(show more) Click to highlight new comments since: Today at 10:17 PM

Moderation Log

28 "Can we know what to do about AI?": An Introduction

by JonahS

9th Jul 2013

2 min read

28

I'm currently working on a research project for MIRI, and I would welcome feedback on my research as I proceed. In this post, I describe the project.

As a part of an effort to steel-man objections to MIRI's mission, MIRI Executive Director Luke Muehlhauser has asked me to develop the following objection:

"Even if AI is somewhat likely to arrive during the latter half of this century, how on earth can we know what to do about it now, so far in advance?"

In Luke's initial email to me, he wrote:

I think there are plausibly many weak arguments and historical examples suggesting that P: "it's very hard to nudge specific distant events in a positive direction through highly targeted actions or policies undertaken today." Targeted actions might have no lasting effect, or they might completely miss their mark, or they might backfire.

If P is true, this would weigh against the view that a highly targeted intervention today (e.g. Yudkowsky's Friendly AI math research) is likely to positively affect the future creation of AI, and might instead weigh in favor of the view that all we can do about AGI from this distance is to engage in broad interventions likely to improve our odds of wisely handling future crises in general — e.g. improving decision-making institutions, spreading rationality, etc.

I'm interested in abstract arguments for P, but I'm even more interested in historical data. What can we learn from seemingly analogous cases, and are those cases analogous in the relevant ways? What sorts of counterfactual history can we do to clarify our picture?

1896: Svante Arrhenius's prediction of anthropogenic climate change.
1935: Leo Szilard's ~1935 attempts to keep his patent of the atomic bomb secret from Germany.
1950-1980: Efforts to win the Cold War decades later, such as increasing education for gifted children.
1960: Norbert Weiner highlighting the dangers of artificial intelligence.
1972: The circle of ideas and actions around the The Limits to Growth, a book about the consequences of unchecked population growth and economic growth.
1975: The WASH-1400 reactor safety study, which attempted to assess the risks associated with nuclear reactors.
1975: The Asilomar Conference on Recombinant DNA, which set up guidelines to ensure the safety of recombinant DNA technology.
1978: China's one-child policy to reduce population growth.
1980: The Ford Foundation setting up a policy think in India that helped India recover from its 1991 financial crisis
1988: Early climate change mitigation efforts
1992+: Asteroid strike deflection efforts
???: Possible deliberate long term efforts to produce revolutionary scientific technologies.
????: Long term computer security research

In addition, we selected

The Signal and the Noise: Why So Many Predictions Fail — but Some Don't by Nate Silver
Expert Political Judgment: How Good Is It? How Can We Know? by Philip Tetlock

as background reading.

I would greatly appreciate any ideas from the Less Wrong community concerning potential historical examples and relevant background reading.

Over the coming weeks, I'll be making a series of discussion board posts on Less Wrong reporting on my findings, and linking these posts here.

Personal Blog

28

Mentioned in

35Some highlights from Nate Silver's "The Signal and the Noise"

26Notes on "The Limits to Growth" and surrounding material

16The WASH-1400 Reactor Safety Study

13Svante Arrhenius's Prediction of Climate Change

11Norbert Wiener on automation and unemployment

New Comment

Rendering 0/69 comments, sorted by

top scoring

(show more) Click to highlight new comments since: Today at 10:17 PM

Moderation Log

More from JonahS

Curated and popular this week

69Comments

Comment Permalink

JonahS13y40

Thanks Nick. I didn't give enough detail as to what I'm looking for in my original post. My initial reactions

CFCs?

Yes, Luke and I talked about this — I forgot to list it.

Asteroid tracking (if they weren't planning on asteroids in the next ten years, seems like natural risks with average return periods > 10 years could be a good source of examples, though I feel this maybe isn't what you are asking for because it is so statistical rather than scenario-based)

I think that I'm looking for things involving more speculation. Also, we don't know what the impact of asteroid tracking effots will be.

Legislation where you don't expect the legislation to have effects for more than 10 years. Some international treaties/conventions might be examples

Here there's a clear historical precedent that people are using to inform their decision making.

Religion examples. In 1833 William Miller predicted the return of Jesus Christ in 1943 and people took specific actions.

How many people?

Space exploration examples? Maybe these things required far advance planning. Voyager may be an example?

I'll brood on this. Intuitively, it doesn't seem relevant, but I have trouble placing my finger on why.

Futures markets? How far do they go out?

I'll investigate.

Simon-Ehrlich wager

I'm looking at this as a part of the Limits to Growth investigation.

Sleeper agents? I have heard stories that some Russian agents were sleepers in the US for greater than 10 years, waiting to strike at the right time if war broke out. May want to check on it.

I'll investigate.

Eugenics people

Did they have any impact? Could they plausibly have?

People conducting very long longitudinal studies

These seem like they fall into the category "general scientific research that could have humanitarian value" rather than being driven by specific predictions and being aimed at influencing specific outcomes.

There are some mundane examples that fit your description but I don't think are what you are asking for. E.g., people working on buildings that take more than 10 years to complete, people buying something for a young daughter and saving it for when she gets married (I guess this used to be a thing), people getting mortgages, people saving for retirement, people studying anatomy when 16 in hopes of becoming a doctor, making a trust fund for your kid.

Here there's a clear historical precedent that people are using to inform their decision making.

Nick_Beckstead13y20

Religion examples. In 1833 William Miller predicted the return of Jesus Christ in 1943 and people took specific actions.

How many people?

I couldn't find the answer on the Wikipedia page, but it sounds like it was a big deal. One data point is this (from the Wikipedia page):

As the various dates of Christ’s predicted return approached, Millerite publishing went into high gear. In May 1843, 21,000 copies of the various Millerite papers were published for distribution each week. In New York alone, in the five-month period ending April 1843, 600,000 copi

... (read more)

10Protagoras13y

I recall happening upon an ancient book on biology in a library, and glancing at what someone in the field thought about eugenics a hundred years ago. The author was as racist as one would expect for his time, but he considered eugenics an absolute waste of time on the basis that actually having a significant impact would require interventions on a vastly larger scale than anybody was really imagining; in practice, he thought it obvious that any deliberate selection effects would be miniscule and completely swamped by the ongoing effects of normal human mate selection practices. I don't know if his view was widespread, but it does seem to be true of eugenics on the scale it was usually discussed or attempted by anyone except the Nazis, and something that at least some people had figured out even before eugenics went out of favor for other reasons.

See in context