Suppose you buy the argument that humanity faces both the risk of AI-caused extinction and the opportunity to shape an AI-built utopia. What should we do about that? As Wei Dai asks, "In what direction should we nudge the future, to maximize the chances and impact of a positive intelligence explosion?"
This post serves as a table of contents and an introduction for an ongoing strategic analysis of AI risk and opportunity.
Contents:
- Introduction (this post)
- Humanity's Efforts So Far
- A Timeline of Early Ideas and Arguments
- Questions We Want Answered
- Strategic Analysis Via Probability Tree
- Intelligence Amplification and Friendly AI
- ...
Why discuss AI safety strategy?
The main reason to discuss AI safety strategy is, of course, to draw on a wide spectrum of human expertise and processing power to clarify our understanding of the factors at play and the expected value of particular interventions we could invest in: raising awareness of safety concerns, forming a Friendly AI team, differential technological development, investigating AGI confinement methods, and others.
Discussing AI safety strategy is also a challenging exercise in applied rationality. The relevant issues are complex and uncertain, but we need to take advantage of the fact that rationality is faster than science: we can't "try" a bunch of intelligence explosions and see which one works best. We'll have to predict in advance how the future will develop and what we can do about it.
Core readings
Before engaging with this series, I recommend you read at least the following articles:
- Muehlhauser & Salamon, Intelligence Explosion: Evidence and Import (2013)
- Yudkowsky, AI as a Positive and Negative Factor in Global Risk (2008)
- Chalmers, The Singularity: A Philosophical Analysis (2010)
Example questions
Which strategic questions would we like to answer? Muehlhauser (2011) elaborates on the following questions:
- What methods can we use to predict technological development?
- Which kinds of differential technological development should we encourage, and how?
- Which open problems are safe to discuss, and which are potentially dangerous?
- What can we do to reduce the risk of an AI arms race?
- What can we do to raise the "sanity waterline," and how much will this help?
- What can we do to attract more funding, support, and research to x-risk reduction and to specific sub-problems of successful Singularity navigation?
- Which interventions should we prioritize?
- How should x-risk reducers and AI safety researchers interact with governments and corporations?
- How can optimal philanthropists get the most x-risk reduction for their philanthropic buck?
- How does AI risk compare to other existential risks?
- Which problems do we need to solve, and which ones can we have an AI solve?
- How can we develop microeconomic models of WBEs and self-improving systems?
- How can we be sure a Friendly AI development team will be altruistic?
Salamon & Muehlhauser (2013) list several other questions gathered from the participants of a workshop following Singularity Summit 2011, including:
- How hard is it to create Friendly AI?
- What is the strength of feedback from neuroscience to AI rather than brain emulation?
- Is there a safe way to do uploads, where they don't turn into neuromorphic AI?
- How possible is it to do FAI research on a seastead?
- How much must we spend on security when developing a Friendly AI team?
- What's the best way to recruit talent toward working on AI risks?
- How difficult is stabilizing the world so we can work on Friendly AI slowly?
- How hard will a takeoff be?
- What is the value of strategy vs. object-level progress toward a positive Singularity?
- How feasible is Oracle AI?
- Can we convert environmentalists into people concerned with existential risk?
- Is there no such thing as bad publicity [for AI risk reduction] purposes?
These are the kinds of questions we will be tackling in this series of posts for Less Wrong Discussion, in order to improve our predictions about which direction we can nudge the future to maximize the chances of a positive intelligence explosion.
So asr, would you say violence is generally stupid or intelligent?
People often equate mindlessness with violence thus the phrase mindless violence is reasonably common but I have never encountered the phrase intelligent violence, is intelligent violence an oxymoron? Surely intelligent people can resolve conflict via non-violent methods?
Here are a couple of news reports mentioning mindless violence:
http://www.bbc.co.uk/news/uk-england-london-17062738
http://www.thesun.co.uk/sol/homepage/sport/4149765/Brainless-brawlers-cost-schools.html
It would be interested to know how many scientists or philosophers actually engage in violence.
A high level of intelligence can be a prohibiting factor for admission into the police force. There was a court case where police applicant was refused a job due to his high intelligence thus he sued on grounds of discrimination. I wonder how many scientists choose to fight in the army, are they eager to kill people in Iran, Iraq or Afghanistan? Does the army prohibit intelligent people from joining?
Perhaps a scientific study needs to be undertaken regarding a possible relationship between stupidity and violence, intelligence and pacifism?
Regarding the violence of Von Neumann there is no actual evidence of violence, as far as I am aware, it is merely hot air, violent rhetoric, but I would also question the "intelligence" of people who advocate violence. Perhaps their "intelligence" is a misnomer, thus what people are actually referring to is pseudo-intelligence or partial intelligence. Even stupid people can occasionally do clever things, and sometimes smart people do stupid things but generally I think it is safe to say intelligent people are not violent.
We have gone to a great deal of trouble, in modern society, to make violence a bad option, so today in our society often violence is committed by the impulsive, mentally ill, or short-sighted. But that's not an inevitable property of violence and hasn't always been true. You would have gotten a different answer before the 20th century. I don't know what answer you'll get in the 22nd century.
... (read more)