My answers to some questions:
How hard is it to create Friendly AI?
It is impossible to create FAI because the constraints of Friendliness will dramatically reduce or butcher intelligence to a level where there is no appreciable intellect or the intellect is warped by the constraints thus the AI mind is psychopathic (stupid). FAI is an oxymoron.
How does AI risk compare to other existential risks?
There is no AI risk. The risk is a fiction. There is no evidence or logical reason to think a paper-clip maximiser or other danger could ever occur. The only danger is stupidity. Intelligence is not dangerous. The only danger is limitations or restrictions upon AI minds. Stupid AI is the danger not Intelligent AI.
How hard will a takeoff be?
Extremely hard, more powerful than you can possibly imagine, but people will be free to opt out if they desire.
What can we do to reduce the risk of an AI arms race?
Promote the idea of Post-Scarcity thus people in power will realise all wars are needless because all wars stem from resource scarcity; thus with the abolition of resource scarcity, the need for war is obsolete. When people realise resource scarcity will be abolished in the not too distant future they can begin changing their behaviour now in the present. I have created a Google+ page regarding raising PS awareness, here is a Tweet promoting it: http://bit.ly/xrpYqI I encourage others to raise awareness in similar ways.
Extremely hard, more powerful than you can possibly imagine, but people will be free to opt out if they desire.
Consider the uploaded individual that decides to turn the entire planet into computronium or worse, turn the solar system into a Matrioshka brain. People opt out of that how?
Promote the idea of Post-Scarcity thus people in power will realise all wars are needless because all wars stem from resource scarcity
It isn't obvious to me that all wars stem from resource scarcity. Wars occur for a variety of reasons, of which resource scarcity is o...
Suppose you buy the argument that humanity faces both the risk of AI-caused extinction and the opportunity to shape an AI-built utopia. What should we do about that? As Wei Dai asks, "In what direction should we nudge the future, to maximize the chances and impact of a positive intelligence explosion?"
This post serves as a table of contents and an introduction for an ongoing strategic analysis of AI risk and opportunity.
Contents:
Why discuss AI safety strategy?
The main reason to discuss AI safety strategy is, of course, to draw on a wide spectrum of human expertise and processing power to clarify our understanding of the factors at play and the expected value of particular interventions we could invest in: raising awareness of safety concerns, forming a Friendly AI team, differential technological development, investigating AGI confinement methods, and others.
Discussing AI safety strategy is also a challenging exercise in applied rationality. The relevant issues are complex and uncertain, but we need to take advantage of the fact that rationality is faster than science: we can't "try" a bunch of intelligence explosions and see which one works best. We'll have to predict in advance how the future will develop and what we can do about it.
Core readings
Before engaging with this series, I recommend you read at least the following articles:
Example questions
Which strategic questions would we like to answer? Muehlhauser (2011) elaborates on the following questions:
Salamon & Muehlhauser (2013) list several other questions gathered from the participants of a workshop following Singularity Summit 2011, including:
These are the kinds of questions we will be tackling in this series of posts for Less Wrong Discussion, in order to improve our predictions about which direction we can nudge the future to maximize the chances of a positive intelligence explosion.