What could an FAI project look like? Louie points out that it might look like Princeton's Institute for Advanced Study:
Created as a haven for thinking, the Institute [for Advanced Study] remains for many the Shangri-la of academe: a playground for the scholarly superstars who become the Institute's permanent faculty. These positions carry no teaching duties, few administrative responsibilities, and high salaries, and so represent a pinnacle of academic advancement. The expectation is that given this freedom, the professors at the Institute will think the big thoughts that can propel social and intellectual progress. Over the years the permanent faculty has included Nobel laureates as well as recipients of almost every other intellectual honor. Among the mathematicians, there have been several winners of the Fields Medal…
If the permanent faculty makes up the intellectual foundation of the Institute, the lifeblood is provided by the parade of international visitors who bring a continuous influx of new ideas. They may come for as little as an afternoon, or as long as a few years, in which case they take up temporary positions as Institute "members."
Idea #1: Write a good, very technical Open Problems in Friendly Artificial Intelligence, get a few of the best mathematicians/physicists who care about FAI accepted as visitors, have them talk to faculty and visitors at about the technical problems related to FAI.
Idea #2: Convince wealthy donors to endow a chair at the Institute for Advanced Study for somebody to do FAI research. (Princeton may not mind us sending another brilliant person and a bunch of money their way.)
Similar research institutes: PARC, Bell Labs, Perimeter Institute, maybe others?
How do you maintain secrecy in such a setting? Or is there a new line of thought that says secrecy isn't necessary for an FAI project?
Suppose you buy the argument that humanity faces both the risk of AI-caused extinction and the opportunity to shape an AI-built utopia. What should we do about that? As Wei Dai asks, "In what direction should we nudge the future, to maximize the chances and impact of a positive intelligence explosion?"
This post serves as a table of contents and an introduction for an ongoing strategic analysis of AI risk and opportunity.
Contents:
Why discuss AI safety strategy?
The main reason to discuss AI safety strategy is, of course, to draw on a wide spectrum of human expertise and processing power to clarify our understanding of the factors at play and the expected value of particular interventions we could invest in: raising awareness of safety concerns, forming a Friendly AI team, differential technological development, investigating AGI confinement methods, and others.
Discussing AI safety strategy is also a challenging exercise in applied rationality. The relevant issues are complex and uncertain, but we need to take advantage of the fact that rationality is faster than science: we can't "try" a bunch of intelligence explosions and see which one works best. We'll have to predict in advance how the future will develop and what we can do about it.
Core readings
Before engaging with this series, I recommend you read at least the following articles:
Example questions
Which strategic questions would we like to answer? Muehlhauser (2011) elaborates on the following questions:
Salamon & Muehlhauser (2013) list several other questions gathered from the participants of a workshop following Singularity Summit 2011, including:
These are the kinds of questions we will be tackling in this series of posts for Less Wrong Discussion, in order to improve our predictions about which direction we can nudge the future to maximize the chances of a positive intelligence explosion.