(This is a introduction, for those not immersed in the Singularity world, into the history of and relationships between SU, SIAI [SI, MIRI], SS, LW, CSER, FHI, and CfAR. It also has some opinions, which are strictly my own.)
The good news is that there were no Singularity Wars.
The Bay Area had a Singularity University and a Singularity Institute, each going in a very different direction. You'd expect to see something like the People's Front of Judea and the Judean People's Front, burning each other's grain supplies as the Romans moved in.
We are pleased to announce a new vacancy at the Programme on the Impacts of Future Technology. Please forward to any who would be interested.
James Martin Postdoctoral Research Fellowship:
Socio-economic Impacts of Technological Change
with the Programme on the Impacts of Future Technology
University of Oxford
Faculty of Philosophy
The Future of Humanity Institute, Oxford Martin School
Grade 7: £29,249–£39,257 per annum
Protocol reference number: HUM/11043F/E
Applications are invited for a Research Fellowship within the Oxford Martin Programme on the Impacts of Future Technology, an interdisciplinary programme within the Oxford Martin School at Oxford University. This Fellowship is available on a one year full-time or two years part-time fixed term basis.
The Programme, directed by Professor Nick Bostrom, analyzes possibilities related to long-range technological change and potential social impacts of future transformative technologies. Research foci include the future of computing and machine intelligence, existential risks, predictive and evaluative uncertainty, and related philosophical issues.
The postholder will conduct research on socio-economic and strategic impacts of potentially transformative or disruptive future technological innovations, including (but not limited to) advances in computing and machine intelligence, biosecurity and surveillance technology. Academic background is open. Potential areas include economics, political science, legal theory, and sociology; other relevant areas include environmental economics, game theory, and risk management. A multidisciplinary background would be favourable.
For further particulars and application details, please see:
The deadline for applications is Monday 9th April.
After reading the FHI achievement report for 2008-2010, I thought it might be useful to compare their achievements to those of SIAI during the same time period. Since SIAI does not have an equivalent report, I've mostly pulled the data of their achievements from the SIAI blog.
My intention here is to help figure out which organization makes better use of my donations. For that purpose, I'm only looking at actual concrete outputs, and ignoring achievements such as successful fundraising drives or the hiring of extra staff.
For citation counts, I'm using Google Scholar data as-is. Note that this will include both self-cites and some cites from pages that really shouldn't be counted, since Google Scholar seems to be a bit liberal about what it includes in its database. I'm unsure as to whether or not the citation counts are very meaningful, since there hasn't been much time for anyone to cite papers published in 2010, say. But I'm including them anyway.
Future of Humanity Institute
Publications. The Achievement Report highlights three books and 22 journal articles. In addition, FHI staff has written 34 book chapters for academic volumes, including Companion to Philosophy of Technology; New Waves in Philosophy of Technology; Philosophy: Theoretical and Empirical Explorations; and Oxford Handbook of Neuroethics.
The three are the hardcover and paperback editions of Human Enhancement, as well as a paperback edition of Anthropic Bias: Observation Selection Effects in Science and Philosophy. Human Enhancement has been cited 22 times. Anthropic Bias was originally published in 2002, so I'm not including its citation count.
The highlighted 22 journal articles had been cited 59 times in total. The overwhelmingly most cited article was Cognitive Enhancement: Methods, Ethics, Regulatory Challenges in Science and Engineering Ethics, with 39 cites. The runner-up was Probing the Improbable: Methodological. Challenges for Risks with Low Probabilities and High Stake, with 5 cites. The remaining articles had 0-3 cites. But while Cognitive Enhancement is listed as a 2009 paper, it's worth noting that the first draft version of it was posted on Nick Bostrom's website back in 2006, and it has had time to accumulate cites since then. If we exclude it, FHI's 2008-2010 papers have been cited 20 times.
It's not listed in the Achievement Report, but I also want to include the 2008 Whole Brain Emulation Roadmap, which has been cited 15 times, bringing the total count (excluding Cognitive Enhancement) to 35.
Presentations. FHI members have given a total of 95 invited lectures and conference presentations.
Media appearances. Some 100 media appearances, including print, radio, and television appearances, since January 2009. These include BBC television, New Scientist, National Geographic, The Guardian, ITV, Bloomberg News, Discovery Channel, ABC, Radio Slovenia, Wired Magazine, BBC world service, Volkskrant (German newspaper), Utbildningsradion (Swedish national radio), Mehr News Agency (Iranian), Mladina Weekly (Slovenian magazine), Jyllands-Posten and Weekenavisen (Danish newspapers), Bayerisher Rundfunk (German radio), The History Channel, O Estado de São Paulo (Brazillian newspaper), Euronews, Kvallsposten (Swedish newspaper), City Helsinki (Finnish radio), Focus, Dutch Film and Television Academy, The Smart Manager (Indian magazine), Il Sole 24 Ore (Italian monthly), The Bulletin of the Atomic Sciences, Time Magazine, Astronomy Now, and Radio Bar-Kulan (Kenya).
Visitors. "The Institute receives many requests from students and scholars who wish to visit the Institute, only a few of which are accepted because of capacity limitations. The FHI has hosted a number of distinguished academic visitors over the past two years within its various areas of activity, such as Profs. David Chalmers, Michael Oppenheimer, and Thomas Homer-Dixon."
Policy advice. The Achievement Report highlights 23 groups or events which have received policy advice from either Nick Bostrom or Anders Sandberg. These include the World Economic Forum, the Public Services Offices of the Prime Minister's Office of Singapore, the UK Home Office, If (Stockholm insurance company), Jane Street Capital, IARPA (Intelligence Advanced Research Projects Activity) for US Government, The Swedish Institute for Infectious Disease Control, and setting up a research network, "A differential view of enhancement", within the Volkswagen Foundation.
Organized events. Three organized events. 1: Cognitive Enhancement Workshop. 2: Symposium on cognitive enhancement and related ethical and policy issues. 3: Uncertainty, Lags and Nonlinearity: Challenges to governance in a turbulent world.
Publications. The SIAI publications page has 15 papers from the 2008-2010 period, of which 11 are listed under "recent publications", 1 under "software", and 3 under "talks and working papers". Of these, Superintelligence does not imply benevolence has been cited once. The rest all have no citations.
The Sequences were written during this time period. They consist of about a million words, and might very well have a bigger impact than all the other FHI and SIAI articles together - though that's very hard to quantify.
Presentations and Media Appearances. The SIAI blog mentions a number of media appearances and presentations at various venues, but I don't have the energy to go through them all and count. From a quick eyeballing of the blog, though, SIAI has nowhere near as many presentations and media appearances as FHI.
Visitors. The Visiting Fellows page has a list of 27 Visiting Fellows from around the world, who attend or hold degrees from universities including Harvard, Stanford, Yale, Cambridge, Carnegie Mellon, Auckland University, Moscow Institute of Physics and Technology, and the University of California-Santa Barbara
Online communities and tools. Less Wrong was founded in 2009, and Google Analytics says that by the end of 2010, it had had over a million unique visitors.
Note that LW is an interesting case: as an FHI/SIAI collaboration, both organizations claim credit for it. However, since LW is to such a huge extent Eliezer's creation, and I'm not sure of what exactly the FHI contribution to LW is, I'm counting it as an SIAI and not a joint achievement.
SIAI also created the Uncertain Future, a tool for estimating the probability of AI.
Ben Goertzel, acting as the SIAI Director of Research at the time, organized the 2008 and 2009 conferences on Artificial General Intelligence. He also co-organized a 2009 workshop on machine consciousness,
Based on this data, which organization is more deserving of my money? Hard to say, especially since SIAI has been changing a lot. The general AGI research, for instance, isn't really something that's being pursued anymore, and Ben Goertzel is no longer with the organization. Eliezer is no longer writing the sequences, which were possibly the biggest SIAI achievement of the whole 2008-2010 period.
Still, FHI's accomplishments seem a lot more impressive overall, suggesting that they might be a better target for the money. On the other hand, they are not as tightly focused on AI as SIAI is.
One imporant question is also the amount of funding the two organizations have had: accomplishing a lot is easier if you have more money. If an organization has thrice as much money, they should be expected to achieve thrice as much. SIAI's revenue was $426,000 in 2008 and $628,000 in 2009. FHI's funding was around $711,000 for 10/2008 - 10/2009. I don't know the 2010 figure for either organization. The FHI report also says the following:
To appreciate the significance of what has been accomplished, it should be kept in mind that the FHI has been understaffed for much of this period. One of our James Martin Research Fellows, Dr Rebecca Roache, has been on maternity leave for the past year. Our newest James Martin Research Fellow, Dr Eric Mandelbaum, who was recruited from an extremely strong field of over 170 applicants, has been in post for only two months. Thus, for half of the two-year period, FHI’s research staff has consisted of two persons, Professor Nick Bostrom and Dr Anders Sandberg.
SIAI - An Examination notes that in both 2008 and 2009, SIAI paid salaries to three people, so for a while at least, the amount of full-time staff in the two organizations was roughly comparable.
(Updated 2011-12-16 due to a comment by Nick Bostrom.)
'Existential Risk FAQ' by Nick Bostrom
(2011) Version 1.0
Short answers to common questions
'Existential Risk Prevention as the Most Important Task for Humanity' by Nick Bostrom
(2011) Working paper (revised)
Existential risks are those that threaten the entire future of humanity. Many theories of value imply that even relatively small reductions in net existential risk have enormous expected value. Despite their importance, issues surrounding human-extinction risks and related hazards remain poorly understood. In this paper, I clarify the concept of existential risk and develop an improved classification scheme. I discuss the relation between existential risks and basic issues in axiology, and show how existential risk reduction (via the maxipok rule) can serve as a strongly action-guiding principle for utilitarian concerns. I also show how the notion of existential risk suggests a new way of thinking about the ideal of sustainability.