My concerns about the SIAI are mostly about their competence. It seems rather easy for me to imagine another organisation in the SIAI's niche doing a much better job. Are 63 chapters of a Harry Potter fanfic really helping, for instance?
That isn't an SIAI thing, that's Eliezer's thing. But if you really want to know, it seems from anecdotal evidence that HPMR is helping raise the general sanity waterline. Not only has it made more people be interested in LW in general, I can personally attest to it helping modify irrational beliefs that friends have had.
(Also, Tim, I know you are very fond of capitalizing "DOOM" and certain other phrases, but the rest of us find it distracting and disruptive. Could you please consider not doing it here?)
Also, if they think using fear of THE END OF THE WORLD is a good way to stimulate donations, I would be very interested to see information about the effect on society of such marketing. Will it produce a culture of fear? What about The risks of caution?
I'm not sure why you think they think that doomsday predictions are a good way to stimulate donations. They are simply being honest in their goals. Empirically, existential risk is not a great motivator for getting money. Look for example at how much trouble people concerned with asteroid impacts have getting money (although now that the WISE survey is complete we're in much better shape in understanding and handling that risk.)
My general impression is that spreading the DOOM virus around is rarely very constructive. It may well be actively harmful.
So should people not say what they are honestly thinking?
In financial markets, prophesying market crashes may actually help make them happen, since the whole system works like a big rumour mill - and if a crash is coming, it makes sense to cash in and buy gold - and, if everyone does that, then the crash happens. A case of the self-fulfilling prophesy. The prophet may look smug - but if only they had kept their mouth shut!
Yes, that can happen in markets. What is the analogy here? Is there a situation where simply talking about the risk of unFriendly AI will somehow make unFriendly AI more likely? (And note, improbable decision-theory basilisks don't count.)
Have the DOOM merchants looked into this kind of thing? Where are their reassurances that prophesying DOOM - and separating passing punters from their cash in the process - is a harmless pass-time, with no side effects?
If your standard is that they have to be clear there are no side effects, that's a pretty high standard. How certain do they need to be? To return to the asteroid example, thanks to the WISE mission we now are tracking about 95% of all asteroids that could pose an extinction threat if they impacted, and are tracking a much higher percentage of those that live in severely threatening orbits. But, whenever we spend any money it means we might be missing that small percentage. We'll feel really stupid if our donations to any cause turn out not to matter because we missed another one. If a big asteroid hits the Earth tomorrow we'll feel really dumb. By the same token, we'll feel really stupid if tomorrow someone makes an approximation of AIXI devoted to playing WoW that goes foom. The fact that we have the asteroids charted won't make any difference. No matter how good an estimate we do, there's a chance we'll be wrong. And no matter what happens there are side effects, simply due at minimum to the fact that we have a finite set of resources. And the more we talk about any issue the less we are focusing on others. And yes, obviously if fooming turns out to not be an issue, there will be negative side effects. So where is the line?
Michael Anissimov posted the following on the SIAI blog:
Thanks to the generosity of two major donors; Jaan Tallinn, a founder of Skype and Ambient Sound Investments, and Edwin Evans, CEO of the mobile applications startup Quinly, every contribution to the Singularity Institute up until January 20, 2011 will be matched dollar-for-dollar, up to a total of $125,000.
Interested in optimal philanthropy — that is, maximizing the future expected benefit to humanity per charitable dollar spent? The technological creation of greater-than-human intelligence has the potential to unleash an “intelligence explosion” as intelligent systems design still more sophisticated successors. This dynamic could transform our world as greatly as the advent of human intelligence has already transformed the Earth, for better or for worse. Thinking rationally about these prospects and working to encourage a favorable outcome offers an extraordinary chance to make a difference. The Singularity Institute exists to do so through its research, the Singularity Summit, and public education.
We support both direct engagements with the issues as well as the improvements in methodology and rationality needed to make better progress. Through our Visiting Fellows program, researchers from undergrads to Ph.Ds pursue questions on the foundations of Artificial Intelligence and related topics in two-to-three month stints. Our Resident Faculty, up to four researchers from three last year, pursues long-term projects, including AI research, a literature review, and a book on rationality, the first draft of which was just completed. Singularity Institute researchers and representatives gave over a dozen presentations at half a dozen conferences in 2010. Our Singularity Summit conference in San Francisco was a great success, bringing together over 600 attendees and 22 top scientists and other speakers to explore cutting-edge issues in technology and science.
We are pleased to receive donation matching support this year from Edwin Evans of the United States, a long-time Singularity Institute donor, and Jaan Tallinn of Estonia, a more recent donor and supporter. Jaan recently gave a talk on the Singularity and his life at a entrepreneurial group in Finland. Here’s what Jaan has to say about us:
“We became the dominant species on this planet by being the most intelligent species around. This century we are going to cede that crown to machines. After we do that, it will be them steering history rather than us. Since we have only one shot at getting the transition right, the importance of SIAI’s work cannot be overestimated. Not finding any organisation to take up this challenge as seriously as SIAI on my side of the planet, I conclude that it’s worth following them across 10 time zones.”
– Jaan Tallinn, Singularity Institute donor
Make a lasting impact on the long-term future of humanity today — make a donation to the Singularity Institute and help us reach our $125,000 goal. For more detailed information on our projects and work, contact us at institute@intelligence.org or read our new organizational overview.
-----
Kaj's commentary: if you haven't done so recently, do check out the SIAI publications page. There are several new papers and presentations, out of which I thought that Carl Shulman's Whole Brain Emulations and the Evolution of Superorganisms made for particularly fascinating (and scary) reading. SIAI's finally starting to get its paper-writing machinery into gear, so let's give them money to make that possible. There's also a static page about this challenge; if you're on Facebook, please take the time to "like" it there.
(Full disclosure: I was an SIAI Visiting Fellow in April-July 2010.)