The $125,000 Summer Singularity Challenge

Kaj_Sotala

Thanks to the generosity of several major donors^†, every donation to the Singularity Institute made now until August 31, 2011 will be matched dollar-for-dollar, up to a total of $125,000.

Donate now!

(Visit the challenge page to see a progress bar.)

Now is your chance to double your impact while supporting the Singularity Institute and helping us raise up to $250,000 to help fund our research program and stage the upcoming Singularity Summit… which you can register for now!

^† $125,000 in backing for this challenge is being generously provided by Rob Zahra, Quixey, Clippy, Luke Nosek, Edwin Evans, Rick Schwall, Brian Cartmell, Mike Blume, Jeff Bone, Johan Edström, Zvi Mowshowitz, John Salvatier, Louie Helm, Kevin Fischer, Emil Gilliam, Rob and Oksana Brazell, Guy Srinivasan, John Chisholm, and John Ku.

2011 has been a huge year for Artificial Intelligence. With the IBM computer Watson defeating two top Jeopardy! champions in February, it’s clear that the field is making steady progress. Journalists like Torie Bosch of Slate have argued that “We need to move from robot-apocalypse jokes to serious discussions about the emerging technology.” We couldn’t agree more — in fact, the Singularity Institute has been thinking about how to create safe and ethical artificial intelligence since long before the Singularity landed on the front cover of TIME magazine.

The last 1.5 years were our biggest ever. Since the beginning of 2010, we have:

Held our annual Singularity Summit, in San Francisco. Speakers included Ray Kurzweil, James Randi, Irene Pepperberg, and many others.
Held the first Singularity Summit Australia and Singularity Summit Salt Lake City.
Held a wildly successful Rationality Minicamp.
Published seven research papers, including Yudkowsky’s much-awaited ‘Timeless Decision Theory‘.
Helped philosopher David Chalmers write his seminal paper ‘The Singularity: A Philosophical Analysis‘, which has sparked broad discussion in academia, including an entire issue of Journal of Consciousness Studies and a book from Springer devoted to responses to Chalmers’ paper.
Launched the Research Associates program.
Brought MIT cosmologist Max Tegmark onto our advisory board, published our Singularity FAQ, and much more.

In the coming year, we plan to do the following:

Hold our annual Singularity Summit, in New York City this year.
Publish three chapters in the upcoming academic volume The Singularity Hypothesis, along with several other papers.
Improve organizational transparency by creating a simpler, easier-to-use website that includes Singularity Institute planning and policy documents.
Publish a document of open research problems related to Friendly AI, to clarify the research space and encourage other researchers to contribute to our mission.
Add additional skilled researchers to our Research Associates program.
Publish well-researched documents making the case for existential risk reduction as optimal philanthropy.
Diversify our funding sources by applying for targeted grants and advertising our affinity credit card program.

We appreciate your support for our high-impact work. As PayPal co-founder and Singularity Institute donor Peter Thiel said:

“I’m interested in facilitating a forum in which there can be… substantive research on how to bring about a world in which AI will be friendly to humans rather than hostile… [The Singularity Institute represents] a combination of very talented people with the right problem space [they’re] going after… [They’ve] done a phenomenal job… on a shoestring budget. From my perspective, the key question is always: What’s the amount of leverage you get as an investor? Where can a small amount make a big difference? This is a very leveraged kind of philanthropy.”

Donate now, and seize a better than usual chance to move our work forward. Credit card transactions are securely processed through Causes.com, Google Checkout, or PayPal. If you have questions about donating, please call Amy Willey at (586) 381-1801.

From the SingInst blog:

Thanks to the generosity of several major donors^†, every donation to the Singularity Institute made now until August 31, 2011 will be matched dollar-for-dollar, up to a total of $125,000.

Donate now!

(Visit the challenge page to see a progress bar.)

The last 1.5 years were our biggest ever. Since the beginning of 2010, we have:

Held our annual Singularity Summit, in San Francisco. Speakers included Ray Kurzweil, James Randi, Irene Pepperberg, and many others.
Held the first Singularity Summit Australia and Singularity Summit Salt Lake City.
Held a wildly successful Rationality Minicamp.
Published seven research papers, including Yudkowsky’s much-awaited ‘Timeless Decision Theory‘.
Helped philosopher David Chalmers write his seminal paper ‘The Singularity: A Philosophical Analysis‘, which has sparked broad discussion in academia, including an entire issue of Journal of Consciousness Studies and a book from Springer devoted to responses to Chalmers’ paper.
Launched the Research Associates program.
Brought MIT cosmologist Max Tegmark onto our advisory board, published our Singularity FAQ, and much more.

In the coming year, we plan to do the following:

Hold our annual Singularity Summit, in New York City this year.
Publish three chapters in the upcoming academic volume The Singularity Hypothesis, along with several other papers.
Improve organizational transparency by creating a simpler, easier-to-use website that includes Singularity Institute planning and policy documents.
Publish a document of open research problems related to Friendly AI, to clarify the research space and encourage other researchers to contribute to our mission.
Add additional skilled researchers to our Research Associates program.
Publish well-researched documents making the case for existential risk reduction as optimal philanthropy.
Diversify our funding sources by applying for targeted grants and advertising our affinity credit card program.

We appreciate your support for our high-impact work. As PayPal co-founder and Singularity Institute donor Peter Thiel said:

“I’m interested in facilitating a forum in which there can be… substantive research on how to bring about a world in which AI will be friendly to humans rather than hostile… [The Singularity Institute represents] a combination of very talented people with the right problem space [they’re] going after… [They’ve] done a phenomenal job… on a shoestring budget. From my perspective, the key question is always: What’s the amount of leverage you get as an investor? Where can a small amount make a big difference? This is a very leveraged kind of philanthropy.”

Ah; now we begin to compare different things. To compare the effort of pulling the lever, against the utility of the additional lives. At this point, yes, the actual magnitude and not just the sign of the difference between f(A+B-C+D) and f(A+D) becomes important; yet D is unknown and unknowable. This means that the magnitude of the difference can only be known with certainty if f(x) is linear; in the case of a nonlinear f(x), the magnitude cannot be known with certainty. I can easily pick out a nonlinear, monotonically increasing function such that the difference between f(A+B-C+D) and f(A+D) can be made arbitrarily small for any positive integer A, B and C (where A+B>C) by simply selecting a suitable positive integer D. A simple example would be f(x)=sqrt(x).

Now, the hypothetical moral agent is in a quandary. Using effort to pick a solution costs utilions. The cost is a simple, straightforward constant; he known how much that costs. But, with f(x)=sqrt(x), without knowing D, he cannot tell whether the utilions of saving the people is greater or lesser than the utilion cost of picking a solution. (For the purpose of simplicity, I will assume that no-one will ever know that he was in a position to make the choice - that is, his reputation is safe, no matter what he selects). Therefore, he has to make an estimate. He has to guess a value of D. There are multiple strategies that can be followed here:

Try to estimate the most probable value of D. This would require something along the lines of the Drake equation - picking the most likely numbers for the different elements, picking the most likely size of an extraterrestrial civilisation, and doing some multiplication.
Take the most pessimistic possible value of D; D=0. That is, plan as though I am in the worst possible universe; if I am correct, and D=0, then I take the correct action, while if I am incorrect and D later proves greater than zero, then that is a pleasant surprise. This guards against getting an extremely unpleasant surprise if it later turns out that D is substantially lower than the most likely estimate; utilions in the future are more likely to go up than down.
Ignore the cost, and simply take the option that saves the most lives, regardless of effort. This strategy actually reduces the cost slightly (as one does not need to expend the very slight cost of calculating the cost), and has the benefit of allowing immediate action. It is the option that I would prefer that everyone who is not me should take (because if other people take it, then I have a greater chance of getting my life saved at the cost of no effort on my part). I might choose this option out of a sense of fairness (if I wish other people to take this option, it is only reasonable to consider that other people may wish me to take it) or out of a sense of duty (saving lives is important).

Try to estimate the most probable value of D.

More precisely, you take the expected value over your probability distribution for D, i.e. if $\\textstyle \\sum\_\{D=0\}^\{\+\\infty\}\(f\(A\+B\-C\+D\$ -f(A+D))p(D)) exceeds the cost of pulling the lever then you pull it.

ETA: In case you're wondering, I used this to display the equation.

-4MugaSofer13y

Remember, we're trying to approximate human morality here. How may people will kill off billions for a cookie?

32

The $125,000 Summer Singularity Challenge

32

32

32

The $125,000 Summer Singularity Challenge

32

32