On the other hand, the mission of the SIAI is founded on the belief that if anyone succeeds at AGI without solving the Friendliness problem, they will destroy the world. Eliezer has said in an interview a year or two back that he does not think that anyone currently working on AGI has any chance of succeeding. But if not now, then some day the question will have to be faced:
What do you do if you really believe that someone's research has a substantial chance of destroying the world?
What do you do if you really believe that someone's research has a substantial chance of destroying the world?
Go batshit crazy.
Something a number of LWers should probably be cautious of.
http://www.nature.com/news/2011/110822/full/476373a.html