private_messaging comments on Suggest alternate names for the "Singularity Institute" - Less Wrong

24 Post author: lukeprog 19 June 2012 04:42AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (152)

You are viewing a single comment's thread. Show more comments above.

Comment author: private_messaging 21 June 2012 11:22:32AM *  -2 points [-]

The mission of SI is to mitigate the potential x-risk from a recursively self-improving intelligence, which is rather different from what VInge and Kurzwell had in mind and rather more mundane.

In which way? Recall the EY vs Hansen foom debate. The foom is a singularity by an AI in a basement on commodity hardware, couple weeks to extremely superhuman, and all the 'work' done here assumes something so strongly superhuman you don't need to be concerned with any algorithmic complexity or anything, and it's only goals that matter (and the goals are somehow physical, like number of paperclips or number of staples). Idea taken from Vinge by the way.

If anything, the Singularity Institute is already too broad, as it is only concerned with particularly extreme form of technological singularity.

There's a crucial difference between strongly anti-social process of changing the string to maximize donations, and reasonably good willed name change: in reasonable name change, first you redefine the goals and get some plan then you make a name reflecting the goals (and what you are actually doing). E.g. you change your mind and decide to dedicate some of the work to something like self driving car safety. Then, in light of this broader focus, you come up with the new name "AI safety institute" or something similar. You keep what you're actually planning on doing in sight and you try to summarize it. In the anti-social process, you sit and model the society's response, and you look how society responded, and you try to come up with best string, typically hiding away any specificities behind euphemisms because, ultimately, being specific lets people know more about you quicker.