9eB1 comments on Some alternatives to “Friendly AI” - Less Wrong

19 Post author: lukeprog 15 June 2014 07:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (44)

You are viewing a single comment's thread.

Comment author: 9eB1 15 June 2014 09:29:28PM *  8 points [-]

I'm more worried that a term like "safe AGI" could provoke a response of "So you're trying to make sure that a system which is smarter than humans, and able to operate in arbitrary real-world environments, and able to invent new technologies to achieve its goals, will be safe? Let me save you some time and tell you right now that's impossible. Your research program is a pipe dream."

If someone has this reaction, then can't you just say "mission accomplished" and not worry about it too much? In any case, I think "AI safety" is probably the most beneficial to your goals. I would also not be too worried about AI researchers having a knee-jerk response to the term, for the same reasons you do.

Comment author: Mestroyer 15 June 2014 10:57:41PM 2 points [-]

I agree. "AGI Safety"/"Safe AGI" seems like the best option. if people say, "Let me save you some time and tell you right now that's impossible" half of the work is done. The other half is just convincing them that we have to do it anyway because otherwise everyone is doomed. (This is of course, as long as they are using "impossible" in a loose sense. If they aren't, the problem can probably be fixed by saying "our definition of safety is a little bit more loose than the one you're probably thinking of, but not so much more loose that it becomes easy").

Comment author: VAuroch 16 June 2014 11:29:42PM -1 points [-]

Yeah, my kneejerk reaction to someone saying "Fat chance, that's impossible." is to retort "Should we be trying to make an unsafe AI and hoping to reap the benefits of our superintelligence even if it's not guaranteed to destroy us all, or should we be trying to stop everyone else in the entire world from doing that? Because that seems just as impossible."