TimFreeman comments on Friendly to who? - Less Wrong

2 Post author: TimFreeman 16 April 2011 11:43AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (38)

You are viewing a single comment's thread. Show more comments above.

Comment author: TimFreeman 16 April 2011 01:00:13PM *  1 point [-]

Ah. I interpreted the challenge "Oh, so you get to say what 'Friendly' means" to mean that the speaker is objecting that the listener seems to feel that he's entitled to say what the AI is supposed to do. You seem to be having a different question of what's the intended general meaning of the term.

I had taken as given that a "Friendly" AI is, by definition, one that you trust to do what you want, even if it's much smarter and more powerful than you. If your desires are contradictory, it should still do something reasonable.

This leaves "reasonable" and "what you want" undefined. Filling in those definitions with something technically well-defined and getting to the point where you can reasonably expect it to be stable when it's much smarter than it's designers is the crux of the problem.

Wikipedia's definition at http://en.wikipedia.org/wiki/Friendly_artificial_intelligence is: "A Friendly Artificial Intelligence or FAI is an artificial intelligence (AI) that has a positive rather than negative effect on humanity." Of course, this leaves "positive" undefined, which is not any better than leaving "what you want" or "reasonable" undefined.

At the end of the day, you don't need the concept. The question the creators of the AI will ask themselves will be "Do we want to run this AI?"

Comment author: Perplexed 16 April 2011 07:40:52PM 2 points [-]

At the end of the day, you don't need the concept [of Friendly AI]. The question will be "Do we want to run this AI?"

I worry that the question will be "What do you mean we, Tonto?"

Comment author: TimFreeman 17 April 2011 02:04:26AM 0 points [-]

I agree that that's a valid worry, but I intended to look at things from the point of view of the people making the AI so I was expressing a different worry. I edited my post to clarify.