DanielLC comments on AI indifference through utility manipulation - Less Wrong

4 Post author: Stuart_Armstrong 02 September 2010 05:06PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (53)

You are viewing a single comment's thread.

Comment author: DanielLC 07 September 2010 11:00:09PM 0 points [-]

As an alternative, how about telling it that it won't be destroyed? This could have problems. It might assume that a meteor won't hit the Earth, as it would destroy the AI, which won't happen. Perhaps you could tell it that it won't be destroyed by people. Another problem is that it might give its off switch to everyone, figuring it will make them slightly happier to know that they're in control, and it doesn't realize someone is going to press theirs almost immediately.

This is equivalent to setting the utility of it being destroyed to the expected utility.