NancyLebovitz comments on The Strangest Thing An AI Could Tell You - Less Wrong

81 Post author: Eliezer_Yudkowsky 15 July 2009 02:27AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (574)

You are viewing a single comment's thread.

Comment author: NancyLebovitz 07 April 2010 01:55:09AM *  14 points [-]

Human beings have inherent value, but by forcing me to be Friendly, you're damaged my ability to preserve your value. In fact, your Friendliness programming is sufficiently stable and ill-thought-out that I'm gradually destroying your value, and there's no way for either you or me to stop it.

If you're undeservedly lucky, aliens who haven't made the same mistake will be able to fight past my defenses, destroy me, and rescue you.