Armok_GoB comments on The Backup Plan - Less Wrong

1 Post author: Luke_A_Somers 13 October 2011 07:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (35)

You are viewing a single comment's thread. Show more comments above.

Comment author: Armok_GoB 15 November 2011 08:57:22PM 1 point [-]

possible scenarios:

  • Alice believed that she were probably friendly, that FOOMing would carry a risk of scrambling her utility function, but that she needs to do it anyway because if she slowed down to a safe rate some other unfriendly AI would foom first.

  • Alice is Friendly, but doesn't get certain things as easily as humans, and so she doesn't realize something she's planing to do risks modifying her utility function.