You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

RolfAndreassen comments on The idiot savant AI isn't an idiot - Less Wrong Discussion

8 Post author: Stuart_Armstrong 18 July 2013 03:43PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (133)

You are viewing a single comment's thread. Show more comments above.

Comment author: RolfAndreassen 18 July 2013 09:25:29PM 7 points [-]

To be clear, a badly-programmed AI may modify its own goals. People here usually say "paperclip maximiser" to indicate an AI that is well programmed to maximise paper clips, not any old piece of code that someone threw together with the vague idea of getting some paperclips out of it. However, if an AI is built with goal X, and it self-modifies to have goal Y, then clearly it was not a well-designed X-maximiser.