You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Nornagest comments on [LINK] Wait But Why - The AI Revolution Part 2 - Less Wrong Discussion

17 Post author: adamzerner 04 February 2015 04:02PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (87)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nornagest 05 February 2015 05:55:50PM *  2 points [-]

Clippy and other thought experiments in its genre depend on a solution to the value stability problem, without which the goals of self-modifying agents tend to collapse into a loose equivalent of wireheading. That just doesn't get as much attention, both because it's less dramatic and because it's far less dangerous in most implementations.

Comment author: pinyaka 06 February 2015 02:41:29PM 0 points [-]

That's helpful to know. I just missed the assumption that wireheading doesn't happen and now we're more interested in what happens next.

Comment author: Gram_Stone 08 February 2015 11:16:18PM -1 points [-]

Can you elaborate on this or provide link(s) to further reading?