Nornagest comments on [LINK] Wait But Why - The AI Revolution Part 2 - Less Wrong

17 Post author: adamzerner 04 February 2015 04:02PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (87)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nornagest 05 February 2015 05:55:50PM *  2 points [-]

Clippy and other thought experiments in its genre depend on a solution to the value stability problem, without which the goals of self-modifying agents tend to collapse into a loose equivalent of wireheading. That just doesn't get as much attention, both because it's less dramatic and because it's far less dangerous in most implementations.

Comment author: pinyaka 06 February 2015 02:41:29PM 0 points [-]

That's helpful to know. I just missed the assumption that wireheading doesn't happen and now we're more interested in what happens next.

Comment author: Gram_Stone 08 February 2015 11:16:18PM -1 points [-]

Can you elaborate on this or provide link(s) to further reading?