roystgnr comments on Why are we not starting to map human values? - Less Wrong

6 Post author: djm 22 January 2014 02:03PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (29)

You are viewing a single comment's thread. Show more comments above.

Comment author: roystgnr 22 January 2014 10:51:52PM 4 points [-]

Legal systems are (at best) a "satisficer"s map of human values, necessary but insufficient. If you want to spend your life making as many paper clips as possible within the constraints of the law, we'll be disappointed in you but we won't forceably stop you, because it's your life. If an AGI decides it wants the same thing, I we'd rightly consider it a failure, even if we were lucky enough that it wanted to be similarly constrained.