You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Giles comments on Why No Wireheading? - Less Wrong Discussion

16 [deleted] 18 June 2011 11:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (112)

You are viewing a single comment's thread. Show more comments above.

Comment author: Giles 21 June 2011 01:41:26AM 0 points [-]

I agree; the second heuristic doesn't apply particularly well to this scenario. Some terminal values seem to come from a part of the brain which isn't open to introspection, so I'd expect them to arise as a result of evolutionary kludges and random cultural influences rather than necessarily making any logical sense.

The thing is, once we have a value system that's reasonably stable (i.e. what we want is the same as what we want to want) then we don't want to change our preferences even if we can't explain where they arise from.