timtyler comments on Open Thread June 2010, Part 3 - Less Wrong

6 Post author: Kevin 14 June 2010 06:14AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (606)

You are viewing a single comment's thread. Show more comments above.

Comment author: khafra 15 June 2010 10:28:27AM 3 points [-]

Have you read the Heaven post by denisbider and the two follow-ups constituting a mini-wireheading series? There have been other posts on the difference between wanting and liking; but it illustrates a fairly strong problem with wireheading: Even if all we're worried about is "subjective states," many people won't want to be put in that subjective state, even knowing they'll like it. Forcing them into it or changing their value system so they do want it are ethically suboptimal solutions.

So, it seems to me that if anything other than maximized absolute wireheading for everyone is the AI's goal, it's gonna start to get complicated.

Comment author: multifoliaterose 15 June 2010 11:26:16PM 0 points [-]

Thanks for the references to the posts which I had not seen before and which I find relevant. I'm sympathetic toward denisbider's view, but will read the comments to see if I find diverging views compelling.