You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Ghatanathoah comments on Desires You're Not Thinking About at the Moment - Less Wrong Discussion

1 Post author: Ghatanathoah 20 February 2013 09:41AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (37)

You are viewing a single comment's thread. Show more comments above.

Comment author: Ghatanathoah 21 February 2013 12:09:35AM -1 points [-]

By "something serious", do you mean that you think that a change in terminal goals would require them to become mentally disabled, or just that it would require a fairly substantial change?

I think probably the latter. But regardless of which it is, I think that anyone would regard such a change as highly undesirable, since it is kind of hard to pursue one's terminal goals if the person you've changing into no longer has the same goals.

If the latter, then reversing the change seems cruel to the modified person if they are still functional.

You're right of course. It would be similar to if a person died, and the only way to resurrect them would be to kill another still living person. The only situation where it would be desirable would be if the changed person inflicted large disutilities on others, or the original created huge utilities for others. For instance, if a brilliant surgeon who saves dozens of lives per year is changed into a cruel sociopath (the sociopath is capable of functioning perfectly fine in society, they're just evil) who commits horrible crimes, reversing the change would be a no-brainer. But in a more normal situation you are right that it would be cruel.