You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

AlexMennen comments on Desires You're Not Thinking About at the Moment - Less Wrong Discussion

1 Post author: Ghatanathoah 20 February 2013 09:41AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (37)

You are viewing a single comment's thread. Show more comments above.

Comment author: AlexMennen 20 February 2013 11:35:21PM 0 points [-]

I think the conceptual distinction you're trying to make is more like terminal vs instrumental goals rather than general vs specific goals, although they may be correlated. I generally think of "values" as synonymous with "terminal goals", and those are what I was referring to. I agree that people's terminal goals change much less often than instrumental goals.

Generally, I think a person's general values are fairly stable, and it is hard to change them without doing something serious to their brain. I think continuing to respect those values, and maybe trying to repair whatever happened to that person's brain, is probably a good idea.

By "something serious", do you mean that you think that a change in terminal goals would require them to become mentally disabled, or just that it would require a fairly substantial change? If the former, then that is a fairly strong claim, which I would like to see evidence for. If the latter, then reversing the change seems cruel to the modified person if they are still functional.

Comment author: Ghatanathoah 21 February 2013 12:09:35AM -1 points [-]

By "something serious", do you mean that you think that a change in terminal goals would require them to become mentally disabled, or just that it would require a fairly substantial change?

I think probably the latter. But regardless of which it is, I think that anyone would regard such a change as highly undesirable, since it is kind of hard to pursue one's terminal goals if the person you've changing into no longer has the same goals.

If the latter, then reversing the change seems cruel to the modified person if they are still functional.

You're right of course. It would be similar to if a person died, and the only way to resurrect them would be to kill another still living person. The only situation where it would be desirable would be if the changed person inflicted large disutilities on others, or the original created huge utilities for others. For instance, if a brilliant surgeon who saves dozens of lives per year is changed into a cruel sociopath (the sociopath is capable of functioning perfectly fine in society, they're just evil) who commits horrible crimes, reversing the change would be a no-brainer. But in a more normal situation you are right that it would be cruel.