Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Ghatanathoah comments on Hedonic vs Preference Utilitarianism in the Context of Wireheading - Less Wrong Discussion

6 Post author: jkaufman 29 June 2012 01:50PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (53)

You are viewing a single comment's thread. Show more comments above.

Comment author: Ghatanathoah 21 February 2013 07:35:40AM 0 points [-]

Yes, but only until they're modified. The desire fulfillment of their future selves will outweigh the desire unfulfillment of their present selves, resulting in a net increase in desire fulfillment.

One way this is typically resolved is something called the "prior existence view." This view considers it good to increase the desire fulfillment of those who already exist, and those who will definitely exist in the future, but does not necessarily grant extra points for creating tons of new desires and then fulfilling them. The prior-existence view would therefore hold that it is wrong to create or severely modify a person if doing so would inflict an unduly large disutility on those who would exist prior to that person's creation.

The prior-existence view captures many strong moral intuitions, such as that it is morally acceptable to abort a fetus to save the life of the mother, and that it is wrong to wirehead someone against their will. It does raise some questions, like what to do if raising the utility of future people will change their identity, but I do not think these are unresolvable.