Normal_Anomaly comments on Human errors, human values - Less Wrong

32 Post author: PhilGoetz 09 April 2011 02:50AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (135)

You are viewing a single comment's thread. Show more comments above.

Comment author: Swimmer963 09 April 2011 03:14:21PM 7 points [-]

Most people choose the many dust specks over the torture. Some people argued that "human values" includes having a utility aggregation function that rounds tiny (absolute value) utilities to zero, thus giving the "dust specks" answer. No, Eliezer said; this was an error in human reasoning. Is it an error, or a value?

I'm not sure. I think the answer most people give on this has more to do with fairness than rounding to zero. Yeah, it's annoying for me to get a dust speck in my eye, but it's unfair that someone should be tortured for 50 years just to spare me (and 3^^^3 others) from dust specks. I would choose getting a dust speck in my eye over someone else being tortured, and I think most people are similar enough to me that I can assume the same of the other 3^^^3 people.

Comment author: Normal_Anomaly 09 April 2011 04:17:22PM 1 point [-]

This matches my intuition on the subject. It also matches my intuition of the problem of Nozick's Utility Monster. Yes, total utility will be maximized if we let the monster eat everyone, but it introduces a large disparity: huge utility for the monster, huge disutility for everyone else.

The question is, is this a "valid value" or a problem? The only way I can see to answer this is to ask if I would self-modify to not caring about fairness, and I don't know the answer.