AlephNeil comments on Poll: What value extra copies? - Less Wrong

5 [deleted] 22 June 2010 12:15PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (136)

You are viewing a single comment's thread. Show more comments above.

Comment author: AlephNeil 24 June 2010 11:30:30AM 0 points [-]

Ah, I see.

Having a 'limited sphere of consequence' is actually one of the core ideas of deontology (though of course they don't put it quite like that).

Speaking for myself, although it does seem like an ugly hack, I can't see any other way of escaping the paranoia of "Pascal's Mugging".

Comment deleted 24 June 2010 11:47:33PM [-]
Comment author: AlephNeil 25 June 2010 09:22:03AM *  0 points [-]

Certainly, but how is a bounded utility function anything other than a way of sneaking in a 'delimited sphere of consequence', except that perhaps the 'sphere' fades out gradually, like a Gaussian rather than a uniform distribution?

To be clear, we should disentangle the agent's own utility function from what the agent thinks is ethical. If the agent is prepared to throw ethics to the wind then it's impervious to Pascal's Mugging. If the agent is a consequentialist who sees ethics as optimization of "the universe's utility function" then Pascal's Mugging becomes a problem, but yes, taking the universe to have a bounded utility function solves the problem. But now let's see what follows from this. Either:

  1. We have to 'weight' people 'close to us' much more highly than people far away when calculating which of our actions are 'right'. So in effect, we end up being deontologists who say we have special obligations towards friends and family that we don't have towards strangers. (Delimited sphere of consequence.)

  2. If we still try to account for all people equally regardless of their proximity to us, and still have a bounded utility function, then upon learning that the universe is Vast (with, say, Graham's number of people in it) we infer that the universe is 'morally insensitive' to the deaths of huge numbers of people, whoever they are: Suppose we escape Pascal's Mugging by deciding that, in such a vast universe, a 1/N chance of M people dying is something we can live with (for some M >> N >> 1.) Then if we knew for sure that the universe was Vast, we ought to be able to 'live with' a certainty of M/N people dying. And if we're denying that it makes a moral difference how close these people are to us then these M/N people may as well include, say, the citizens of one of Earth's continents. So then if a mad tyrant gives you perfect assurance that they will nuke South America unless you give them your Mars bar (and perfect assurance that they won't if you do) then apparently you should refuse to hand it over (on pain of inconsistency with your response to Pascal's Mugging.)

Comment deleted 25 June 2010 11:02:50AM *  [-]