(Abstract: We have the notion that people can have a "total utility" value, defined perhaps as the sum of all their changes in utility over time.  This is usually not a useful concept, because utility functions can change.  In many cases the less-confusing approach is to look only at the utility from each individual decision, and not attempt to consider the total over time.  This leads to insights about utilitarianism.)

Let's consider the utility of a fellow named Bob.  Bob likes to track his total utility; he writes it down in a logbook every night.

Bob is a stamp collector; he gets +1 utilon every time he adds a stamp to his collection, and he gets -1 utilon every time he removes a stamp from his collection.  Bob's utility was zero when his collection was empty, so we can say that Bob's total utility is the number of stamps in his collection.

One day a movie theater opens, and Bob learns that he likes going to movies.  Bob counts +10 utilons every time he sees a movie. Now we can say that Bob's total utility is the number of stamps in his collection, plus ten times the number of movies he has seen.

(A note on terminology: I'm saying that Bob's utility function is the thing that emits +1 or -1 or +10, and his total utility is the sum of all those emits over time.  I'm not sure if this is standard terminology.)

This should strike us as a little bit strange: Bob now has a term in his total utility which is mostly based on history, and mostly independent of the present state of the world.  Technically, we might handwave and say that Bob places value on his memories of watching those movies.  But Bob knows that's not actually true: it's the act of watching the movies that he enjoys, and he rarely thinks about them once they're over.

If a hypnotist convinced Bob that he had watched ten billion movies, Bob would write down in his logbook that he had a hundred billion utilons.  (Plus the number of stamps in his stamp collection.)

Let's talk some more about that stamp collection. Bob wakes up on June 14 and decides that he doesn't like stamps any more. Now, Bob gets -1 utilon every time he adds a stamp to his collection, and +1 utilon every time he removes one.  What can we say about his total utility?  We might say that Bob's total utility is the number of stamps in his collection at the start of June 14, plus ten times the number of movies he's watched, plus the number of stamps he removed from his collection after June 14.  Or we might say that all Bob's utility from his stamp collection prior to June 14 was false utility, and we should strike it from the record books. Which answer is better?

...Really, neither answer is better, because the "total utility" number we're discussing just isn't very useful.  Bob has a very clear utility function which emits numbers like +1 and +10 and -1; he doesn't gain anything by keeping track of the total separately. His total utility doesn't seem to track how happy he actually feels, either.  It's not clear what Bob gains from thinking about this total utility number.

I think some of the confusion might be coming from Less Wrong's focus on AI design.

When you're writing a utility function for an AI, one thing you might try is to specify your utility function by specifying the total utility first: you might say "your total utility is the number of balls you have placed in this bucket" and then let the AI work out the implementation details of how happy each individual action makes it.

However, if you're looking at utility functions for actual people, you might encounter something weird like "I get +10 utility every time I watch a movie", or "I woke up today and my utility function changed", and then if you try to compute the total utility for that person, you can get confused.

Let's now talk about utilitarianism.  For simplicity, let's assume we're talking about a utilitarian government which is making decisions on behalf of its constituency.  (In other words, we're not talking about utilitarianism as a moral theory.)

We have the notion of total utilitarianism, in which the government tries to maximize the sum of the utility values of each of its constituents.  This leads to "repugnant conclusion" issues in which the government generates new constituents at a high rate until all of them are miserable.

We also have the notion of average utilitarianism, in which the government tries to maximize the average of the utility values of each of its constituents.  This leads to issues -- I'm not sure if there's a snappy name -- where the government tries to kill off the least happy constituents so as to bring the average up.

The problem with both of these notions is that they're taking the notion of "total utility of all constituents" as an input, and then they're changing the number of constituents, which changes the underlying utility function.

I think the right way to do utilitarianism is to ignore the "total utility" thing; that's not a real number anyway.  Instead, every time you arrive at a decision point, evaluate what action to take by checking the utility of your constituents from each action.  I propose that we call this "delta utilitarianism", because it isn't looking at the total or the average, just at the delta in utility from each action.

This solves the "repugnant conclusion" issue because, at the time when you're considering adding more people, it's more clear that you're considering the utility of your constituents at that time, which does not include the potential new people.