(nods) I'm happy to posit that the trillion ViliamBur-clones, identical or not, genuinely are better off; otherwise of course the entire thing falls apart. (This isn't just "happy," and it's hard to say exactly what it is, but whatever it is I see no reason to believe it's logically incompatible with some people just being better at it than others. In LW parlance, we're positing that ViliamBur is much better at having Fun than everybody else. In traditional philosophical terms, we're positing that ViliamBur is a Utility Monster.)
Their copies do not exist yet -- so it makes sense to create trillion copies of A, and kill everyone else including (the single copy of) B
No.
That the copies do not exist yet is irrelevant.
The fact that you happened to express the wish is irrelevant, let alone when you did so.
What matters is the expected results of various courses of action.
In your original scenario, what was important was that the expected result of bulk-replicating you was that the residents of the universe are subsequently better off. (As I say, I reluctantly endorse the FAI doing this even against your stated wishes.) In the modified scenario where B is even more of a Utility Monster than you are, it bulk-replicates B instead. If the expected results of bulk-replicating A and B are equipotential, it picks one (possibly based on other unstated relevant factors, or at random if you really are equipotential).
Incidentally, one of the things I had to ignore in order to accept your initial scenario was the FAI's estimated probability that, if it doesn't wipe everyone else out, sooner or later someone even more utility-monsterish than you (or B) will be born. Depending on that probability, it might not bulk-replicate either of you, but instead wait until a suitable candidate is born. (Indeed, a utilitarian FAI that values Fun presumably immediately gets busy constructing a species more capable of Fun than humans, with the intention of populating the universe with them instead of us.)
But generally the idea was that calculations about "happiness for most people" can be manipulated if some group of people desires great reproduction (assuming their children will mostly inherit their preferences), which gradually increases the importance of wishes of given group.
Again, calculations about utility (which, again, isn't the same as happiness, though it's hard to say exactly what it is) have absolutely nothing to do with wishes in the sense you're using the term here (that is, events that occur at a particular time). It may have something to do with preferences, to the extent that the FAI is a preference utilitarian... that is, if its calculations of utility are strongly contingent on preference-having entities having their preferences satisfied, then it will choose to satisfy preferences.
Even the world ruled by utilitarian Friendly AI would allow fights between groups, where the winning strategy is to "wish for a situation, where it is utilitarian to help us and to destroy our enemies".
Again, no. Wishing for a situation as a strategic act is completely irrelevant. Preferring a situation might be, but it is very odd indeed to refer to an agent having a strategic preference... strategy is what I implement to achieve whatever my preferences happen to be. For example, if I don't prefer to populate the universe with clones of myself, I won't choose to adopt that preference just because adopting that preference will make me more successful at implementing it.
That said, yes, the world ruled by utilitarian FAI will result in some groups being successful instead of others, where the winning groups are the ones whose existence maximizes whatever the FAI's utility definition is.
In such world, the outside-hateful inside-loving hugely reproducing groups with preserved preferences would have an "evolutionary advantage", so they would gradually destroy everyone else.
If they don't have corresponding utility-inhibiting factors, which I see no reason to believe they necessarily would, yes, that's true. Well, not necessarily gradually... they might do so immediately.
Is this important?
Indeed, a utilitarian FAI that values Fun presumably immediately gets busy constructing a species more capable of Fun than humans, with the intention of populating the universe with them instead of us.
Oh.
I would hope that the FAI would instead turn us into the species most capable of fun. But considering the remaining time of the universe and all the fun the new species will have there, the difference between (a) transforming us or (b) killing us and creating the other species de novo, is negligible. The FAI would probably choose the faster solution, be...
This is based on a discussion in #lesswrong a few months back, and I am not sure how to resolve it.
Setup: suppose the world is populated by two groups of people, one just wants to be left alone (labeled Jews), the other group hates the first one with passion and want them dead (labeled Nazis). The second group is otherwise just as "good" as the first one (loves their relatives, their country and is known to be in general quite rational). They just can't help but hate the other guys (this condition is to forestall the objections like "Nazis ought to change their terminal values"). Maybe the shape of Jewish noses just creeps the hell out of them, or something. Let's just assume, for the sake of argument, that there is no changing that hatred.
Is it rational to exterminate the Jews to improve the Nazi's quality of life? Well, this seems like a silly question. Of course not! Now, what if there are many more Nazis than Jews? Is there a number large enough where exterminating Jews would be a net positive utility for the world? Umm... Not sure... I'd like to think that probably not, human life is sacred! What if some day their society invents immortality, then every death is like an extremely large (infinite?) negative utility!
Fine then, not exterminating. Just send them all to concentration camps, where they will suffer in misery and probably have a shorter lifespan than they would otherwise. This is not an ideal solutions from the Nazi point of view, but it makes them feel a little bit better. And now the utilities are unquestionably comparable, so if there are billions of Nazis and only a handful of Jews, the overall suffering decreases when the Jews are sent to the camps.
This logic is completely analogous to that in the dust specks vs torture discussions, only my "little XML labels", to quote Eliezer, make it more emotionally charged. Thus, if you are a utilitarian anti-specker, you ought to decide that, barring changing Nazi's terminal value of hating Jews, the rational behavior is to herd the Jews into concentration camps, or possibly even exterminate them, provided there are enough Nazi's in the world who benefit from it.
This is quite a repugnant conclusion, and I don't see a way of fixing it the way the original one is fixed (to paraphrase Eliezer, "only lives worth celebrating are worth creating").
EDIT: Thanks to CronoDAS for pointing out that this is known as the 1000 Sadists problem. Once I had this term, I found that lukeprog has mentioned it on his old blog.