I think the simpler solution is just to use a bounded utility function. There are several things suggesting we do this, and I really don't see any reason to not do so, instead of going through contortions to make unbounded utility work.

Consider the paper of Peter de Blanc that you link -- it doesn't say a computable utility function won't have convergent utilities, but rather that it will iff said function is bounded. (At least, in the restricted context defined there, though it seems fairly general.) You could try to escape the conditions of the theorem, or you could just conclude that utility functions should be bounded.

Let's go back and ask the question of why we're using probabilities and utilities in the first place. Is it because of Savage's Theorem? But the utility function output by Savage's Theorem is always bounded.

OK, maybe we don't accept Savage's axiom 7, which is what forces utility functions to be bounded. But then we can only be sure that comparing expected utilities is the right thing to do for finite gambles, not for infinite ones, so talking about sums converging or not -- well, it's something that shouldn't even come up. Or alternatively, if we do encounter a situation with infinitely many choices, each of differing utility, we simply don't know what to do.

Maybe we're not basing this on Savage's theorem at all -- maybe we simply take probability for granted (or just take for granted that it should be a real number and ground it in something like Cox's theorem -- after all, like Savage's theorem, Cox's theorem only requires that probability be finitely additive) and are then deriving utility from the VNM theorem. The VNM theorem doesn't prohibit unbounded utilities. But the VNM theorem once again only tells us how to handle finite gambles -- it doesn't tell us that infinite gambles should also be handled via expected utility.

OK, well, maybe we don't care about the particular grounding -- we're just going to use probability and utility because it's the best framework we know, and we'll make the probability countably additive and use expected utility in all cases hey, why not, seems natural, right? (In that case, the AI may want to eventually reconsider whether probability and utility really is the best framework to use, if it is capable of doing so.) But even if we throw all that out, we still have the problem de Blanc raises. And, um, all the other problems that have been raised with unbounded utility. (And if we're just using probability and utility to make things nice, well, we should probably use bounded utility to make things nicer.)

I really don't see any particular reason utility has to be unbounded either. Eliezer Yudkowsky seems to keep using this assumption that utility should be unbounded, or just not necessarily bounded, but I've yet to see any justification for this. I can find one discussion where, when the question of bounded utility functions came up, Eliezer responded, "[To avert a certain problem] the bound would also have to be substantially less than 3^^^^3." -- but this indicates a misunderstanding of the idea of utility, because utility functions can be arbitrarily (positively) rescaled or recentered. Individual utility "numbers" are not meaningful; only ratios of utility differences. If a utility function is bounded, you can assume the bounds are 0 and 1. Talk about the value of the bound is as meaningless as anything else using absolute utility numbers; they're not amounts of fun or something.

Sure, if you're taking a total-utilitarian viewpoint, then your (decision-theoretic) utility function has to be unbounded, because you're summing a quantity over an arbitrarily large set. (I mean, I guess physical limitations impose a bound, but they're not logical limitations, so we want to be able to assign values to situations where they don't hold.) (As opposed to the individual "utility" functions that your'e summing, which is a different sort of "utility" that isn't actually well-defined at present.) But total utilitarianism -- or utilitarianism in general -- is on much shakier ground than decision-theoretic utility functions and what we can do with them or prove about them. To insist that utility be unbounded based on total utilitarianism (or any form of utilitarianism) while ignoring the solid things we can say seems backwards.

Not everything has to scale linearly, after all. There seems to be this idea out there that utility must be unbounded because there are constants C_1 and C_2 such that adding to the world of person of "utility" (in the utilitarian sense) C_1 must increase your utility (in the decision-theoretic sense) by C_2, but this doesn't need to be so. This to me seems a lot like insisting "Well, no matter how fast I'm going, I can always toss a baseball forward in my direction at 1 foot per second relative to me; so it will be going 1 foot per second faster than me, so the set of possible speeds is unbounded." As it turns out, the set of possible speeds is bounded, velocities don't add linearly, and if you toss a baseball forward in your direction at 1 foot per second relative to you, it will not be going 1 foot per second faster.

My own intuition is more in line with earthwormchuck163's comment -- I doubt I would be that joyous about making that many more people when so many are going to be duplicates or near-duplicates of one another. But even if you don't agree with this, things don't have to add linearly, and utilities don't have to be unbounded.