One of the simplest and most intuitive is aggregative utilitarianism, in which we define a utility for each person, add them all together, and make the choice with the larger total utility.

I suggest using the phrase "additive utilitarianism" rather than "aggregative utilitarianism". It was entirely my fault for saying aggregative utilitarianism in my comment, which was a misnomer; I got it mixed up with aggregative consequentialism. (All flavors of utilitarianism are by definition aggregative because they take into account the utilities from some collection of beings, but not all flavors are additive.)

Note: VincentYu has pointed out in the comments below that VNM utility may be able to deal with the infinites in this problem without taking limits.

Unfortunately, I think that ascribes too much power to VNM utility functions (that term itself is a LessWrongism; elsewhere, they would be called cardinal utility functions or just utility functions). If we had our hands on a VNM utility function, we would be okay (we simply ask it which option it prefers!), but the VNM theorem simply asserts the existence of a utility function given certain basic axioms, and it doesn't give us the utility function! So, unfortunately, VNM utility also falls flat on its face unless we already know what we prefer. (An important point is that VNM utility functions cannot work with the "utility" described in the problem. It's an unfortunate historical accident that the word "utility" is overloaded, because VNM utility requires careful handling.)

If we fail to specify the exact type of decision theory we’re using, it is entirely unclear whether taking infinite limits would lead to a self-consistent solution.

If we want to make a decision based on [additive] utility, the infinite problem is ill posed; it has no unique solution unless we take on additional assumptions.

Great, I think these are good clarifications!