I am not a fan of unbounded utilities, but it is worth noting that most (all?) the problems with unbounded utilties are actually a problem with utility functions that are not integrable with respect to your probabilities. It feels basically okay to me to have unbounded utilities as long as extremely good/bad events are also sufficiently unlikely.
The space of allowable probability functions that go with an unbounded utility can still be closed under finite mixtures and conditioning on positive probability events.
Indeed, if you think of utility functions as coming from VNM, and you a space of lotteries closed under finite mixtures but not arbitrary mixtures, I think there are VNM preferences that can only correspond to unbounded utility functions, and the space of lotteries is such that you can't make St. Petersburg paradoxes. (I am guessing, I didn't check this.)
I think bounded functions are the only computable functions that are integrable WRT the universal semi-measure. I think this is equivalent to de Blanc 2007?
The construction is just the obvious one: for any unbounded computable utility function, any universal semi-measure must assign reasonable probability to a St Petersburg game for that utility function (since we can construct a computable St Petersburg game by picking a utility randomly then looping over outcomes until we find one of at least the desired utility).
I think this argument is cool, and I appreciate how distilled it is.
Basically just repeating what Scott said but in my own tongue: this argument leaves open the option of denying that (epistemic) probabilities are closed under countable combination, and deploying some sort of "leverage penalty" that penalizes extremely high-utility outcomes as extremely unlikely a priori.
I agree with your note that the simplicitly prior doesn't implement leverage penalties. I also note that I'm pretty uncertain myself about how to pull off leverage penalties correctly, assuming they're a good idea (which isn't clear to me).
I note further that the issue as I see it arises even when all utilities are finite, but some are ("mathematically", not merely cosmically) large (where numbers like 10^100 are cosmically large, and numbers like 3^^^3 are mathematically large). Like, why are our actions not dominated by situations where the universe is mathematically large? When I introspect, it doesn't quite feel like the answer is "because we're certain it isn't", nor "because utility maxes out at the cosmological scale", but rather something more like "how would you learn that there may or may not be 3^^^3 hap...
TL;DR: I think that the discussion in this post is most relevant when we talk about the utility of whole universes. And for that purpose, I think a leverage penalty doesn't make sense.
A leverage penalty seems more appropriate for saying something like "it's very unlikely that my decisions would have such a giant impact," but I don't think that should be handled in the utility function or decision theory.
Instead, I'd say: if it's possible to have "pivotal" decisions that affect 3^^^3 people, then it's also possible to have 3^^^3 people in "normal" situations all making their separate (correlated) decisions, eating 3^^^3 sandwiches, and so the stakes of everything are similarly mathematically big.
plus a sense that you should be suspicious that any given action is more likely to get 3^^^3 utility than any other
I think that if utilities are large but bounded, then I feel like everything "adds up to normality"---if there is a way to get 3^^^3 utility, it seems like "maximize option value, figure out what's going on, stay sane" is a reasonable bet for maximizing EV (e.g. by maximizing probability of the great outcome).
Intuitively, this also seems like what you should end up doing even if...
if it's possible to have "pivotal" decisions that affect 3^^^3 people, then it's also possible to have 3^^^3 people in "normal" situations all making their separate (correlated) decisions, eating 3^^^3 sandwiches, and so the stakes of everything are similarly mathematically big.
Agreed.
This seems to put you in a strange position though: you are not only saying that high-value outcomes are unlikely, but that you have no preferences about them. That is, they aren't merely impossible-in-reality, they are impossible-in-thought-experiments.
Perhaps I'm being dense, but I don't follow this point. If I deny that my epistemic probabilities are closed under countable weighted sums, and assert that the hypothesis "you can actually play a St. Petersburg game for n steps" is less likely than it is easy-to-describe (as n gets large), in what sense does that render me unable to consider St. Petersburg games in thought experiments?
How would you learn that there may or may not be a 10^100 future people with our choices as the fulcrum? Why would the same process not generalize? (And if it may happen in the future but not now, is that 0 probability?)
The same process generalizes.
My point was...
My point was that this doesn't seem consistent with anything like a leverage penalty.
I'm not particulalry enthusiastic about "artificial leverage penalties" that manually penalize the hypothesis you can get 3^^^3 happy people by a factor of 1/3^^^3 (and so insofar as that's what you're saying, I agree).
From my end, the core of my objection feels more like "you have an extra implicit assumption that lotteries are closed under countable combination, and I'm not sold on that." The part where I go "and maybe some sufficiently naturalistic prior ends up thinking long St. Petersburg games are ultimately less likely than they are simple???" feels to me more like a parenthetical, and a wild guess about how the weakpoint in your argument could resolve.
(My guess is that you mean something more narrow and specific by "leverage penalty" than I did, and that me using those words caused confusion. I'm happy to retreat to a broader term, that includes things like "big gambles just turn out not to unbalance naturalistic reasoning when you're doing it properly (eg. b/c finding-yourself-in-the-universe correctly handles this sort of thing somehow)", if you have one.)
(My guess is that part of the ...
From my end, the core of my objection feels more like "you have an extra implicit assumption that lotteries are closed under countable combination, and I'm not sold on that." [...] It seems to me that your argument contains a fourth, unlisted assumption, which is that lotteries are closed under countable combination. Do you agree?
My formal argument is even worse than that: I assume you have preferences over totally arbitrary probability distributions over outcomes!
I don't think this is unlisted though---right at the beginning I said we were proving theorems about a preference ordering defined over the space of probability distributions over a space of outcomes . I absolutely think it's plausible to reject that starting premise (and indeed I suggest that someone with "unbounded utilities" ought to reject this premise in an even more dramatic way).
If you're trying to object to some other thing I said about leverage penalties, my guess is that I miscommunicated my position
It seems to me that our actual situation (i.e. my actual subjective distribution over possible worlds) is divergent in the same way as the St Petersburg lottery, at least with respect to quantiti...
Ok, cool, I think I see where you're coming from now.
I don't think this is unlisted though ...
Fair! To a large degree, I was just being daft. Thanks for the clarification.
It seems to me that our actual situation (i.e. my actual subjective distribution over possible worlds) is divergent in the same way as the St Petersburg lottery, at least with respect to quantities like expected # of happy people.
I think this is a good point, and I hadn't had this thought quite this explicitly myself, and it shifts me a little. (Thanks!)
(I'm not terribly sold on this point myself, but I agree that it's a crux of the matter, and I'm sympathetic.)
But at that point it seems much more likely that preferences just aren't defined over probability distributions at all
This might be where we part ways? I'm not sure. A bunch of my guesses do kinda look like things you might describe as "preferences not being defined over probability distributions" (eg, "utility is a number, not a function"). But simultaneously, I feel solid in my ability to use probabliity distributions and utility functions in day-to-day reasoning problems after I've chunked the world into a small finite number of possible acti...
The proof doesn't run for me. The only way I know to be able to rearrange the terms in a infinite series is if the starting starting series converges and the resultant series converges. The series doesn't fullfill the condition so I am not convinced the rewrite is a safe step.
I am a bit unsure about my maths so I am going to hyberbole the kind of flawed logic I read into the proof. Start with series that might not converge 1+1+1+1+1+1... (oh it indeed blatantly diverges) then split each term to have a non-effective addition (1+0)+(1+0)+(1+0)+(1+0)... . Blatantly disregard safety rules about paranthesis messing with series and just treat them as paranthesis that follow familiar rules 1+0+1+0+1+0+1+0+1... so 1+1+1+1... is not equal to itself. (unsafe step leads to non-sense)
With converging series it doesn't matter whether we get "twice as fast" to the limit but the "rate of ascension" might matter to whatever analog a divergent series would have to a value.
Maybe I'm just a less charitable person - it seems very easy to me for someone to say the words "I have unbounded utility" without actually connecting any such referent to their decision-making process.
We can show that there's a tension between that verbal statement and the basic machinery of decision-making, and also illustrates how the practical decision-making process people use every day doesn't act like expected utilities diverge.
And I think the proper response to seeing something like this happen to you is definitely not to double down on the verbal statement that sounded good. It's to stop and think very skeptically about whether this verbal statement fits with what you can actually ask of reality, and what you might want to ask for that you can actually get. (I've written too many posts about why it's the wrong move to want an AI to "just maximize my utility function." Saying that you want to be modeled as if you have unbounded utility [of this sort that lets you get divergent EV] is the same order of mistake.)
If you think people can make verbal statements that are "not up for grabs," this probably seems like gross uncharitableness.
People like talking about extensions of the real numbers, but those don't help you avoid any of the contradictions above. For example, if you want to extend < to a preference order over hyperreal lotteries, it's just even harder for it to be consistent.
I'm a recent proponent of hyperreal utilities. I totally agree that hyperreals don't solve issues with divergent / St. Petersburg-style lotteries. I just think hyperreals are perfect for describing and comparing potentially-infinite-utility universes, though not necessarily lotteries over those universes. (This doesn't contradict Paul; I'm just clarifying.)
Separately, while cases like this do make it feel like we "should give up on probabilities as an abstraction for describing uncertainty," this conclusion makes me feel quite nihilistic about decision-under-uncertainty; I will be utterly shocked if "a totally different kind of machinery to understand or analyze those preferences" is satisfactory.
I think the dominance principle used in this post is too strong and relatively easy to deny. I think that the Better impossibility results for unbounded utilities are actually significantly better.
The examples in this post all work by not just having divergent sums but unbounded single elements. When I look at this, my immediate takeaway is that we need a model that includes time. Outcomes should be of the form , where indicates at which timestep they happen.
You are then allowed to look at infinite sequences iff the are strictly increasing. The sums do not need to converge, but there does need to be a global bound for all individual utilities, i.e. that upper-bounds all .
This avoids all the problems i...
Note: I've turned on two-axis voting for this post to see if it's helpful for this kind of discussion.
Relevant comment from the sequences (I had this in mind when writing parts of the OP but didn't remember who wrote it, and failed to recognize the link because it was about Newcomb's problem):
...Another example: It was argued by McGee that we must adopt bounded utility functions or be subject to "Dutch books" over infinite times. But: The utility function is not up for grabs. I love life without limit or upper bound: There is no finite amount of life lived N where I would prefer a 80.0001% probability of living N years to an 0.00
(Note: I've edited this comment a lot since first posting it, mostly small corrections, improving the definition of the order to be better behaved and adding more points.)
I think orders which satisfy (Symmetric) Unbounded Utilities but not (Weak) Dominance or Homogeneous Mixtures can be relatively nice, so we shouldn't be too upset about giving up (Weak) Dominance and Homogenous Mixtures. Basically no nice order will be sensitive to the kinds of probability rearrangements you've done (which we can of course conclude from your results, if we want "nice" to ...
Maybe some of the problem is coming from trying to extend dominance to infinite combinations of lotteries. If we're saying that the utility function is the thing that witnesses some coherence in the choices we make between lotteries, maybe it makes sense to ask for choices between finite combinations of lotteries but not infinite ones? Any choice we actually make, we end up making by doing some "finitary" sort of computation (not sure what this really means, if anything), and perhaps in particular it's always understandable as a choice between finite lotte...
Another common way out is to assume that any two "infinitely good" outcomes are incomparable, and therefore to reject Dominance.[3] This results in being indifferent to receiving $1 in every world (if the expectation is already infinite), or doubling the probability of all good worlds, which seems pretty unsatisfying.
Why are these unsatisfying? Intuitively, if I already have infinite money (in expectation) then why should I care about getting to infinite + 1 or infinite x 2 money?
The point you mention about all decisions having infinite utility in exp...
This argument is extremely similar to Beckstead and Thomas' argument against Recklessness in A paradox for tiny probabilities and enormous values.
If I understand correctly, this argument also appeared in Eliezer Yudkowsky's post "The Lifespan Dilemma", which itself credits one of Wei Dai's comment here. The argument given in The Lifespan Dilemma is essentially identical to the argument in Beckstead and Thomas' paper.
I spent some time trying to fight these results, but have failed!
Specifically, my intuition said we should just be able to look at the flattened distributions-over-outcomes. Then obviously the rewriting makes no difference, and the question is whether we can still provide a reasonable decision criterion when the probabilities and utilities don't line up exactly. To do so we need some defined order or limiting process for comparing these infinite lotteries.
My thought was to use something like "choose the lottery whose samples look better". For instance, exa...
I really like this post because it directly clarified my position on ethics, namely making me abandon unbounded utilities. I want to give this post a Δ and +4 for doing that, and for being clearly written and fairly short.
One interesting case where this theorem doesn't apply would be if there are only finitely many possible outcomes. This is physically plausible: consider multiplying the maximum data density¹ by the spacetime hypervolume of your future light cone from now until the heat death of the universe.
¹ <https://physics.stackexchange.com/questions/2281/maximum-theoretical-data-density>
I just quickly browsed this post. Based on the overall topic, you might also be interested in these inconsistency results in infinitary utiliatarianism written by my PhD advisor (a set theorist) and his wife (a philosopher).
Your proofs all rely on lotteries over infinite numbers of outcomes. Is that necessary? Maybe a restriction to finite lotteries avoids the paradox.
Thank you for making explicit the idea that the problems with "unbounded utilities" don't even require the existence of a utility function, just a very weak assumption about preference ordering with respect to probability mixtures and the existence of "arbitrarily strong" outcomes.
I should note that this is more or less the same thing that Alex Mennen and I have been pointing out for quite some time, even if the exact framework is a little different. You can't both have unbounded utilities, and insist that expected utility works for infinite gambles.
IMO the correct thing to abandon is unbounded utilities, but whatever assumption you choose to abandon, the basic argument is an old one due to Fisher, and I've discussed it in previous posts! (Even if the framework is a little different here, this seems essentially similar.)
I'm glad t...
Promoted to curated: I've been thinking about unbounded utilities for a while, and while I agree with a bunch of the top commenters on the actual relevant element being the integrateability of utilities, and that this can save some unbounded utility assignments (in a way that feels relevant to my current beliefs on the topic).
I do think nevertheless that this post is the best distillation of a bunch of the impossibility arguments I've seen floating around for the last decade, and I think it is an actually important question when trying to decide how to relate to the future. So curating it seems appropriate.
The examples and results in your post are very interesting and surprising. Thanks for writing this.
I'm inclined to reject the dominance axioms you've assumed, at least for mixtures of infinitely many lotteries. I think stochastic dominance is a more fundamental axiom, avoids inconsistency and doesn't give any obviously wrong answers on finite payoff lotteries (even mixtures of infinitely many lotteries or outcomes with intuitively infinite expected value, including St. Petersburg and Pasadena). See Christian Tarsney's "Exceeding Expectations: Stochastic Do...
This doesn't hold if you restrict to utility functions that asymptote to a finite value, correct?
Very nice, thanks. I agree with others that if one were intent on keeping unbounded utilities, it seems simplest to give up on probability distributions that have infinite support (it seems one can avoid all these paradoxes by restricting oneself to finite-support distributions only). I guess this is similar to what you mean by "abandoning probability itself", but do note that you can keep probability so long as all the supports are finite.
Some people think that they have unbounded utility functions. This isn't necessarily crazy, but it presents serious challenges to conventional decision theory. I think it probably leads to abandoning probability itself as a representation of uncertainty (or at least any hope of basing decision theory on such probabilities). This may seem like a drastic response, but we are talking about some pretty drastic inconsistencies.
This result is closely related to standard impossibility results in infinite ethics. I assume it has appeared in the philosophy literature, but I couldn't find it in the SEP entry on the St. Petersburg paradox so I'm posting it here. (Even if it's well known, I want something simple to link to.)
(ETA: this argument is extremely similar to Beckstead and Thomas' argument against Recklessness in A paradox for tiny probabilities and enormous values. The main difference is that they use transitivity +"recklessness" to get a contradiction whereas I argue directly from "non-timidity." I also end up violating a dominance principle which seems even more surprising to violate, but at this point it's kind of like splitting hairs. I give a slightly stronger set of arguments in Better impossibility results for unbounded utilities.)
Weak version
We'll think of preferences as relations < over probability distributions over some implicit space of outcomes Ω (and we'll identify outcomes with the constant probability distribution). We'll show that there is no relation < which satisfies three properties: Antisymmetry, Unbounded Utilities, and Dominance.
Note that we assume nothing about the existence of an underlying utility function. We don't even assume that the preference relation is complete or transitive.
The properties
Antisymmetry: It's never the case that both A<B and B<A.
Unbounded Utilities: there is an infinite sequence of outcomes X1,X2,X4,X8,… each "more than twice as good" as the last.[1] More formally, there exists an outcome X0 such that:
That is, X1 is not as good as a 12 chance of X2, which is not as good as a 14 chance of X4, which is not as good as a 18 chance of X8... This is nearly the weakest possible version of unbounded utilities.[3]
Dominance: let A0,A1,… and B0,B1,… be sequences of lotteries, and p0,p1,… be a sequence of probabilities that sum to 1. If Ai<Bi for all i, then ∑ipiAi<∑ipiBi.
Inconsistency proof
Consider the lottery X∞=12X0+14X1+18X2+116X4+…
We can write X∞ as a mixture:
By definition X0<12X0+12X1. And for each k, Unbounded Utilities implies that X2k<12X0+12X2k+1. Thus Dominance implies X∞<X∞, contradicting Antisymmetry.
How to avoid the paradox?
By far the easiest way out is to reject Unbounded Utilities. But that's just a statement about our preferences, so it's not clear we get to "reject" it.
Another common way out is to assume that any two "infinitely good" outcomes are incomparable, and therefore to reject Dominance.[4] This results in being indifferent to receiving $1 in every world (if the expectation is already infinite), or doubling the probability of all good worlds, which seems pretty unsatisfying.
Another option is to simply ignore small probabilities, which again leads to rejecting even the finite version of Dominance---sometimes when you mix together lotteries something will fall below the "ignore it" threshold leading the direction of your preference to reverse. I think this is pretty bizarre behavior, and in general ignoring small probabilities is much less appealing than rejecting Unbounded Utilities.
All of these options seem pretty bad to me. But in the next section, we'll show that if the unbounded utilities are symmetric---if there are both arbitrarily good and arbitrarily bad outcomes---then things get even worse.
Strong version
I expect this argument is also known in the literature; but I don't feel like people around LW usually grapple with exactly how bad it gets.
In this section we'll show there is no relation < which satisfies three properties: Antisymmetry, Symmetric Unbounded Utilities, and Weak Dominance.
(ETA: actually I think that even with only positive utilities you already violate something very close to Weak Dominance, which Beckstead and Thomas call Prospect-Outcome dominance. I find this version of Weak Dominance slightly more compelling, but Symmetric Unbounded Utilities is a much stronger assumption than Unbounded Utilities or non-Timidity, so it's probably worth being aware of both versions. In a footnote[5] I also define an even weaker dominance principle that we are forced to violate.)
The properties
Antisymmetry: It's never the case that both A>B and B>A.
Symmetric Unbounded Utilities. There is an infinite sequence of outcomes X1,X−2,X4,X−8,… each of which is "more than twice as important" as the last but with opposite sign. More formally, there is an outcome X0 such that:
That is, a certainty of X1 is outweighed by a 12 chance of X−2, which is outweighed by a 14 chance of X4, which is outweighed by a 18 chance of X−8....
Weak Dominance.[5] For any outcome X, any sequence of lotteries B0,B1,…, and any sequence of probabilities p0,p1,… that sum to 1:
Inconsistency proof
Now consider the lottery X±∞=12X1+14X−2+18X4+116X−8+132X16+…
We can write X±∞ as the mixture:
By Unbounded Utilities each of these terms is <X0. So by Weak Dominance, X±∞<X0.
But we can also write X±∞ as the mixture:
By Unbounded Utilities each of these terms is >X0. So by Weak Dominance X±∞>X0. This contradicts Antisymmetry.
Now what?
As usual, the easiest way out is to abandon Unbounded Utilities. But if that's just the way you feel about extreme outcomes, then you're in a sticky situation.
You could allow for unbounded utilities as long as they only go in one direction. For example, you might be open to the possibility of arbitrarily bad outcomes but not the possibility of arbitrarily good outcomes.[6] But the asymmetric version of unbounded utilities doesn't seem very intuitively appealing, and you still have to give up the ability to compare any two infinitely good outcomes (violating Dominance).
People like talking about extensions of the real numbers, but those don't help you avoid any of the contradictions above. For example, if you want to extend < to a preference order over hyperreal lotteries, it's just even harder for it to be consistent.
Giving up on Weak Dominance seems pretty drastic. At that point you are talking about probability distributions, but I don't think you're really using them for decision theory---it's hard to think of a more fundamental axiom to violate. Other than Antisymmetry, which is your other option.
At this point I think the most appealing option, for someone committed to unbounded utilities, is actually much more drastic: I think you should give up on probabilities as an abstraction for describing uncertainty, and should not try to have a preference relation over lotteries at all.[7] There are no ontologically fundamental lotteries to decide between, so this isn't necessarily so bad. Instead you can go back to talking directly about preferences over uncertain states of affairs, and build a totally different kind of machinery to understand or analyze those preferences.
ETA: replacing dominance
Since writing the above I've become more sympathetic to violations of Dominance and even Weak Dominance---it would be pretty jarring to give up on them, but I can at least imagine it. I still think violating "Very Weak Dominance"[5] is pretty bad, but I don't think it captures the full weirdness of the situation.
So in this section I'll try to replace Weak Dominance by a principle I find even more robust: if I am indifferent between X and any of the lotteries Ai, then I'm also indifferent between X and any mixture of the lotteries Ai. This isn't strictly weaker than Weak Dominance, but violating it feels even weirder to me. At any rate, it's another fairly strong impossibility result constraining unbounded utilities.
The properties
We'll work with a relation ≤ over lotteries. We write A=B if both A≤B and B≤A. We write A<B if A≤B but not A=B. We'll show that ≤ can't satisfy four properties: Transitivity, Intermediate mixtures, Continuous symmetric unbounded utilities, and Indifference to homogeneous mixtures.
Intermediate mixtures. If A<B, then A<12A+12B<B.
Transitivity. If A≤B and B≤C then A≤C.
Continuous symmetric unbounded utilities. There is an infinite sequence of lotteries X1,X−2,X4,X−8,… each of which is "exactly twice as important" as the last but with opposite sign. More formally, there is an outcome X0 such that:
That is, a certainty of X1 is exactly offset by a 12 chance of X−2, which is exactly offset by a 14 chance of X4, which is exactly offset by a 18 chance of X−8....
Intuitively, this principle is kind of like symmetric unbounded utilities, but we assume that it's possible to dial down each of the outcomes in the sequence (perhaps by mixing it with X0) until the inequalities become exact equalities.
Homogeneous mixtures. Let X be an outcome, A0,A1,A2,…, a sequence of lotteries, and p0,p1,p2,… be a sequence of probabilities summing to 1. If Ai=X for all i, then ∑piAi=X.
Inconsistency proof
Consider the lottery X±∞=12X1+14X−2+18X4+116X−8+132X16+…
We can write X±∞ as the mixture:
By Unbounded Utilities each of these terms is =X0. So by homogeneous mixtures, X±∞=X0.
But we can also write X±∞ as the mixture:
By Unbounded Utilities each of these terms other than the first is =X0. So by Homogenous Mixtures, the combination of all terms other than the first is =X0. Together with the fact that X1>X0, Intermediate Mixtures and Transitivity imply X±∞>X0. But that contradicts X±∞=X0.
Note that we could replace "more than twice as good" with "at least 0.00001% better" and obtain exactly the same result. You may find this modified version of the principle more appealing, and it is closer to non-timidity as defined in Beckstead and Thomas. Note that the modified principle implies the original by applying transitivity 100000 times, but you don't actually need to apply transitivity to get a contradiction, you can just apply Dominance to a different mixture.
You may wonder why we don't just write X0<X1. If we did this, we'd need to introduce an additional assumption that if A<B, pA+(1−p)X0<pB+(1−p)X0. This would be fine, but it seemed nicer to save some symbols and make a slightly weaker assumption.
The only plausibly-weaker definition I see is to say that there are outcomes X0<X1 and an infinite sequence X>2,X>3,... such that for all n: 1nX>n+(1−1n)X0>X1. If we replaced the > with = then this would be stronger than our version, but with the inequality it's not actually sufficient for a paradox.
To see this, consider a universe with three outcomes X0,X1,X∞ and a preference order < that always prefers lotteries with higher probability of X∞ and breaks ties using by preferring a higher probability of X1. This satisfies all of our other properties. It satisfies the weaker version of the axiom by taking X>n=X∞ for all n, and it wouldn't be crazy to say that it has "unbounded" utilities.
For realistic agents who think unbounded utilities are possible, it seems like they should assign positive probability to encountering a St. Petersburg paradox such that all decisions have infinite expected utility. So this is quite a drastic thing to give up on. See also: Pascal's mugging.
I find this principle pretty solid, but it's worth noting that the same inconsistency proof would work for the even weaker "Very Weak Dominance": for any pair of outcomes with X0<X1, and any sequence of lotteries Bi each strictly better than X1, any mixture of the Bi should at least be strictly better than X0!
Technically you can also violate Symmetric Unbalanced Utility while having both arbitrarily good and arbitrarily bad outcomes, as long as those outcomes aren't comparable to one another. For example, suppose that worlds have a real-valued amount of suffering and a real-valued amount of pleasure. Then we could have a lexical preference for minimizing expected suffering (considering all worlds with infinite expected suffering as incomparable), and try to maximize pleasure only as a tie-breaker (considering all worlds with infinite expected pleasure as incomparable).
Instead you could keep probabilities but abandon infinite probability distributions. But at this point I'm not exactly sure what unbounded utilities means---if each decision involves only finitely many outcomes, then in what sense do all the other outcomes exist? Perhaps I may face infinitely many possible decisions, but each involves only finitely many outcomes? But then what am I to make of my parent's decisions while raising me, which affected my behavior in each of those infinitely many possible decisions? It seems like they face an infinite mixture of possible outcomes. Overall, it seems to me like giving up on infinitely big probability distributions implies giving up on the spirit of unbounded utilities, or else going down an even stranger road.