There may well be an inconsistency, but that particular example doesn't seem to exploit it yet...
U=aPresent+bFutureExpectations
agree, die
U1=aPresent+b*[Expected Future with me Dead]
agree, live
U2=present+b*[alive and $10 richer]
refuse
U3=present+b*[Expected Future with me Alive]
U3 > [.5U2 + .5U1] do not take the deal,
With the cake, however, ExpectedValueFromFutureCake = Null in the case that I am dead, which renders the entire utility function irrelevant. (within the system of the other comment)
Eat cake now (dead or alive) Ux = CakeRightNow + 0
Eat cake later Uy = 0 + ExpectedValueFromFutureCake
Die without eating cake Uz = 0 + null, therefore irrelevant
Ux < Uy so do not eat the cake
What I didn't mention before - as I've outlined it, this utility function won't ever get to eat the cake, since the expected future value is always greater. So there's that flaw. I'm not sure whether this signals that the utility function is silly, or that the cake is silly...maybe both.
However, my utility function is only silly in that you can't even eat the cake before near certain death - I'm guessing your model would have you eat the cake as soon as the probability of your death crossed a certain threshold. But if you were immortal and the cake was just sitting in front of you in all its ever-increasing utility, when would you eat it? The cake will generate a paradox - you always expect more from the cake in the future, yet you will never eat the cake (and once you realize this, your expectations from the cake should drop down to zero - which means you might as well eat it now, but if you wait just a bit longer...)
I think the cake breaks everything and we aught to not use it.
Dying without eating cake surely has a utility. I mean, suppose I know I'm going to die tomorrow. I still assign different utilities to different ways I could spend today, I don't say the utility of today is null in all cases.
Or are you saying that it's possible to have a silly utility function that doesn't assign any value to eating the cake before dying compared to not eating the cake and then dying at the same time? Sure, but that utility function is silly.
Imagine that the universe is approximately as it appears to be (I know, this is a controversial proposition, but bear with me!). Further imagine that the many worlds interpretation of Quantum mechanics is true (I'm really moving out of Less Wrong's comfort zone here, aren't I?).
Now assume that our universe is in a situation of false vacuum - the universe is not in its lowest energy configuration. Somewhere, at some point, our universe may tunnel into true vacuum, resulting in a expanding bubble of destruction that will eat the entire universe at high speed, destroying all matter and life. In many worlds, such a collapse need not be terminal: life could go one on a branch of lower measure. In fact, anthropically, life will go on somewhere, no matter how unstable the false vacuum is.
So now assume that the false vacuum we're in is highly unstable - the measure of the branch in which our universe survives goes down by a factor of a trillion every second. We only exist because we're in the branch of measure a trillionth of a trillionth of a trillionth of... all the way back to the Big Bang.
None of these assumptions make any difference to what we'd expect to see observationally: only a good enough theory can say that they're right or wrong. You may notice that this setup transforms the whole universe into a quantum suicide situation.
The question is, how do you go about maximising expected utility in this situation? I can think of a few different approaches: