Now that you mention that, that's true, and it gives me several other weird ideas. The box gives you tokens that you exchange for utilons, which seem like they are supposed to be defined as "Whatever you want/define them to be, based on your values."
Ergo, imagine a Happy Michaelos that gets about twice as much positive utilons from everything compared to Sad Michaelos. Sad Michaelos gets twice as much NEGATIVE utilons from everything compared to Happy Michaelos.
Let's say a cookie grants Happy Michaelos 1 utilons. It would take two cookies to grant Sad Michaelos 1 utilons. Let's say a stubbed toe grants Sad Michaelos -1 utilons. It would take two stubbed toes to grant Happy Michaelos -1 utilons.
So if Happy Michaelos or Sad Michaelos gets to open the box and they are friends who substantially share utility and cookies... It should be Sad Michaelos who does so (both will get more cookies that way.)
As far as I can tell, this is a reasonable interpretation of the box.
So, I should probably figure out how the people below would work, since they are increasingly unreasonable interpretations of the box:
Extremely Sad Michaelos:
Is essentially 1 million times worse off than Sad Michaelos. Ergo, it the logic above holds, Extremely Sad Michaelos gets 2 million cookies from turning in a single token.
Hyper Pout Michaelos:
Is essentially 1 billion times worse off than Sad Michaelos. He also has a note in his utility function that he will receive -infinity(aleph 0) utilons if he does not change his utility function back to Sad Michaelos's utility function within 1 second after the box is powerless and he has converted all of his tokens. If the logic above holds, Hyper Pout Michaelos gets 1 billion times more cookies than Sad Michaelos, and then gets to enjoy substantially more utilons from them!
Omnidespairing Michaelos:
Is almost impossible to grant utilons to. The certainty of omnipotence grants him 1 utilon. Everything else that might be positive (say, a 99% chance of omnipotence) grants him 0 utilons.
This is a coherent utility function. You can even live and have a normal life with it if you also want to avoid negative utilons (eating might only grant -infinite (aleph 0) utilons and not eating might grant -infinite (aleph 1) utilons.
Box Cynical Despairmax Michaelos:
Gets some aleph of negative infinite utilons from every decision whatsoever. Again, he can make decisions and go throughout the day, but any number of the tokens that the box grants don't seem to map to anything relevant on his utility function. For instance, waiting a day might cost him - infinite (aleph 2) utilons. Adding a finite number of utilons is irrelevant. He immediately opens the box so he can discard the useless tokens and get back to avoiding the incomprehensible horrors of life, and this is (as far as I can tell) a correct answer for him.
It seems like at least some of the utility functions above cheat the box, but I'm not sure which ones go to far, if the sample is reasonable. They all give entirely different answers as well:
1: Go through life as sad as possible.
2: Go through life pretending to be sad to get more and then actually be happy later.
3: Only omnipotence will make you truly happy. Anything else is an endless horror.
4: Life is pain, and the box is trying to sell you something useless, ignore it and move on.
If changing my utility function has expected positive results, based both on my current utility function and in the proposed change, then...
Here the problem is that the utilon is not a unit that can be converted into any other unit, including physical phenomena.
Let's say you have a box that has a token in it that can be redeemed for 1 utilon. Every day, its contents double. There is no limit on how many utilons you can buy with these tokens. You are immortal. It is sealed, and if you open it, it becomes an ordinary box. You get the tokens it has created, but the box does not double its contents anymore. There are no other ways to get utilons.
How long do you wait before opening it? If you never open it, you get nothing (you lose! Good day, sir or madam!) and whenever you take it, taking it one day later would have been twice as good.
I hope this doesn't sound like a reductio ad absurdum against unbounded utility functions or not discounting the future, because if it does you are in danger of amputating the wrong limb to save yourself from paradox-gangrene.
What if instead of growing exponentially without bound, it decays exponentially to the bound of your utility function? If your utility function is bounded at 10, what if the first day it is 5, the second 7.5, the third 8.75, etc. Assume all the little details, like remembering about the box, trading in the tokens, etc, are free.
If you discount the future using any function that doesn't ever hit 0, then the growth rate of the tokens can be chosen to more than make up for your discounting.
If it does hit 0 at time T, what if instead of doubling, it just increases by however many utilons will be adjusted to 1 by your discounting at that point every time of growth, but the intervals of growth shrink to nothing? You get an adjusted 1 utilon at time T - 1s, and another adjusted 1 utilon at T - 0.5s, and another at T - 0.25s, etc? Suppose you can think as fast as you want, and open the box at arbitrary speed. Also, that whatever solution your present self precommits to will be followed by the future self. (Their decision won't be changed by any change in what times they care about)
EDIT: People in the comments have suggested using a utility function that is both bounded and discounting. If your utility function isn't so strongly discounting that it drops to 0 right after the present, then you can find some time interval very close to the present where the discounting is all nonzero. And if it's nonzero, you can have a box that disappears, taking all possible utility with it at the end of that interval, and that, leading up to that interval, grows the utility in intervals that shrink to nothing as you approach the end of the interval, and increasing the utility-worth of tokens in the box such that it compensates for whatever your discounting function is exactly enough to asymptotically approach your bound.
Here is my solution. You can't assume that your future self will make the optimal decision, or even a good decision. You have to treat your future self as a physical object that your choices affect, and take the probability distribution of what decisions your future self will make, and how much utility they will net you into account.
Think if yourself as a Turing machine. If you do not halt and open the box, you lose and get nothing. No matter how complicated your brain, you have a finite number of states. You want to be a busy beaver and take the most possible time to halt, but still halt.
If, at the end, you say to yourself "I just counted to the highest number I could, counting once per day, and then made a small mark on my skin, and repeated, and when my skin was full of marks, that I was constantly refreshing to make sure they didn't go away...
...but I could let it double one more time, for more utility!"
If you return to a state you have already been at, you know you are going to be waiting forever and lose and get nothing. So it is in your best interest to open the box.
So there is not a universal optimal solution to this problem, but there is an optimal solution for a finite mind.
I remember reading a while ago about a paradox where you start with $1, and can trade that for a 50% chance of $2.01, which you can trade for a 25% chance of $4.03, which you can trade for a 12.5% chance of $8.07, etc (can't remember where I read it).
This is the same paradox with one of the traps for wannabe Captain Kirks (using dollars instead of utilons) removed and one of the unnecessary variables (uncertainty) cut out.
My solution also works on that. Every trade is analogous to a day waited to open the box.