But in the single-shot scenario, after it comes down tails, what motivation does an ideal game theorist have to stick to the decision theory?

That's what the problem is asking!

This is a decision-theoretical problem. Nobody cares about it for immediate practical purpose. "Stick to your decision theory, except when you non-rigorously decide not to" isn't a resolution to the problem, any more than "ignore the calculations since they're wrong" was a resolution to the ultraviolet catastrophe.

Again, the point of this experiment is that we want a rigorous, formal explanation of exactly how, when, and why you should or should not stick to your precommitment. The original motivation is almost certainly in the context of AI design, where you don't HAVE a human homunculus implementing a decision theory, the agent just is its decision theory.

Sorry, but I'm not in the habit of taking one for the quantum superteam.

If you're not willing to "take one for the team" of superyous, I'm not sure you understand the implications of "every implementation of you is you."

And I don't think that it really helps to solve the problem;

It does solve the problem, though, because it's a consistent way to formalize the decision so that on average for things like this you are winning.

it just means that you don't necessarily care so much about winning any more. Not exactly the point.

I think you're missing the point here. Winning in this case is doing the thing that on average nets you the most success for problems of this class, one single instance of it notwithstanding.

Plus we are explicitly told that the coin is deterministic and comes down tails in the majority of worlds.

And this explains why you're missing the point. We are told no such thing. We are told it's a fair coin and that can only mean that if you divide up worlds by their probability density, you win in half of them. This is defined.

What seems to be confusing you is that you're told "in this particular problem, for the sake of argument, assume you're in one of the worlds where you lose." It states nothing about those worlds being over represented.

So say it's repeated. Since our observable universe will end someday, there will come a time when the probability of future flips is too low to justify paying if the coin lands tails. Your argument suggests you won't pay, and by assumption Omega knows you won't pay. But then on the previous trial you have no incentive to pay, since you can't fool Omega about your future behavior. This makes it seem like non-payment propagates backward, and you miss out on the whole sequence.

There will never be any more 10K; there is no motivation any more to give 100. Following my precommitment, unless it is externally enforced, no longer makes any sense.

This is the point of the thought experiment.

Omega is a predictor. His actions aren't just based on what you decide, but on what he predicts that you will decide.

If your decision theory says "nah, I'm not paying you" when you aren't given advance warning or repeated trials, then that is a fact about your decision theory even before Omega flips his coin. He flips his coin, gets heads, examines your decision theory, and gives you no money.

But if your decision theory pays up, then if he flips tails, you pay $100 for no possible benefit.

Neither of these seems entirely satisfactory. Is this a reasonable feature for a decision theory to have? Or is it pathological? If it's pathological, how do we fix it without creating other pathologies?

In "The Evil Plutocrat" all parties would probably cooperate to vote the bill down, since otherwise they will be sending a message that they could be played in that (and probably other) fashion, which will deminish their future profits, as other lobbyists would try to play them instead of bribing.

is the decision to give up $100 when you have no real benefit from it, only counterfactual benefit, an example of winning?

No, it's a clear loss.

The only winning scenario is, "the coin comes down heads and you have an effective commitment to have paid if it came down tails."

By making a binding precommitment, you effectively gamble that the coin will come down heads. If it comes down tails instead, clearly you have lost the gamble. Giving the $100 when you didn't even make the precommitment would just be pointlessly giving away money.

Decision theory is an attempt to formalize the human decision process. The point isn't that we really are unsure whether you should leave people to die of thirst, but how we can encode that in an actual decision theory. Like so many discussions on Less Wrong, this implicitly comes back to AI design: an AI needs a decision theory, and that decision theory needs to not have major failure modes, or at least the failure modes should be well-understood.

If your AI somehow assigns a nonzero probability to "I will face a massive penalty unless I do this really weird action", that ideally shouldn't derail its entire decision process.

The beggars-and-gods formulation is the same problem. "Omega" is just a handy abstraction for "don't focus on how you got into this decision-theoretic situation". Admittedly, this abstraction sometimes obscures the issue.

It seems to me the answer becomes more obvious when you stop imagining the counterfactual you who would have won the $10000, and start imagining the 50% of superpositions of you who are currently winning the $10000 in their respective worlds.

Every implementation of you is you, and half of them are winning $10000 as the other half lose $100. Take one for the team.

I think that what really does my head in about this problem is, although I may right now be motivated to make a commitment, because of the hope of winning the 10K, nonetheless my commitment cannot rely on that motivation, because when it comes to the crunch, that possibility has evaporated and the associated motivation is gone. I can only make an effective commitment if I have something more persistent - like the suggested $1000 contract with a third party. Without that, I cannot trust my future self to follow through, because the reasons that I would currently like it to follow through will no longer apply.

MBlume stated that if you want to be known as the sort of person who'll do X given Y, then when Y turns up, you'd better do X. That's a good principle - but it too can't apply, unless at the point of being presented with the request for $100, you still care about being known as that sort of person - in other words, you expect a later repetition of the scenario in some form or another. This applies as well to Eliezer's reasoning about how to design a self-modifying decision agent - which will have to make many future decisions of the same kind.

Just wanting the 10K isn't enough to make an effective precommitment. You need some motivation that will persist in the face of no longer having the possibility of the 10K.

Precommitments are used in decision-theoretic problems. Some people have proposed that a good decision theory should take the action that it would have precommitted to, if it had known in advance to do such a thing. This is an attempt to examine the consequences of that.