if your decision theory pays up, then if he flips tails, you pay $100 for no possible benefit.

But in the single-shot scenario, after it comes down tails, what motivation does an ideal game theorist have to stick to the decision theory?

Like Parfit's hitchhiker, although in advance you might agree that it's a worthwhile deal, when it comes to the point of actually paying up, your motivation is gone, unless you have bound yourself in some other way.

The beggars-and-gods formulation is the same problem.

I don't think so; I think the element of repetition substantially alters it - but in a good way, one that makes it more useful in designing a real-world agent. Because in reality, we want to design decision theories that will solve problems multiple times.

At the point of meeting a beggar, although my prospects of obtaining a gold coin this time around are gone, nonetheless my overall commitment is not meaningless. I can still think, "I want to be the kind of person who gives pennies to beggars, because overall I will come out ahead", and this thought remains applicable. I know that I can average out my losses with greater wins, and so I still want to stick to the algorithm.

In the single-shot scenario, however, my commitment becomes worthless once the coin comes down tails. There will never be any more 10K; there is no motivation any more to give 100. Following my precommitment, unless it is externally enforced, no longer makes any sense.

So the scenarios are significantly different.

This is an attempt to examine the consequences of that.

Yes, but if the artificial scenario doesn't reflect anything in the real world, then even if we get the right answer, therefore what? It's like being vaccinated against a fictitious disease; even if you successfully develop the antibodies, what good do they do?

It seems to me that the "beggars and gods" variant mentioned earlier in the comments, where the opportunity repeats itself each day, is actually a more useful study. Sure, it's much more intuitive; it doesn't tie our brains up in knots, trying to work out a way to intend to do something at a point when all our motivation to do so has evaporated. But reality doesn't have to be complicated. Sometimes you just have to learn to throw in the pebble.

Does this particular thought experiment really have any practical application?

I can think of plenty of similar scenarios that are genuinely useful and worth considering, but all of them can be expressed with much simpler and more intuitive scenarios - eg when the offer will/might be repeated, or when you get to choose in advance whether to flip the coin and win 10000/lose 100. But with the scenario as stated - what real phenomenon is there that would reward you for being willing to counterfactually take an otherwise-detrimental action for no reason other than qualifying for the counterfactual reward? Even if we decide the best course of action in this contrived scenario - therefore what?

I was just re-reading the sequences, and I have to say that as a teacher I really think you're misjudging what is happening here.

Much of learning, it seems, is building up a mental framework, starting from certain concepts and attaching new concepts to them so that they can easily be recalled later and so you can use the connections between concepts to develop your own thinking later..

From my point of view, it looks like the student (perhaps as long as a year ago) had successfully created a new concept node in their mind, "heat conduction". They had connected this node to the concepts of heat transfer and physics. And even though they likely hadn't activated this node at all in perhaps a year, they were able to take a specific example of something they saw in the real world, generalize it to something that might be related to the more general topic of heat transfer in physics, and create a hypothesis of heat conduction.

If you saw a machine learning algorithm that was able to do all that, you'd really be impressed! Something like Watson might be able to go from the concept of heat transfer to heat conduction, but it wouldn't be able to generalize from a specific example of heat transfer it saw in the real world.

Now, they might not yet have a lot of details attached to the "heat conduction" concept node in their head. But that's ok, that they can learn, that gives you something to build on as a teacher. If you teach it well and they can attach some details and images and maybe some math to the concept of "heat conduction" in the head, then hopefully next time they'll say "Maybe heat conduction? Hmmm, no, that doesn't work." which is even better. But there's more going on here then just "guessing a password"; this is part of what constructing a model of the world looks like while the process is only partly completed.

The computations in the second situation have the same fidelity and granularity as the first, and are morally equivalent. Your X is bigger (and therefore takes more to compute) than A(S), exactly because it contains both worlds. At any point in the physics simulation, you can subtract out Rt and get A(S)t.

OK, but this only moves the problem to a different place. Say I can find an operation that let's me transform the data irreversibly, in such a way that I can still get my computations and answers done, but no way of getting the original states back. What would you say then?

It seems to me you are taking my assumption of linearity on the wrong level. To be exact, I need the assumption of linearity of the operator of calculating future-time snapshots (fixed in the article).

This is entirely different from your example.

Imagine for example how the Fourier Transform is linear as an operation.

From the point of view of physics, it contains garbage,

But a miracle occurs, and your physics simulation still works accurately for the individual components...?

I get that your assumption of "linear physics" gives you this. But I don't see any reason to believe that physics is "linear" in this very weird sense. In general, when you do calculations with garbage, you get garbage. If I time-evolve a simulation of (my house plus a bomb) for an hour, then remove all the bomb components at the end, I definitely do not get the same result as running a simulation with no bomb.

And apparently insurance companies can make money because the expected utility of buying insurance is lower than it's price.

No, the expected monetary value of insurance is lower than its price. (Assuming that the insurance company's assessment of your risk level is accurate.) You're equivocating between money and utility, which is the source of your confusion.

Suppose I offered a simple wager: we flip a coin, and if it comes up heads, I give you a million dollars. But if it comes up tails, you owe me a million dollars, and I get every cent you earn until that debt is paid. Is this bet fair?

Monetarily, yes. But even if I skew the odds in your favor a little, maybe 60/40, I'll bet you still don't want to take it. Why not? Isn't an expected return of $200,000 wildly in your favor?

Yeah, but that doesn't matter. An extra million dollars would make your life somewhat better; spending the next twenty years flat broke would make your life drastically worse. Expected utility is very negative.

The utility of money is sometimes claimed to be logarithmic. For small amounts of money you can use a linear approximation, but if the outcome can shift you to a totally different region of the curve, the concavity becomes very important.

Instantaneous action at a distance would be locality.