I thought the answer Vladimir Nesov already posted solved Counterfactual Mugging for a quantum coin?

Basically, all the local decisions come from the same computation that would be performed to set the most general precommitment for all possible states of the world. The expected utility maximization is defined only once, on the global state space, and then the actual actions only retrieve the global solution, given encountered observations. The observations don't change the state space over which the expected utility optimization is defined (and don't change the optimal global solution or preference order on the global solutions), only what the decisions in a given (counterfactual) branch can affect. Since the global precommitment is the only thing that defines the local agents' decisions, the "commitment" part can be dropped, and the agents' actions can just be defined to follow the resulting preference order.

In this solution, there is no belief updating; there is just decision theory. (All probabilities are "timestamped" to the beliefs of the agent's creator when the agent was created.) This means that the use of Bayesian belief updating with expected utility maximization may be just an approximation that is only relevant in special situations which meet certain independence assumptions around the agent's actions. In the more general Newcomb-like family of situations, computationally efficient decision algorithms might use a family of approximations more general than Bayesian updating.

There would, for example, be no such thing as "posterior probability of 'coin comes up heads'" or "probability that you are a Boltzmann brain"; there would only be a fraction of importance-measure that brains with your decision algorithm could affect. As Vladimir Nesov commented:

Agents self-consistent under reflection are counterfactual zombies, indifferent to whether they are real or not.

Anna and I noticed this possible decision rule around four months before Vladimir posted it (with "possible observations" replaced by "partial histories of sense data and actions", and also some implications about how to use limited computing power on "only what the decisions in a given (counterfactual) branch can affect" while still computing predicted decisions on one's other counterfactual branches well enough to coordinate with them). But we didn't write it up to a polished state, partly because we didn't think it seemed enough like it was the central insight in the area. Mostly, that was because this decision rule doesn't explain how to think about any logical paradoxes of self-reference, such as algorithms that refer to each others' output. It also doesn't explain how to think about logical uncertainty, such as the parity of the trillionth digit of pi, because the policy optimization is assumed to be logically omniscient. But maybe we were wrong about how central it was.