Be sure to consider the possibility of the worlds spontaneously constructing the agent in some epistemic state, or dissolving it. Also, when a (different) agent thinks about our agent, it might access a statement about the agent's strategy that involves many different epistemic states. For this reason, the agent's strategy controls many more worlds than where the agent is instantiated "normally". This makes the problem of figuring out which of the world programs contain the agent very non-trivial, depending on what state of the agent are we talking about, and what kind of worlds are we considering, and not just by the order in which the agent program expects observations.

These considerations made me write off Bayesian updating as a non-fundamental technique that shouldn't be shoehorned into a more general decision theory for working with arbitrary preference. I currently suspect that there is no generally applicable simple trick, and FAI decision theory should instead seek to clarify the conceptual issues, and then work on optimizing brute force algorithms that follow from that picture. Think abstract interpretation, not variational mean field.