Isn't this just the AI reflection problem?

The problem is that an AI needs to model itself, which seems exactly the problem that you're talking about.

The answer obviously has something to do with the fact that your utility program isn't perfect if running on real hardware, so it will approximate its own effect on utility.

Or, if the question means something else, the answer could be the thing TDT does, and the program gives you the utility assuming that it doesn't run. This probably isn't what a utility function is for, though; we likely want it to calculate utility for states where it does exist, in which case the reflection problem needs to be solved.