the initial program does whatever it likes (including in particular simulation of the original AI), and then outputs a number, which the AI aims to control.

I don't think this can be right. I expect it's impossible to create a self-contained abstract map of the world (or human value, of which the world is an aspect), the process of making observations has to be part of the solution.

(But even if we are talking about a "number", what kind of number is that? Why would something simple like a real number be sufficient to express relevant counterfactual utility values that are to be compared? I don't know enough to make such assumptions.)