I saw your comment; the last section ('Beyond Solomonoff?') speaks to the worry you raised. Somewhere in AIXI's hypothesis space is a reasoner R that is a reductionist about R; AIXI can simulate human scientists, for example. But nowhere in AIXI's hypothesis space is a reasoner that is a native representation of AIXI as 'me', as the agent doing the hypothesizing.

One way I'd put this is that AIXI can entertain every physical hypothesis, but not every indexical hypothesis. Being able to consider all the objective ways the world could be doesn't mean you're able to consider all the ways you could be located in the world.

AIXI's hypothesis space does include experts on AIXI that could give it advice about how best to behave like a naturalist. Here the problem isn't that the hypotheses are missing, but that they don't look like they'll be assigned a reasonable prior probability.