Therefore, they cannot identify "that computer running the code" with "me", and would cheerfully destroy themselves in the pursuit of their goals/reward.

I am curious as to why an AIXI like entity would need to model itself (and all its possible calculations) in order to differentiate the code it is running with the external universe.

The human in charge of a reward channel could work for initial versions, but once its intelligence grew wouldn't it know what was happening (like the box AI example - not likely to work in the long term).