Very interesting.

Some time ago I posted a comment about raising AIs with a caregiver. Basically rules given to the child/AI cause it to search for circumventions whereas rewarding positive behaviors could be modelled as shaping the motivation structure of the child/AI. At least for children positively reinforced behaviors cause searching for new behaviors in that direction and implicitly inhibit other behaviors.

Only the theoretical model you gave for the motivation part does look quite different from my model. The children model seems to work more like heavily (for an advanced AI) penalizing the search outside the rewarded areas. This is different from the usual temporal discounting, so that might nontheless be another AI control approach. Search distance would need to be quantified for this and that is more difficult than time discounting.