The problem with the specific scenario given, with experimental modification/duplication, rather than careful proof based modification, is that is liable to have the same problem that we have with creating systems this way. The copies might not do what the agent that created them want.
Which could lead to a splintering of the AI, and in-fighting over computational resources.
It also makes the standard assumptions that AI will be implemented on and stable on the von Neumann style computing architecture.
It also makes the standard assumptions that AI will be implemented on and stable on the von Neumann style computing architecture.
Of course, if it's not, it could port itself to such if doing so is advantageous.
ITT we talk about whatever.