There is a huge and growing body of knowledge about controlling multi-agent systems. Unfortunately, so far as I know, little of it deals with the scenario in which the agents are busily constructing more agents.

That does happen quite a bit in genetic and memetic algorithms - and artificial life systems.