If there is more to be gained by adding an additional AI then there is to be gained by scaling up the individual AI, then the best strategy for the AI is to create more AI's with the same utility function.

Edited to add: Unless, perhaps, the AI had an explicit dislike of creating others, in which case it would be a matter of which effect was stronger.