In terms of natural selection, couldn't homo sapiens be considered a FOOM?

Indeed. The questioner asked after "the size of the smallest program that would constitute a starting point of a recursively self-improving AI", though.

Humanity is not a program - and it is huge.

HP/Apple/Microsoft/Facebook may be better examples. They are smaller systems, they often started very small, they clearly exploded - and it isn't difficult to imagine a company with one figurehead human.