And if a culture endorsed killing most male babies (or with modern medical technology, conceived only girls most of the time), and implemented polygamy, they'd be using their child-raising resources efficiently and would increase their growth rate linearly with the number of women.

Thinking about this some more: the reason this isn't a historical winner is rather obvious - a mostly female population can't field a big army and is invaded by young males from other groups. Also, the women have incentives to move to a different group where they could have a man all to themselves (or to invite foreign men), so a few men would have to effectively rule and police a mostly female society.