Your genes don't want you to know you're the father, they want you to be the father, and female sexual exclusivity helps with that.

Yes, each male's genes would want him to gather a large harem and enforce sexual exclusivity for "his" females, even if he couldn't preferentially contribute resources to his own children. But how is he supposed to accomplish this in an egalitarian forager band?

On the other hand, a female's genes would want her to be sexually exclusive, voluntarily, if that meant her children's father would preferentially contribute resources to them. Otherwise, they prefer sperm competition, since that gives her a better chance of getting the fittest sperm, and increases the genetic diversity of her children. (At least that's my understanding of the authors' logic.)