I have three children. Does loving one mean that the other two are not special to me?? Does a parent only have enough love for one child? Why should it be so different for lovers?

I didn't understand this line of argument before I was poly, and I don't understand it now. Yes. Of course if you have multiple children they're individually less special to you! You have less time and energy for each, less brain-space to store facts about each, and you aren't even culturally allowed to have a favorite! There's a sense in which you "love them all equally", sure, but I'd be willing to bet that something like 75% of parents would be unable to claim that under Veritaserum.

As for why it should be different for lovers, the psychology about lovers and children is very different. It's a conceit of our current sensibilities that we even use the same word to refer to how we feel about those, our siblings, our pets, and ice cream. There is no reason in principle why we couldn't have been hardwired for extreme strict romantic monogamy and still love lots of children.