I don't have it handy to grab specifics, but Sex at Dawn discusses a couple dozen different cultures in which promiscuity is the norm. Some of those cultures shame the non-promiscuous, to varying extents.
Human society as we currently know it is less than 10,000 years old. That's an incredibly short timescale for major evolutionary changes.
It is a cultural universal that people are discouraged from having sex as often and with as many people as they want to. Every culture I've ever heard of imposes many restrictions on sex. I've never heard of a culture that shames people for being too stingy with sex.
If we assume that culture is adaptive, this means that the human sex drive is too strong for humans in society. Why is this? As sex drive is a phenotypic feature with extraordinarily strong selective pressure, why haven't we evolved to have the proper sex drive?
One reason could be that reduced sex drive is selected for at the level of the group, while higher sex drive is selected for at the level of the individual.