I would guess that the culprit is the difference between speed of technological advance and biological evolution.
As we have become richer in resources and free time, we both have the option to and the tools required for overstimulating any of our "drives" (sex, apetite for eating food, socializing online), without having had the time for individual level evolution to catch up. We see adaptations at the group/cultural level because changes there happen more quickly, and are easier to notice. If the pressure remains sustained for a long enough time, we'll see biological changes, too.
It is a cultural universal that people are discouraged from having sex as often and with as many people as they want to. Every culture I've ever heard of imposes many restrictions on sex. I've never heard of a culture that shames people for being too stingy with sex.
If we assume that culture is adaptive, this means that the human sex drive is too strong for humans in society. Why is this? As sex drive is a phenotypic feature with extraordinarily strong selective pressure, why haven't we evolved to have the proper sex drive?
One reason could be that reduced sex drive is selected for at the level of the group, while higher sex drive is selected for at the level of the individual.