And there's pressure to have some amount of sex. It may be relevant that asexuals started coming out after homosexuals did. This could be because their situation was less desperate, but I think not liking sex at all is in some ways considered weirder than liking non-standard sex.
Also, virginity past a certain age (varies by sub-culture and by gender) is considered odd.
I think the real compulsion isn't exactly to restrict sex, it's to have rules about sex.
It may be relevant that asexuals started coming out after homosexuals did.
I don't think asexuals were really in the closet until relatively recently. After all, many denominations of Christianity provide people with reasonably high status positions that require that the person abstain from sex. Even the denominations that don't have monastic traditions wouldn't look down on someone who abstains from sex. It wasn't until the sexual liberation movement promulgated the idea that anyone who isn't interested enough in sex is a prude and probably repressed that asexuality became something unacceptable.
It is a cultural universal that people are discouraged from having sex as often and with as many people as they want to. Every culture I've ever heard of imposes many restrictions on sex. I've never heard of a culture that shames people for being too stingy with sex.
If we assume that culture is adaptive, this means that the human sex drive is too strong for humans in society. Why is this? As sex drive is a phenotypic feature with extraordinarily strong selective pressure, why haven't we evolved to have the proper sex drive?
One reason could be that reduced sex drive is selected for at the level of the group, while higher sex drive is selected for at the level of the individual.