I think it used to be more complicated than that, especially for men. My impression is that men who weren't interested in sex were admired by some but considered abnormal by more people. Still, that's just an impression.