But even if so libertarian ideals let alone practice are something we've been on net moving away from them on most matters in the past century at least, but moved towards them when it came to sexuality, this seems an anomaly.

In the United States at least, prostitution and incest were legal until the twentieth century. The trend towards consensual sex tolerance is young and could easily reverse.