It appears to me that practically all discussion of this assumes that there is such a thing as inherent gender and that it can differ from that suggested by the arrangement of your genitals at birth.

Is it not a common right-wing point of view that transsexuality is a sexual fetish, and that the dichotomy between sex (biological) and gender (social) is a left-wing fiction? This is frequently part of a package of views that is more focused on criticism of homosexuality and feminism, but I remember encountering critics of transsexuality that were gay or gay-sympathetic. I wasn't able to find anything like that the past half-hour, though.