From memory of recently seeing excerpts from The Polymath: The Life and Opinions of Samuel R. Delany, Gentleman:

Delany spent a while living in a hotel which mostly catered to transsexuals, and he found it unnerving to not know what gender the person he was taking to was. He speculated that wanting to be sure about gender was hard-wired.

After about five weeks, he realized he'd taken elevator rides with people of non-obvious gender, and it didn't bother him at all.