Well, obviously it's far more complicated than one bit; like most brain features, it's built into the structure of the brain in a somewhat or totally distributed fashion, and through some developmental quirk, some or all of that structure develops in a way inappropriate to their DNA and physical layout. The more complex it is, the more I would expect genderqueer and nonbinary people to be common from increasingly nonstandard configurations of whatever that structure looks like as opposed to fairly limited values it could take on (at least the two).

Most trans people I know felt extremely uncomfortable with their sexual characteristics and assigned gender before ever hearing of the concept of a transsexual person; my ex-boyfriend jokes that he really should have figured it out sooner, given how he would devour literally any media that had crossdressing main characters, and he was raised heavily-Orthodox Jewish where the concept was not at all available. This is a pretty significant obstacle to it being a memetic phenomenon in all/most cases.

I would agree that it's to some degree a memetic fad in the case of nonbinary/genderqueer people; definitely a number of people I know slide around somewhat on the gender spectrum in what seems to be a semi-deliberate act of protest against restrictive gender norms rather than particular pain at being called the gender they were raised as. But there are also nonbinary people whose beliefs are much more deeply held, and who feel intense, crippling emotional pain (i.e. are triggered) when referred to as their raised gender rather than their chosen gender. Generally these people find the opposite binary painful to a significantly lesser extent, which supports the idea that they might be physiologically/neurologically indistinguishable from binary trans people, but they're definitely distinct from the weaker category of nonbinary identification. This is probably a necessary stopping point on the path to the inevitable death of socially-constructed gender.

In short, I think your position, while more reasonable than my past one, is mixing up a couple different phenomena and missing some data, and drawing broad, false conclusions as a result.