I often seem to run into problems when I use the de facto label for this group. For example, when I say, "I've been hanging out with rationalists lately," I notice that many people immediately go on the defensive. They might ask why you need a group in order to be rational, or they might say that they don't believe that people are inherently rational. Of course, I made none of those claims, I simply indicated that I was hanging out with rationalists.
You might think that "rationalist" is simply a descriptive label, but it carries positive connotations -- and what people tend to hear is "I'm a superior thinker to you," or maybe "I'm a part of this group, which ascribes the label 'rationalist' to itself, to make ourselves seem higher status than we really are."
Why does this matter?
The community doesn't exist in a vacuum; how the community is viewed from the outside matters. As the community grows and as people gain awareness of it, branding becomes important. People talk to each other, and communities gain reputations. Even if you believe that we are a loose collection of individuals, as soon as you assign a name to yourself, that is sufficient to form a group identity.
The people we interact with tend to share similar interests. The population of New York may be in the millions, and yet I run into the same people at different functions without coordination.
The more negative perceptions associated with a group, the more rapidly evaporative cooling of groups will occur.
What to do?
It's far better to talk about good things that you've gained from being in the group. It's better to say what the group does, not what the group is.
But beyond that, it's about time the community picked a better label to use. I have one idea, but I'll hold off on proposing solutions.
This is a bit of a Red Queen's Race: as the "arbitrary" proper noun becomes associated with property X, people start to respond to it as a generic referent to property X. If I want to avoid those responses by this strategy, I end up having to discard one term after another after another, always looking for a term that people don't have a referent for. It's kind of the opposite of clear communication, and gets tedious after a couple of decades.
It's also possible to turn this around. I was a member of an aphasics support group for a while (while recovering from brain damage) and in general this got framed, not as "my environment is insufficiently supportive of aphasics" but "I need more support for dealing with my aphasia than my environment provides." The difference is at best subtle, but it's frequently the difference between people feeling accused of inadequacy and not.
For what it's worth: I've been to a few Mensa gatherings (a coworker of mine was an active member and invited me regularly) and tend to think of Mensa, not as "people more intelligent...", but people who care more about being intelligent. I was a Toastmasters member for several years, I tend to think of Toastmasters as people who care more about speaking well. I've been intermittently active here for a while, and tend to think of the folks here as people who care more about rationality (and in some cases about behaving rationally). If that perception were more ubiquitous it might help... being implicitly accused of not caring enough about positive trait X is less of a challenge than being implicitly accused of lacking X.
This depends on how widely the name is known. "Mensa" is known enough, so many people have associations with it. When a random person on a street will know who "LessWrongians" are, then... well, hopefully at that time the waterline of sanity will be higher than today. But until then, "LessWrongians" means nothing to most people; and if necessary, you can always reframe it as a group of Yudkowsky's fanfic fans.