I often seem to run into problems when I use the de facto label for this group. For example, when I say, "I've been hanging out with rationalists lately," I notice that many people immediately go on the defensive. They might ask why you need a group in order to be rational, or they might say that they don't believe that people are inherently rational. Of course, I made none of those claims, I simply indicated that I was hanging out with rationalists.
You might think that "rationalist" is simply a descriptive label, but it carries positive connotations -- and what people tend to hear is "I'm a superior thinker to you," or maybe "I'm a part of this group, which ascribes the label 'rationalist' to itself, to make ourselves seem higher status than we really are."
Why does this matter?
The community doesn't exist in a vacuum; how the community is viewed from the outside matters. As the community grows and as people gain awareness of it, branding becomes important. People talk to each other, and communities gain reputations. Even if you believe that we are a loose collection of individuals, as soon as you assign a name to yourself, that is sufficient to form a group identity.
The people we interact with tend to share similar interests. The population of New York may be in the millions, and yet I run into the same people at different functions without coordination.
The more negative perceptions associated with a group, the more rapidly evaporative cooling of groups will occur.
What to do?
It's far better to talk about good things that you've gained from being in the group. It's better to say what the group does, not what the group is.
But beyond that, it's about time the community picked a better label to use. I have one idea, but I'll hold off on proposing solutions.
A somehow arbitrary name is good, because it allows better compartmentalization.
Just like "Mensa" is better than "people more intelligent than you", or "Toastmasters" is better than "people who can speak better than you", also "LessWrongians" is better than "people who are more rational than you".
Of course we usually don't say "people who are more rational than you"; we say "rationalists" instead... but saying it differently does not prevent the audience from decoding the (real or percieved) original meaning.
Being a member of a group with arbitrary name is a hobby. Being a member of a group with some property X in the name suggests that your environment is somehow non-X or not-enough-X, otherwise you would not need such group.
It's easy to start seeing "LessWrong" as an arbitrary label if you're used to the name, but it totally does have a confrontational meaning in the way that "Mensa" and "Toastmasters" don't and in the way that "more rational than you" does.