I often seem to run into problems when I use the de facto label for this group. For example, when I say, "I've been hanging out with rationalists lately," I notice that many people immediately go on the defensive. They might ask why you need a group in order to be rational, or they might say that they don't believe that people are inherently rational. Of course, I made none of those claims, I simply indicated that I was hanging out with rationalists.
You might think that "rationalist" is simply a descriptive label, but it carries positive connotations -- and what people tend to hear is "I'm a superior thinker to you," or maybe "I'm a part of this group, which ascribes the label 'rationalist' to itself, to make ourselves seem higher status than we really are."
Why does this matter?
The community doesn't exist in a vacuum; how the community is viewed from the outside matters. As the community grows and as people gain awareness of it, branding becomes important. People talk to each other, and communities gain reputations. Even if you believe that we are a loose collection of individuals, as soon as you assign a name to yourself, that is sufficient to form a group identity.
The people we interact with tend to share similar interests. The population of New York may be in the millions, and yet I run into the same people at different functions without coordination.
The more negative perceptions associated with a group, the more rapidly evaporative cooling of groups will occur.
What to do?
It's far better to talk about good things that you've gained from being in the group. It's better to say what the group does, not what the group is.
But beyond that, it's about time the community picked a better label to use. I have one idea, but I'll hold off on proposing solutions.
One of my least popular comments on Less Wrong was that nobody was a "decent rationalist." Perhaps now is the time to explain what I meant by that.
Rationality is an ideal. Whether or not it's a particularly good ideal, it's definitely not a good description of any actually existing people, which proposition is approximately what this entire site is about. To me, being a "decent rationalist" would entail being decently rational: not perfectly rational, but at least mostly rational. It's clear that nobody approaches that state.
When people describe themselves as "rationalists", perhaps some of them mean that they aspire to the ideal of rationality. But it sounds like they believe that they actually practice rationality. At best, this would be dishonest boasting; at worst it would be self-delusion.
So perhaps that's why people react negatively to the label: they hear it as a claim of an implausible achievement, not a belief system or social group.
(It gets worse when you use the term to identify the social club rather than a rather broad set of beliefs, because then you end up saying that someone is not a "rationalist" or an "objectivist" or a "libertarian". It's sort of like how certain academics now use the term "philosopher" to mean "person teaching philosophy at a university" or "person submitting papers to philosophy journals", by which standard Socrates wasn't a philosopher.)
In the years that I've been watching this social group, I've struggled with the question of what to call it when talking about it to other people. "Eliezer's cult" seems unnecessarily derogatory, as does the tongue-in-cheek "Bayesian Conspiracy". "Yudkowskians" is accurate and not derogatory but perhaps unnecessarily limiting, and surely oversimplified. "Less Wrong" is the best label I have, which would make individuals "Lesswrongers".
There's another possible reason people might react negatively: in the 20th century, any number of atrocities were justified on the basis of being "scientific", "modern", or "rational": dialectical materialism, Levittown, indiscriminate use of pesticides, Mutually Assured Destruction, Schelling's losing strategy in the Vietnam war, low-cost housing developments, childbirth under anesthesia, radium water treatments, lobotomies, electroshock, The Projects, razing neighborhoods to run interstate highways through downtown, IMF neoliberal economic policies, eugenics, and so on.
It turns out that conflating your position with knowledge and rationality, and your opponent's position with ignorance and insanity, is such an effective rhetorical strategy that you can use it to ram through all sorts of terrible ideas. Perhaps because of this, a lot of people have developed a sort of memetic allergic reaction to explicit claims of rationality.