A long blog post explains why the author, a feminist, is not comfortable with the rationalist community despite thinking it is "super cool and interesting". It's directed specifically at Yvain, but it's probably general enough to be of some interest here.
http://apophemi.wordpress.com/2014/01/04/why-im-not-on-the-rationalist-masterlist/
I'm not sure if I can summarize this fairly but the main thrust seems to be that we are overly willing to entertain offensive/taboo/hurtful ideas and this drives off many types of people. Here's a quote:
In other words, prizing discourse without limitations (I tried to find a convenient analogy for said limitations and failed. Fenders? Safety belts?) will result in an environment in which people are more comfortable speaking the more social privilege they hold.
The author perceives a link between LW type open discourse and danger to minority groups. I'm not sure whether that's true or not. Take race. Many LWers are willing to entertain ideas about the existence and possible importance of average group differences in psychological traits. So, maybe LWers are racists. But they're racists who continually obsess over optimizing their philanthropic contributions to African charities. So, maybe not racists in a dangerous way?
An overly rosy view, perhaps, and I don't want to deny the reality of the blogger's experience. Clearly, the person is intelligent and attracted to some aspects of LW discourse while turned off by other aspects.
Our caveman/cavewomen brains think that we will only ever interact with a very small number of people, and losing the respect of anyone could materially worsen our chances of survival in a crisis. Consequently, many are terrified of public speaking or even of contributing to Internet debates such as on LessWrong. I suspect that the lower you perceive your status to be in the tribe, the greater the fear you will have of further weakening your position by saying something that others criticize.
Some communities go out of their way to create "safe spaces" that limit criticism to attract participants who would otherwise be too fearful to join discussions. LW's implicit philosophy (which I don't disagree with) is that a cost of participating is that you are fair game for blunt criticism. Alas, such a philosophy probably repels some potential participants who would otherwise make intelligent comments.
I face a similar trade-off in my classes. (I teach at a women's college.) Giving honest/blunt feedback during class discussions or on papers can cause a very negative emotional reaction in some students. Interestingly, students who went to high school in Asia are much better (on average) than Americans at handling criticism because they got so much more of it in high school than their American counterparts did, but my Asian students are (on average) far more fearful of public speaking than Americans, because they did so much less of it.
Does it seem at all worrying that your explanation hinges on members of the in-group having a lot of positive characteristics that members of the out-group lack? "We're just too honest and unflinching in the face of criticism. If only the out-group were so gifted!"
There's probably more than one thing going on here; among them some evaporative cooling.