The issue is the NRs are loud, where as LW's pro-enlightenment majority is not so loud about being pro-enlightenment. I mean, when Eliezer said what you just said in your first paragraph, Michael Anissimov whined about how deeply shocked he was that Eliezer would be distancing himself from the NRs. He could only act shocked because Eliezer's statement was so unusual. Mostly people sort of ignore them, or watch them in vague amusement (something I've done a lot of on Twitter, I confess), with Yvain putting a lot of energy into discussing them in very respectful terms.
My theory is they get a relatively respectful hearing because they're meta-contrarian rather than uneducated, but as Yvain says in that post you can't tell the truth of a position by where it lies in the signaling game. Meta-contrarian crackpots are still crackpots, and LessWrong should be striving not to be associated with crackpottery even when it comes in a cute meta-contrarian wrapper. On the other hand, it should especially avoid that when people report being turned off from LessWrong by a specific form of crackpottery.
A long blog post explains why the author, a feminist, is not comfortable with the rationalist community despite thinking it is "super cool and interesting". It's directed specifically at Yvain, but it's probably general enough to be of some interest here.
http://apophemi.wordpress.com/2014/01/04/why-im-not-on-the-rationalist-masterlist/
I'm not sure if I can summarize this fairly but the main thrust seems to be that we are overly willing to entertain offensive/taboo/hurtful ideas and this drives off many types of people. Here's a quote:
The author perceives a link between LW type open discourse and danger to minority groups. I'm not sure whether that's true or not. Take race. Many LWers are willing to entertain ideas about the existence and possible importance of average group differences in psychological traits. So, maybe LWers are racists. But they're racists who continually obsess over optimizing their philanthropic contributions to African charities. So, maybe not racists in a dangerous way?
An overly rosy view, perhaps, and I don't want to deny the reality of the blogger's experience. Clearly, the person is intelligent and attracted to some aspects of LW discourse while turned off by other aspects.