I don't see this as a problem, really. The entire point is to have high-value discussions.
High-value discussions here, so far as is apparent to me, seem to be better described as "High-value for modestly wealthy white and ethnic Jewish city-dwelling men, many of them programmers". If it turns out said men get enough out of this to noticeably improve the lives of the huge populations (some of which might even contain intelligent, rational individuals or subgroups), that's all fine and well. But so far, it mostly just sounds like rich programmers signalling at each other.
Which makes me wonder what the hell I'm still doing here; in spite of not feeling particularly welcome, or getting much out of discussions, I haven't felt like not continuing to read and sometimes comment would make a good response. Yet, since I'm almost definitely not going to be able to contribute to a world-changing AI, directly or otherwise, and don't have money to spare for EA or xrisk reduction, I don't see why LW should care. (Ok, so I made a thinly veiled argument for why LW should care, but I also acknowledged it was rather weak.)
But so far, it mostly just sounds like rich programmers signalling at each other.
My LW reading comes out of my Internet-as-television time, and so does Hacker News. The two appear very similar in target audience.
A long blog post explains why the author, a feminist, is not comfortable with the rationalist community despite thinking it is "super cool and interesting". It's directed specifically at Yvain, but it's probably general enough to be of some interest here.
http://apophemi.wordpress.com/2014/01/04/why-im-not-on-the-rationalist-masterlist/
I'm not sure if I can summarize this fairly but the main thrust seems to be that we are overly willing to entertain offensive/taboo/hurtful ideas and this drives off many types of people. Here's a quote:
The author perceives a link between LW type open discourse and danger to minority groups. I'm not sure whether that's true or not. Take race. Many LWers are willing to entertain ideas about the existence and possible importance of average group differences in psychological traits. So, maybe LWers are racists. But they're racists who continually obsess over optimizing their philanthropic contributions to African charities. So, maybe not racists in a dangerous way?
An overly rosy view, perhaps, and I don't want to deny the reality of the blogger's experience. Clearly, the person is intelligent and attracted to some aspects of LW discourse while turned off by other aspects.