If you (or anyone) want to argue that women naturally have lower IQ than men, I would go with an argument based on hormones instead.
Where do you think the differences in hormone levels come from?
Food, genes, certain types of activity such as sports and competitiveness in general, the environment you grow up in, being in a position of authority, to name some factors that influence hormone production.
It's certainly not just the gender divide. If you think that testosterone makes men smarter than women on average, you would also have to accept the conclusion that women with more testosterone than men will be smarter than men on average. All other things being equal, of course.
A long blog post explains why the author, a feminist, is not comfortable with the rationalist community despite thinking it is "super cool and interesting". It's directed specifically at Yvain, but it's probably general enough to be of some interest here.
http://apophemi.wordpress.com/2014/01/04/why-im-not-on-the-rationalist-masterlist/
I'm not sure if I can summarize this fairly but the main thrust seems to be that we are overly willing to entertain offensive/taboo/hurtful ideas and this drives off many types of people. Here's a quote:
The author perceives a link between LW type open discourse and danger to minority groups. I'm not sure whether that's true or not. Take race. Many LWers are willing to entertain ideas about the existence and possible importance of average group differences in psychological traits. So, maybe LWers are racists. But they're racists who continually obsess over optimizing their philanthropic contributions to African charities. So, maybe not racists in a dangerous way?
An overly rosy view, perhaps, and I don't want to deny the reality of the blogger's experience. Clearly, the person is intelligent and attracted to some aspects of LW discourse while turned off by other aspects.