I have the same suspicion as lucidfox.
Perhaps an even stronger bias on my part is that I want to see more people in the world who are like me, i.e., who are interested in and capable of solving some of the same problems as me. When I look back on my life and try to see how I got here, it seems like "not caring much about socializing and social skills" is a big part of it, since that freed up a lot of time and energy to think. I'm worried that things like LW and mini-camp with their message of "being a good rationalist means you should learn social skills" are "killing off" those future people.
Paul Graham made a similar point in "Why Nerds are Unpopular". It's true that socializing doesn't help me solve hard problems, but I still do it because it makes me happier.
Yesterday I attended a meetup where the discussion turned to fashion for a time (because apparently the mini-camp participants were given some instructions on fashion as a useful part of instrumental rationality). (Unfortunately none of us knew much about the topic so the discussion turned into "how can we find an expert to advise us for minimal cost?") It was mentioned that dressing "badly" can be a useful signalling device, and some examples were given. Here's an attempt at a more complete list of possible signals one might be sending by dressing "badly".
The idea here is, if you do decide to start dressing "well", know what you're giving up first. (Of course you're also giving up possibly implying that nobody taught you how to dress and you're not sufficiently strategic to have thought of learning it yourself. Or implying that you don't have the mental, financial, and/or social resources to keep up with fashion. A lot of signaling depends on what your audience already knows about you, or can infer from your other signals.) See also Yvain's related post, Why Real Men Wear Pink and comments there.