I've never been into science fiction, fantasy literature, anime, or D&D, but the alleged popularity of those on LW has never bothered me, nor even particularly stood out. (I mean, I've liked a few instances of the first three, but never anywhere near the point of being involved in their respective subcultures.)

(Admittedly, I am a nerd along several other dimensions — e.g. I'm a computer/math/science person — but those are a lot more actually relevant to understanding and contributing to LW than anime and sci-fi/fantasy are.)