Actually, I find I have the exact opposite problem. I almost never vote. Partly that's because I read Less Wrong through an RSS feed that doesn't even show the vote totals. I only ever vote if, like now, I've gone to the actual site in order to comment.

Even then, I find that I am comparing the quality of Less Wrong posts and comments against the entire corpus of what I read on a daily basis, some of which is great, and some of which is dreck.

So, I tend to only vote when the quality of what is written is extremely good -- enough so that I want to 'reward' it -- or extremely bad, so that I want to punish. The vast majority is in the middle and so I don't bother to vote.