One of the things I've noticed is that some people get downvoted for bike shed reasons; if you're making an opaque technical point, your comment's score will sometimes be more indicative of its politeness (which everyone feels competent at assessing) rather than your comment's correctness (which only a few feel competent at assessing).
On a similar note, the established voting pattern in a thread that you join matters a lot---people establish 'sides'. If you are willing to rebut particular arguments from the side that is already winning socially, expect downvotes regardless of merit (even if the comment on net ends up positive there will still be some downvotes according to existing trend.)
I just noticed that if I hover my mouse over the big green dot with my total karma, it says, "81% positive". Presumably 81% of the votes on my posts and/or comments have been positive.
I checked out the % positive for everyone on the all-time top 15 list:
Average = 90.6%, Standard deviation = 4.93%
So I'm 1.95 standard deviations below average for the top 15. Not only am I at the bottom of the list, we would expect me to be at the bottom of the list of the top 39 users. (Assuming these numbers are representative of the top 39 LessWrong users, which is dubious, and that LessWrong users are "normal", which sounds even more dubious, 97.44% of them have a higher upvote/downvote ratio than me.) I've gotten about 6744 down-votes, a bit more than Alicorn's 6711, but still second to Eliezer's 15225.
How should I interpret this? I could say that I'm the most-controversial poster on the top 15 list, and be proud of that. But if I'd had the highest %positive score, I'm sure I'd be proud of that, too. As long as I'm extreme in some way. Or if I were closest to the average, I suppose I could also be proud of that.
Before checking, would you guess that the top 15 have higher, or lower, % positive scores than most users?