Another problem with "Vote down" is that many people seem to be using it as a way of indicating their disagreement with a comment, rather than to indicate that the comment was inappropriate or invalid.

I've always felt that a valid use of the karma system is to vote up things that you believe are less wrong and vote down things that you believe to be more wrong.