It may be worth noting that "6 downvotes" need not mean that 6 people downvoted you. LW2 has "weighted voting" which means that the number of points your upvotes/downvotes change the victim's karma by depends on your own karma level. So maybe you were downvoted twice by weight-3 users, or three times by weight-2 users; in any case, losing 6 points probably corresponds to <6 downvotes.
I don't disagree with weighted votes per se, but saying "6 downvotes" really is misleading, if there were actually less than 6 votes.
I wonder whether it would be helpful to display the karma in analog form, for example as a line, where longer line would mean more total votes (not in linear proportion, but in a way where infinite number of votes asymptotically corresponds to the full width of the used part of page), and the line has a green part and gray part whose length ratio represents the upvotes/downvotes ratio. And of course tooltip for number, but the idea is that people would get the right idea without seeing the numbers.
What are your first impressions of the public beta?