I'm not sure that's actually more sensible, given that upvotes and downvotes aren't shown separately (does +3 means three people want to see more comments like that and everyone else is indifferent, or does it mean seven people want to see more of them and four want to see fewer?). If they were I would do that too, but right now I mentally assign each post/comment a “deserved” score and upvote/downvoted if the current score is below/above it.