Currently there is (as far as I know) a downvoting limit of 4x your total karma, which seems to serve the same purpose. It's possible, of course, that this limit needs to be lower. But if, say, the limit were half your total karma, then it wouldn't change much to additionally charge karma for downvotes: all karma would remain the same to within a factor of 2.

Of course, charging karma for downvotes would probably also discourage downvoting, because people don't like their numbers going down. I'm not sure if this is a good thing, though; incentives mainly work well for people who are already using the system as intended, so this would discourage legitimate downvoting while leaving retributive downvoting alone.