It's happened to me as well. I argued in favor of an unpopular view and some joker down-voted all of my posts, even ones that had nothing to do with the view in question.

My solution is not to worry so much about karma. Even without the problem of block-downvoting, there are too many other problems with it to make it useful feedback.

Perhaps the block-downvoting problem could be handled by publicizing some of the information about peoples' up and downvotes. On a slightly different note, I would not be surprised at all if it turns out that some posters are operating smite-puppets to downvote their perceived enemies and sock-puppets to upvote their own posts. Or if there are pairs or groups of upvote-allies.