I would argue that when you do this, you owe it to the person you are downvoting to explain WHY you believe they are systematically wrong. A series of downvotes + one helpful comment is far preferable to a simple series of downvotes, even if it costs you karma to do so. As an example:

my response to an apparent troll comment on Brain Preservation

See, just smacking someone without telling them WHY you're smacking them leaves them to all sorts of conjecture as to what happened - if whomever had downvoted 30+ of my posts had left a single comment explaining why, I could have learned from it. As it is, I have no evidence to distinguish retribution from legitimate correction, and no data with which to correct myself even if it IS an attempt at legitimate correction.

Actually, thinking on this further, a series of downvotes plus an immediate comment explaining why is EXACTLY the right behavior - the sudden plunge in karma will get the user's attention, which they can then direct to the reply - the combination of mild social shaming, "score penalizing" and corrective explanation is a quite powerful way to drive home a lesson.