If a user's history is controversial (both upvoted and downvoted) versus only downvoted, then punishing you for downvoting all (90%+) of their comments (if they have more than a few) is completely justified.
At any rate, here is an extra filter to prevent false positives even further - if you look at the comments where only the offender has downvoted and you see neutral comments (those which would have neither been downvoted nor upvoted normally) there, then you know there is a problem.
You're missing the point, you can downvote a comment because you want to see fewer such comments. There is no reason -- absent rules -- that couldn't extend to all of a poster's comments if you want to see fewer comments by that person. There are benefits to sending clear signals. This isn't some official process -- it's simply an expression of your personal preferences, not anyone else's. Your "offender" and "punishment" fixation when conflating "I don't like this behavior" with "this behavior must be punished" is a bit frustrating, given we have rules on a host of issues, but no rule yet on this one. It's simply using your own karma.
Anyways, I'm signing off on this.
Below is a message I just got from jackk. Some specifics have been redacted 1) so that we can discuss general policy rather than the details of this specific case 2) because presumption of innocence, just in case there happens to be an innocuous explanation to this.
So... thoughts? I have mod powers, but when I was granted them I was basically just told to use them to fight spam; there was never any discussion of any other policy, and I don't feel like I have the authority to decide on the suitable course of action without consulting the rest of the community.