I think Jiro is concerned not with total karma but with some combination of (1) the effect on individual comments' reception of their having a negative score and (2) the effect on his reputation of lots of his comments being negatively rated. Of course the latter is going to correlate with total karma but it's more like total recent karma, and it seems like the short of thing readers could be affected by without explicitly noticing.

As to whether it's Nier/Azathoth/Ra again: I wouldn't be surprised, but I'm not sure "way more -1 than -2" is very strong evidence that one person is responsible for all the downvotes. It could equally be a few people who all dislike that sort of comment but don't think it's bad enough to bother downvoting them when they're already negative.