Suppose we find the list of users who downvoted your recent comments, and there are fifteen users on that list, each of whom is an active poster in their own right. What conclusion would you draw from that?

I'd conclude that this is an extremely weird statistical anomaly which is not one user moderating down comments, but looks almost exactly like it is. One user doing a lot of downmods has to apply the downmods to separate comments, so his downmods are spread out. 15 users producing the same total number of downmods independently of each other would produce something a lot closer to a Poisson distribution with an expected value of 1, and there should be a number of comments that have zero downmods just by chance.