Yes, I noticed that too. It seems to happen most frequently to me shortly after getting into political discussions (in a broad sense that includes e.g. questions of race and gender). The obvious hypotheses are (1) the quality of my comments goes way down when I post about politics, (2) there are trigger-happy downvoters out there of various political leanings, and (3) there are, more specifically, trigger-happy downvoters whose political leanings are opposed to mine. #1 might be true but (as you remark) the downvoting seems to be quite indiscriminate and to affect comments I can see no reasonable objection to. #2 and #3 are both fairly plausible but the clearest-but particular cases of mass-downvoting we've had happen to be from a neoreactionary, favouring #3. (And of course my political opponents are, as such, Objectively Evil and therefore that's just the sort of thing they would do.)
My policy when I notice this sort of thing is to post more. Posting a comment has positive expected karma for me even when I'm being mass-downvoted, and my interpretation of mass-downvoting is that it's meant to be intimidatory; for those who can afford it, the best response to attempted intimidation is to refuse to be intimidated.
I got a -1 for posting in the same discussion here, which suggests 3). I also got a -1 for answering a question that specifically asked to explain criticism, and lost a lot of karma for [trying to defend an opinion which is popular in the outside world but is unpopular here. (I got modded down for most of that, it's just that several of them were counterbalanced by people moderating up. The last ones did not have anyone moderating them up and left me in the negatives.)
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Previous Open Thread
Next Open Thread
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should be posted in Discussion, and not Main.
4. Open Threads should start on Monday, and end on Sunday.