That's a reasonably plausible hypothesis -- I'm glad I asked. This is my first experience with rated comments, so I wasn't aware that was common.

A search for other people who have commented about receiving down votes turned up this thread. It seems karma-punishing is common too.