I can't know, but I've seen cases where it seems likely. Sometimes I've seen extended exchanges between 2 people, that no one else seems to be reading, where all the comments have a score of -1.

Personally, I don't think the voting system is working very well. It seems to be used to encourage conformity and punish negativity. I have a lot of points myself, but there's a strong negative correlation between the quality of my comments and posts, and the votes they receive. I'd like it if votes were no longer anonymous. I don't usually make a downvote without explaining why in a comment, myself.