When I try that, I tend to get down-votes and replies complaining that I'm not responding to their arguments.

I don't know the specific details of the instances in question. One thing I am sure about, however, is that people can't downvote comments that you don't make. Sometimes a thread is just a lost cause. Once things get polarized it often makes no difference at all what you say. Which is not to say I am always wise enough to steer clear of arguments. Merely that I am wise enough to notice when I do make that mistake. ;)