I don't do this all the time, but I'd say about half my downvotes are accompanied by a comment arguing with the post, though not always mine if someone else has made the point well. If my complaint is non-obvious, I try to do it every time.