That is not in the rules for the thread. Given the karma toll (and the fact that sufficiently downvoted comment threads get collapsed), it would be a bad idea to make it one of the rules. I think you should simply not vote on comments you agree with, and I suggest reversing any downvotes you've made for this reason.

I do agree that without downvoting it is hard to differentiate between views the community agrees with and views the community has no real opinion about, but I don't think adding this information is worth the disadvantages of downvoting for agreement.