Counterpoint: knowing why other people vote or don't vote is helpful. Voting alone provides little information about what people did or did not like about a comment.

However, it does distract from the primary topic and some sort of side channel for such discussions would be nice.