You're right. Reviewing my feelings on this I discovered that my main "ugh, that's terrible" feeling comes from the observation that a correlated set of people form a control system that wipes out the contributions of others not in a similar or larger implicit alliance. That doesn't imply the solution is to vote independently of the total, though, as there are negative side effects like the one you describe.