If your vote is simply reflecting the information content already available from the other votes, what have you added?
In that case I don't have added any information but I also didn't put any wrong information into the system. In most cases voting will be a combination of adding new information and repeating back information that already available.
It's not to pat yourself on the back for upvoting useful posts.
While not being the main point of voting, feeling good about upvoting useful post is desirable. Having a sense of community is good. Cooperation among people on LW is good. We don't want to foster an environment where everyone feels like his on his own but a environment where people feel like they are cooperating with each other.
you shouldn't be asking "Do I think this post is useful?", but "Is this post, given the current vote total more likely to be useful than other posts with the same post total?"
I disagree. If a lot of people find a post useful because it helped them, I think that's a valuable signal for the person who wrote the post. Answering an easy question when voting leads to more voting than asking a more complicated question.
In that case I don't have added any information but I also didn't put any wrong information into the system.
You put in the information that there was a forth independent vote, when in fact there wasn't.
If a lot of people find a post useful because it helped them, I think that's a valuable signal for the person who wrote the post.
But the upvotes themselves don't affect how useful the post is, it just affects how useful you think it is.
Given LW’s keen interest in bias, it would seem pertinent to be aware of the biases engendered by the karma system. Note: I used to be strictly opposed to comment scoring mechanisms, but witnessing the general effectiveness in which LWers use karma has largely redeemed the system for me.
In “Social Influence Bias: A Randomized Experiment” by Muchnik et al, random comments on a “social news aggregation Web site” were up-voted after being posted. The likelihood of such rigged comments receiving additional up-votes were quantified in comparison to a control group. The results show that users were significantly biased towards the randomly up-voted posts:
At the end of their five month testing period, the comments that had artificially received an up-vote had an average rating 25% higher than the control group. Interestingly, the severity of the bias was largely dependent on the topic of discussion:
The herding behavior outlined in the paper seems rather intuitive to me. If before I read a post, I see a little green ‘1’ next to it, I’m probably going to read the post in a better light than if I hadn't seen that little green ‘1’ next to it. Similarly, if I see a post that has a negative score, I’ll probably see flaws in it much more readily. One might say that this is the point of the rating system, as it allows the group as a whole to evaluate the content. However, I’m still unsettled by just how easily popular opinion was swayed in the experiment.
This certainly doesn't necessitate that we reprogram the site and eschew the karma system. Moreover, understanding the biases inherent in such a system will allow us to use it much more effectively. Discussion on how this bias affects LW in particular would be welcomed. Here are some questions to begin with:
Notes:
In the paper, they mentioned that comments were not sorted by popularity, therefore “mitigating the selection bias.” This of course implies that the bias would be more severe on forums where comments are sorted by popularity, such as this one.
For those interested, another enlightening paper is “Overcoming the J-shaped distribution of product reviews” by Nan Hu et al, which discusses rating biases on websites such as amazon. User gwern has also recommended a longer 2007 paper by the same authors which the one above is based upon: "Why do Online Product Reviews have a J-shaped Distribution? Overcoming Biases in Online Word-of-Mouth Communication"