We already have sections for both popular (up - down > threshold) and controversial (up + down > threshold). Is it that posts are automatically elevated to these states, or does that still need to be done by moderators? Is the throttling of post elevation that EY recently mentioned handled automatically or manually?
If elevation is handled manually by moderators, it seems it makes most sense to keep the tallies private, and let the moderators use bayesian math to adjust for priors. (I personally think that's overkill - might make a fun automation task however.)
The only reason to leave them public is so people can decide which posts to read. There's not enough time in the day for me to keep up with all the posts here -- hell, I can barely keep up with Eliezer's posts on OB.
Instead, it seems they should be kept private to avoid the biases pointed out.
Those like myself will likely only ever read popular posts - at which point it's too late to vote (elevation has already happened), and only occasionally dip into the new or controversial sections when particularly bored. I expect I'm in the majority. (Are we keeping stats of views vs votes? Probably a bit early to tell at this point.)
it seems they should be kept private to avoid the biases pointed out.
Personally I find the scores quite interesting. I like having a sense of what other aspiring rationalists reading alongside me are thinking.
I'm in favor of an option not to see the scores, but surely those here gathered to overcome bias should be allowed to strive to do that themselves without having to be protected from the information.
An information cascade is a problem in group rationality. Wikipedia has excellent introductions and links about the phenomenon, but here is a meta-ish example using likelihood ratios.
Suppose in some future version of this site, there are several well-known facts:
Let's talk about how the very first reader would vote. If they judged the post high quality, then they would multiply the prior likelihood ratio (6:4) times the bayes factor for a high private signal (4:1), get (6*4:4*1) = (6:1) and vote the post up. If they judged the post low quality then they would instead multiply by the bayes factor for a low private signal (1:4), get (6*1:4*4) = (3:8) and vote the post down.
There were two scenarios for the first reader (private information high or low). If we speculate that the first reader did in fact vote up, then there are two scenarios for the second scenario: There are two scenarios for the second reader:
Note that now there are two explanations for ending up two votes up. It could be that the second reader actually agreed, or it could be that the second reader was following the first reader and the prior against their personal judgement. That means that the third reader gets zero information from the second reader's personal judgement! The two scenarios for the third reader, and every future reader, are exactly analogous to the two scenarios for the second reader.
This has been a nightmare scenario of groupthink afflicting even diligent bayesians. Possible conclusions:
Note: Olle found an error that necessitated a rewrite. I apologize.