Just skip the intro, R-code and graphs (too heavy on math).
Question 1:
Is there a difference in karma between posts that received a negative initial comment and those that received a positive initial comment? (Any difference suggests that one or both is having an effect.)
Conclusion 1:
The difference in means has shrunk but not gone away; it’s large enough that 10% of the possible effect sizes (of "a negative initial comment rather than positive") may be zero or actually be positive (increase karma) instead. This is a little concerning, but I don’t take this too seriously:
- this is not a lot of data
- as we’ve seen there are extreme outliers suggesting that the assumptions of normality may be badly wrong
- even at face value, 10 karma points doesn’t seem like it’s large enough to have any important real-world consequences (like make people leave LW who should’ve stayed)
Question 2:
Is there a difference in karma between the two kinds of initial comments, as I began to suspect during the experiment?
Conclusion 2:
As one would hope, neither group of comments ends up with net positive mean score, but they’re clearly being treated very differently: the negative comments get downvoted far more than the positive comments. I take this as perhaps implying that LW’s reputation for being negative & hostile is a bit overblown: we’re negative and hostile to poorly thought out criticisms and arguments, not fluffy praise.
tl;dr: maybe
Full writeup on gwern.net at http://www.gwern.net/Anchoring