gokfar comments on LW anchoring experiment: maybe - Less Wrong

14 Post author: gwern 23 January 2013 10:41PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (23)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 24 January 2013 01:27:43AM *  8 points [-]

Just skip the intro, R-code and graphs (too heavy on math).

Question 1:

Is there a difference in karma between posts that received a negative initial comment and those that received a positive initial comment? (Any difference suggests that one or both is having an effect.)

Conclusion 1:

The difference in means has shrunk but not gone away; it’s large enough that 10% of the possible effect sizes (of "a negative initial comment rather than positive") may be zero or actually be positive (increase karma) instead. This is a little concerning, but I don’t take this too seriously:

  • this is not a lot of data
  • as we’ve seen there are extreme outliers suggesting that the assumptions of normality may be badly wrong
  • even at face value, 10 karma points doesn’t seem like it’s large enough to have any important real-world consequences (like make people leave LW who should’ve stayed)

Question 2:

Is there a difference in karma between the two kinds of initial comments, as I began to suspect during the experiment?

Conclusion 2:

As one would hope, neither group of comments ends up with net positive mean score, but they’re clearly being treated very differently: the negative comments get downvoted far more than the positive comments. I take this as perhaps implying that LW’s reputation for being negative & hostile is a bit overblown: we’re negative and hostile to poorly thought out criticisms and arguments, not fluffy praise.

tl;dr: maybe