ciphergoth comments on Open Thread: March 2009 - Less Wrong

6 Post author: CarlShulman 26 March 2009 04:04AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (68)

You are viewing a single comment's thread. Show more comments above.

Comment author: PhilGoetz 27 March 2009 12:59:44AM *  2 points [-]

Having a slider requires a more-sophisticated data analysis, because different people use different rating scales. Typically psychologists use a multi-point scale, then use Rasch analysis (also called multi-item response theory) on the data.

I would say from my experience that a 5-point scale is not big enough; almost everything gets 3 or 4 points, except from the people (about 2% of raters) who binarize the scale by giving everything either a 1 or a 5. Also, people will not use negative ratings, so don't try to center them on zero. People (or at least Americans) just can't say "zero is average".

Comment author: ciphergoth 27 March 2009 08:36:01AM 0 points [-]

Two things you could do about that:

  • As you hover over the rating button, the text below changes to indicate what that rating would mean. Zero stars means "don't bother", one star means "good enough to stay visible", two stars means "above-average" and so on

  • Allow half stars for more information.

We would use percentile score to make the best use of the votes of binarizing voters without giving them more influence than high-information voters.