gwern comments on How not to sort by a complicated frequentist formula - Less Wrong

3 Post author: Meni_Rosenfeld 01 January 2013 09:58PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (26)

You are viewing a single comment's thread. Show more comments above.

Comment author: gwern 01 January 2013 10:46:27PM *  5 points [-]

Something of interest: Jeffery's interval. Using the lower bound of a credible interval based on that distribution (which is the same as yours) will probably give better results than just using the mean: it is handles small sample sizes more gracefully. (I think, but I'm certainly willing to be corrected.)

I recently did a similar thing for ranking vendors by feedback, using both a Jeffreys interval and a Wilson interval; even on the vendors with little feedback, they were overall pretty similar. IIRC, I don't think they differed by more than 10% anywhere.