= 27d567bc2d48d9f40e9ccc20ea370b15)
How not to sort by a complicated frequentist formula
In How Not To Sort By Average Rating, Evan Miller gives two wrong ways to generate an aggregate rating from a collection of positive and negative votes, and one method he thinks is correct. But the "correct" method is complicated, poorly motivated, insufficiently parameterized, and founded on frequentist statistics. A much simpler model based on a prior beta distribution has more solid theoretical foundation and would give more accurate results.
Evan mentions the sad reality that big organizations are using obviously naive methods. In contrast, more dynamic sites such as Reddit have adopted the model he suggested. But I fear that it would cause irreparable damage if the world settles on this solution.
Should anything be done about it? What can be done?
This is also somewhat meta in that LW also aggregates ratings, and I believe changing the model was once discussed (and maybe the beta model was suggested).
In the notation of that post, I'd say I am interested mostly in the argument over "Whether a Bayesian or frequentist algorithm is better suited to solving a particular problem", generalized over a wide range of problems. And the sort of frequentism I have in mind seems to be "frequentist guarantee" - the process of taking data and making inferences from it on some quantity of interest, and the importance to be given to guarantees on the process.
In my opinion, this would be better suited for the Open Thread.
Would it? Maybe the question (in its current form) isn't good, but I think there are good answers for it. Those answers should be prominently searchable.
What is the best paper explaining the superiority of Bayesianism over frequentism?
Question in title.
This is obviously subjective, but I figure there ought to be some "go-to" paper. Maybe I've even seen it once, but can't find it now and I don't know if there's anything better.
Links to multiple papers with different focus would be welcome. For my current purpose I have a preference for one that aims low and isn't too long.
Last year, MBlume suggested:
Someone should really write a prediction market using bitcoins -- it would be simpler for US-based users to participate.
This now exists at BetsOfBitcoin. I signed up one exactly one month ago.
So far, I've won 13 out of 13 bets. Mostly small bets and low yields, but winning is fun.
Except it's not really a prediction market. You could know the exact probability of an event happening, which is different from the market's opinion, and still not be able to guarantee profit (on average).
Tel Aviv Self-Improvement Meetup Group
I have started the Tel Aviv Self-Improvement Meetup Group. It is not about rationality or LessWrong per se, but it is heavily influenced by rationality dojos and LW posts in the applied rationality, personal optimization and anti-akrasia cluster. As the description says, it is
A group of people helping each other apply rationality to our everyday lives, in order to improve our skills, make the best decisions, become productive and achieve our goals.
If you're interested and in the area, you're welcome to join. If you have any comments or suggestions, based perhaps on experience with similar groups, please share.
but the blue strategy aims to maximize the frequency of somewhat positive responses while the red strategy aims to maximize the frequency of highly positive responses.
It's the other way around.
Still, isn't it better to plan so you won't have to throw food away? For example: if I cook a big pot of lentils to last me all week, and then end up eating at friends' houses several times and my lentils are smelling bad, I'll throw them out, but I'll make a mental note not to make such a big pot next time, or to put some in the freezer if I think I'll be away a lot.
I guess the Umesh principle applies. If you never have to throw food away, you're preparing too little.
I just got bitcoin set up, but my cursory examination of the less technical information available leads me to believe that it would be return-smoothing and generally a more clever idea to join a mining pool than to go it alone. But the one I found appears to be closed. I'm mostly fumbling along on a shallow understanding of what I'm dealing with, here, so somebody tell me: is a bitcoin mining pool the sort of thing where a bunch of people (say, LWers who want to mine bitcoins) can just up and start one? Anybody want to throw in with me? (I contribute zero knowhow, as is probably evident.)
If you haven't already, you can try deepbit.net. I did, and it's working nicely so far.
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
(Link to How Not To Sort By Average Rating.)
Something of interest: Jeffery's interval. Using the lower bound of a credible interval based on that distribution (which is the same as yours) will probably give better results than just using the mean: it handles small sample sizes more gracefully. (I think, but I'm certainly willing to be corrected.)
This is probably vastly exaggerating the possible consequences; it's just a method of sorting, and either the Wilson's interval method and a Bayesian method are definitely far better than the naive methods.
I forgot to link in the OP. Then remembered, and forgot again.
This seems to use specific parameters for the beta distribution. In the model I describe, the parameters are tailored per domain. This is actually an important distinction.
I think using the lower bound of an interval makes every item "guilty until proven innocent" - with no data we assume the item is of low quality. In my method we give the mean quality of all items (and it is important we calibrate the parameters for the domain). Which is better is debatable.