= 27d567bc2d48d9f40e9ccc20ea370b15)
saturn comments on Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (249)
This is the frequentist answer to the same question. Cue standard bayesian vs. frequentist debate.
Of course, you're right that BeerAdvocate could get more accurate rankings with a more fine-tuned prior, but other than that I don't see what's wrong with their method.
I think the "basically a hack" argument isn't entirely without merit in this case, bayesian or frequentist - from what is said in the article, BeerAdvocate chose m without a lot of attention to:
frequentist hat: the relative rate of Type I and Type II errors.
Bayesian hat: the relative probability of a rating increasing versus decreasing with the addition of more reviews.
Well, that's just the entire point of this LW post -- what prior to choose matters a lot. It even matters specifically in the case of BeerAdvocate, who apparently got appreciably different results from changing their value of 10 reviews.
And the solution you suggest is to just go with p=0.05 and pretend this problem isn't unavoidable, right?
I think I specifically said that variance matters. I'll also say that your application matters -- when choosing beers, I would be OK with p much worse than 0.05 since I can afford to order another beer. When choosing charities, it is a harder question.