novalis comments on Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased) - Less Wrong

75 Post author: HoldenKarnofsky 18 August 2011 11:34PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (249)

You are viewing a single comment's thread. Show more comments above.

Comment author: novalis 18 August 2011 08:34:00PM -1 points [-]

Well, that's just the entire point of this LW post -- what prior to choose matters a lot. It even matters specifically in the case of BeerAdvocate, who apparently got appreciably different results from changing their value of 10 reviews.

Comment author: saturn 18 August 2011 08:42:22PM *  3 points [-]

And the solution you suggest is to just go with p=0.05 and pretend this problem isn't unavoidable, right?

Comment author: novalis 18 August 2011 09:26:19PM 1 point [-]

I think I specifically said that variance matters. I'll also say that your application matters -- when choosing beers, I would be OK with p much worse than 0.05 since I can afford to order another beer. When choosing charities, it is a harder question.