Kawoomba comments on Wisdom of the Crowd: not always so wise - Less Wrong

20 Post author: tgb 01 July 2012 08:55PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (20)

You are viewing a single comment's thread. Show more comments above.

Comment author: Kawoomba 02 July 2012 06:22:45PM 0 points [-]

Individual components of that estimation may be subject to bias in a given direction, but over enough sources, over enough people with many different estimation criteria, I wouldn't trust there to necessarily be a demonstrable bias over repeated experiments without deliberate intervention on the part of the experimenter

This can be seen simply as a version of the central limit theorem: Any sum or average of samples from ANY distribution (with finite mean and standard deviation) will be approximately normally distributed (Gaussian) with the approximation better for larger samples. Neato!

Comment author: sixes_and_sevens 02 July 2012 06:40:12PM *  1 point [-]

I'd say it's related to the central limit theorem, but would be cautious about equating the two. We would probably expect a Gaussian distribution from a variable which is the sum or product of a lot of component parts (i.e. lots of different estimator methods), but we wouldn't necessarily expect the mean to coincide with the true value unless some of those estimator methods were reliable, and they didn't collectively skew the distribution in one direction.

(and nit-picking, it's "a well-defined population mean and population standard deviation", which is required for defining the distribution. If you can't trust your sample mean and sample SD to approximate your population mean and SD, it's no longer reliable, and you'd have to use something else, like a t-distribution)