Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

sunwei comments on "Statistical Bias" - Less Wrong

13 Post author: Eliezer_Yudkowsky 30 March 2007 06:55PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (8)

Sort By: Old

You are viewing a single comment's thread.

Comment author: sunwei 19 November 2012 01:53:53PM 0 points [-]

I think this is called "inconsistency" rather than bias. Inconsistency means no matter how many data you have, and however you average them, the result still has directional error.

Comment author: alex_zag_al 07 November 2013 05:49:40PM *  0 points [-]

this is an easy mistake to make. You're thinking about one large sample, from which you derive one good estimate.

When the article says

Statistical bias is error you cannot correct by repeating the experiment many times and averaging together the results.

it means, getting many samples, and from each one of these samples, deriving an estimate from it in isolation.

You: 1 sample of n data points, n->infinity. 1 estimate.

Yudkowsky: n samples of d data points, n->infinity. n estimates.

Comment author: Cyan 07 November 2013 06:32:51PM *  0 points [-]

Just to follow up on alex_zag_al's sibling comment, you can have consistent estimators which are biased for any finite sample size, but are aymptotically unbiased, i.e., the bias shrinks to zero as the sample size increases without bound.

(As alex_zag_al notes, EY's explanation of bias is correct. It means that in some situations "do an analysis on all the data" is not equivalent to "do the analysis on disjoint subjects of the data and average the results" -- the former may have a smaller bias than the latter.)