Lumifer comments on Attempts to Debias Hindsight Backfire! - Less Wrong

7 Post author: Gram_Stone 13 June 2016 04:13PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (9)

You are viewing a single comment's thread. Show more comments above.

Comment author: Lumifer 15 June 2016 02:47:18PM *  1 point [-]

Hm. I tend to think about overfitting as a mistake in picking the level of complexity and about the hindsight bias as a mistake about (prior) probabilities. However you have a point: if you have a choice of models to forecast the future and you pick a particular model while suffering from the hindsight bias, this can be seen as overfitting: you "in-sample" error will be low, but your out-of-sample error will be high.

Another way to look at this is to treat it as confusion between a sample estimate and a true population value. The hindsight bias will tell you that the particular realization that you're observing (=sample) is how it should have been and always will be (=population).