Comment on Attempts to Debias Hindsight Backfire!

Wed, 15 Jun 2016 14:47:18 GMT

Hm. I tend to think about overfitting as a mistake in picking the level of complexity and about the hindsight bias as a mistake about (prior) probabilities. However you have a point: if you have a choice of models to forecast the future and you pick a particular model while suffering from the hindsight bias, this can be seen as overfitting: you "in-sample" error will be low, but your out-of-sample error will be high.

Another way to look at this is to treat it as confusion between a sample estimate and a true population value. The hindsight bias will tell you that the particular realization that you're observing (=sample) is how it should have been and always will be (=population).

Discuss