Today's machine learning algorithms tend to be unreasonably certain (and wrong) about inputs very different from their training data

Read about covariate shift. (More generally ML people are getting into systematic biases now, including causal inference, in a big way).

This has little to do with AGI, though.