I'm not sure that considering things on average is a good measure of surprise

For a comparatively short lifespan, sure. Randomness dominates small sets of numbers.

I feel like the more I learn, the more surprised I am when something is different than I expect it to be. I may be surprised less, but my increased confidence in my model makes those surprises all the more salient.

Extrapolate that process out a hundred years, when it might be years between each significant surprise. A thousand, when it might be decades. Ten thousand, when it might be centuries.