It seems like the north & south poles could be natural schelling points for storing information to transmit to far-future civs: they're cold, so things preserve well, and they're unique points on the earth's surface.

Except that they are not - in geological times, they move (relative to tectonic plates). Even disregarding the drift, they are rather inconvenient as places for information caches, being both under tons of water - although something might have been buried under South pole when it was not under tons of water, but would we find it, without the builders making some kind of beacon? The same applies for a cache under North pole sea floor.

And if you are aiming at less than geological times, highest mountain makes much more sense. In particular, putting something on top of South pole ice is in danger of the ice melting (bad if you account for the civilization risk being climate-based).