Who are we expecting to have buried things there? I can come up with 6 possibilites, is there another you were thinking of?
Modern humans. In this most likely case it's probably not interesting, maybe some Propaganda Preservation Program from the Cold War.
Recent aliens. I would expect if any aliens were about to jaunt over, notice our space-faring potential and bury a cache for us to discover to mark our readiness to join the Galatic Federation, we would have probably noticed them in other ways by now.
Ancient aliens. Why would visitors before intelligent terrestrial life think it worthwhile to bury stuff just in case we evolved? You've got to have a whole lot of faith in your civilization's stability to think that leaving tags everywhere is a better strategy for continuity than just colonising.
Ancient, non-human but earthbound civilization - Silurians. I could believe that another society might do this, and I think this is who the grandparent is suggesting we aim for - but since we're speculating over geological times the location of the poles is quite variable. Unless we have a fair idea of when the sender lived we don't know where to look, and to find out when they lived we'd need to find the cache... Or you could be saying "hmm, those big extinction events kind of look like the one we're causing now, I wonder where the poles were at those times?"
Some recent but forgotten technological human civilization - Atlantis. Maybe, but like the recent aliens I would expect there would be other signs.
The whole of human history is a lie! - Hiigarans. Fun times.
I don't think it's worth specifically scouting around for something, but maybe if we're buying anyway and it's cheap it'd be worth checking.
I was wondering how seriously we've considered storing useful information to improve the chance of rebounding from a global catastrophe. I'm sure this has been discussed previously, but not in sufficient depth that I could find it on a short search of the site. If we value future civilisation, then, it may be worth going to significant length to reduce existential risks.
Some interventions will target specific risky tech, like AI and synthetic biology. However, just as many of today's risks could not have been identified a century ago, we should expect some emerging risks of the coming decades to also catch us by surprise. As argued by Karim Jebari, even if risks are not identifiable, we can take general-purpose methods to reduce them, by analogy to the principles of robustness and safety factors in engineering. One such idea is, to create a store of the kind of items one would want to recover from catastrophe. This idea varies based on which items are chosen and where they are stored.
Nick Beckstead has investigated bunkers, and he basically rejected bunker-improvement because the strength of a bunker would not improve our resilience to known risks like AI, nuclear weapons or biowarfare. However, his analysis was fairly limited in scope. He focused largely on where to put people, food and walls, in order to manage known risks. It would be useful for further analysis to consider where you can put other items, like books, batteries or 3D printers, in an analysis of a range of scenarios that could arise from known or unknown risks. Though we can't currently identify many plausible risks that would leave us without 99% of civilisation, that's still a plausible situation that it's good to equip ourselves to recover from. What information would we store?
The Knowledge, How to Rebuild Civilisation From Scratch would be a good candidate based on its title alone, and a quick skim over i09's review. One could bury Wikipedia, the Internet Archive, or a bunch of other items suggested by The Long Now Foundation. A computer with a battery perhaps? Perhaps all of the above, to ward against the possibility that we miscalculate. Where would we store it? Again, the principle of resilience would seem to dictate that we should store these in a variety of sites. They could be underground and overground, marked and unmarked at busy and deserted sites of varying climate, and with various levels of security. In general, this seems to be neglected, cheap, and unusually valuable, and so I would be interested to hear whether LessWrong has any further ideas about how this could be done well.
Further relevant reading: Adaptation to and Recovery From Global Catastrophe, Svalbard Global Seed Vault (a biodiversity store in the far North of Norway, started by Gates and others).