Let's stow our QM caps and pretend Democritus was right: Atoms turn out to be the fundamental, indivisible units of physical reality after all. Let's further pretend that some flavor of hard determininism is in play: There is nothing apart from the motions and interactions of the atoms that governs the way the Universe ambles through time.

Past, perhaps, the first Planck moment of existence, where we might be initializing all of the constants we'll need, our billiard-ball universe is quite amenable to explanations at the atomic level in the language of causality. Why is Atom A here, and not there, at time t? Because it was caused to be there by the actions of the other atoms, and due to certain properties of A itself, in the time leading up to t.

In theory, this means that any level of abstraction we build up from the atomic one should preserve that ability to be described causally. But the amount of computational power we would need to actually pull that off would be staggering, far, far more than we could possibly fit within 3 pounds of grey matter. So even starting from the most deterministic possible model, as agents within the system, we don't really have the ability to directly leverage that causality.

Instead, we are forced by our own limited resources to construct abstractions that are simple enough for us to reason about. These abstractions throw out a lot of detail! And when you throw away even a small amount of detail, you lose the clean isomorphism-to-reality that allows our earlier statement of causality to be preserved. When you're dealing with atoms, in the billiard ball world, you can always predict where they'll be if you have enough power; when you're dealing with aprroximations of atoms, you lose the "always".

So not only is the map not the territory, if you lose the territory, there is no way to perfectly accurately reconstruct it from the map alone. If you ever find yourself in that unenviable place, you'll have to make (dare I say it) aesthetic decisions in the reconstruction process.

And that's a weird thing to think about, not least because that isn't the case in mathematics. If you have a finite-bandwidth mathematical signal, you can perfectly reconstruct it from a finite number of details about it with the correct sampling conditions. That's just the most direct example. What about how we can use set theory to define the natural numbers, for example, 0 = null; 1 = {null} = {0}; 2 = {{null}, null} = {1, 0}; ... ? In fact, mathematics abounds with territories that can be perfectly reconstructed from very small maps - that's one of the most interesting things about it as a subject.

In other words, it feels awfully like causality should be a rare and fleeting thing, as rare as pouring a bottle of clover honey into your chamomile tea and having it form into a tiny little honey-fairy taking a nice bath.

And yet... All of that clashes with my very enjoyably lived experience that, on the day-to-day human level, causality actually seems to work pretty freaking fine for most of my decisions.

Why? What on Earth is the difference?

I have a boring but serviceable hypothesis: We've just been around long enough to make our world human-shaped enough to let this happen.

Look around you. Chances are ninety percent of the interesting/useful/beautiful/etc. objects around you were either designed by humans, or placed there strategically by them. We might only have 3 pounds of grey matter to work with, but we had the fantastic luck to have a big chunk of that grey matter go towards a really good abstraction of how other humans worked. That means we can perform second-, and even third-order abstractions on how to set things up for them so they will have an easier time. When we wrap this chunk around and use it on ourselves, we often come up with better ways to do things than if we just introspected directly.

At the dawn of human history any one of us might have had a one-in-a-thousand chance of making the correct abstraction of "seed + soil + water = food" ex nihilo. That's okay, because the chances of us being able to spread that discovery to others are much higher. As the centuries unfolded, our ability to efficiently abstract one another might have allowed something like a compound-interest effect to take hold as regards these normally quite rare discoveries of bubbles of causality in the natural world; now we sit here in 2019 with a world where most of the most inscrutable problems we face are much bigger than ourselves. We worry about AGI, but we don't worry about, say, how to make a new T shirt once our last one falls apart. Even if we did have to make a new one by hand, we have mechanisms in place to acquire that knowledge; facing down the problem with nothing at our holster but trial and error is a much scarier proposition.