Physicists tell us that reality is, at the lower levels, described by differential equations. Those describing the temporal evolution of systems are generally hyperbolic differential equations. That is, they have microscopic causal structure: the properties at a point (x,t+dt) depend on the properties only in a small, finite neighbourhood of x at time t. This is what allows larger scale events to be described more abstractly in terms of causal graphs. (A known problem here is the time-symmetry of all these equations. I don't have a solution to that either.)

Some have speculated about the very finest scale being discrete, with fundamental laws of temporal evolution such as those of cellular automata or graph rewriting. That would also provide the fundamental causal structure from which causal structure on the macroscopic level emerges.

A universe whose fundamental laws were all elliptic differential equations -- those in which a change in boundary conditions in one place changes the solution everywhere at once -- would not support causal reasoning. Everything depends on everything else and none of its emergent phenomena would be describable by sparse causal graphs. I'm not sure you could have agents within such a universe, for it to look like anything to them, or any notion of time, but we can imagine such universes from outside.