I don't think you need to know much math to understand it. The way we understand the universe is that there are the laws of physics and the big bang. The laws of physics are a set of differential equations. The big bang is the initial condition. You specify how the universe is at t = 0, and how it changes, and from that you can deduce everything that will happen. You could also go on to deduce everything that did happen, and you'll end up with the arrow of time going the other way. Or you could follow the timeless physics route, and have some huge number of dimensions, and the arrow of time just generally points away from wherever you stuck the big bang.

You don't even need differential equations. A reversible cellular automaton will also work.