One of biases that are extremely prevalent in science, but are rarely talked about anywhere, is bias towards models that are mathematically simple and easier to operate on. Nature doesn't care all that much for mathematical simplicity. In particular I'd say that as a good first approximation, if you think something fits exponential function of either growth or decay, you're wrong. We got so used to exponential functions and how convenient they are to work with, that we completely forgot the nature doesn't work that way.
But what about nuclear decay, you might be asking now... That's as close you get to real exponential decay as you get... and it's not nowhere close enough. Well, here's a log-log graph of Chernobyl release versus theoretical exponential function, plotted in log-log.
Well, that doesn't look all that exponential... The thing is that even if you have perfect exponential decay processes as with single nucleotide decay, when you start mixing a heterogeneous group of such processes, the exponential character is lost. Early in time faster-decaying cases dominate, then gradually those that decay more slowly, somewhere along the way you might have to deal with results of decay (pure depleted uranium gets more radioactive with time at first, not less, as it decays into low half-life nuclides), and perhaps even some processes you didn't have to consider (like creation of fresh radioactive nuclides via cosmic radiation).
And that's the ideal case of counting how much radiation a sample produces, where the underlying process is exponential by the basic laws of physics - it still gets us orders of magnitude wrong. When you're measuring something much more vague, and with much more complicated underlying mechanisms, like changes in population, economy, or processing power.
According to IMF, world economy in 2008 was worth 69 trillion $ PPP. Assuming 2% annual growth and naive growth models, the entire world economy produces 12 cents PPP worth of value in entire first century. And assuming fairly stable population, an average person in 3150 will produce more that the entire world does now. And with enough time dollar value of one hydrogen atom will be higher than current dollar value of everything on Earth. And of course with proper time discounting of utility, life of one person now is worth more than half of humanity millennium into the future - exponential growth and exponential decay are both equally wrong.
To me they all look like clear artifacts of our growth models, but there are people who are so used to them that they treat predictions like that seriously.
In case you're wondering, here are some estimates of past world GDP.
In general, rules of thumb have two dimensions - applicability (that is the size of the domain where it applies) and efficacy (the amount or degree of guidance that the rule provides).
Simplicity, a.k.a Occam's Razor, is mentioned frequently as a guide in these (philosophy of science/atheist/AI aficionado) circles. However, it is notable more for its broad applicability than for its efficacy compared to other, less-broadly-applicable guidelines.
Try formulating a rule for listing natural numbers (positive integers) without repeats that does not generally trend upwards. For example, you could alternate between powers of ten and powers of two: 1, 10, 2, 100, 4, 1000, ... Regardless of the rule, you cannot list the natural numbers from largest to smallest; there is no largest. Whichever you pick as your first, you will eventually be forced past by the "no repeats" clause.
A general learner can be viewed as outputting a list of numbers (coding for hypotheses). Occam's razor is roughly the observation "the numbers will generally trend upwards". There's still a lot up in the air after that observation.
What's with the Occam bashing? Yes, the OP wrote:
"Nature doesn't care all that much for mathematical simplicity."
...but that doesn't make it true: Occam's Razor is great!