I am not persuaded that the harder Bayesians have any more concrete answer. Solmonoff induction is uncomputable and seems to unnaturally favour short hypotheses involving Busy-Beaver-sized numbers. And any computable approximation to it looks to me like brute-forcing an NP-hard problem.

Eh. I like the approach of "begin with a simple system hypothesis, and when your residuals aren't distributed the way you want them to be, construct a more complicated hypothesis based on where the simple hypothesis failed." It's tractable (this is the elevator-talk version of one of the techniques my lab uses for modeling manufacturing systems), and seems like a decent approximation of Solomonoff induction on the space of system models.