From the paper you cited:
"(...) MML usually (but not necessarily) restricts the reference machine to a non-universal form in the interest of computational feasibility. (...) As a result, MML can be, and has routinely been, applied with some confidence to many problems of machine learning (...)"
If there is a disagreement about which hypothesis is simpler, trying to introduce concepts such as ill-defined program lengths that can't be computed, can only obscure the terms of the debate, rather than clarifying them.
There will be such disagreement about many different hypotheses, and even when there's not our common intuition will usely have approximated the informational content density of the hypotheses, their complexity.
How do you suggest to resolve such disagreements, or reach common ground without resorting to an intuition ultimately resting on complexity measures?
How do you use Occam's Razor without an appeal a formal notion that grounds your intuition? What does your intuition rest on, if not information theory?
MML usually (but not necessarily) restricts the reference machine to a non-universal form in the interest of computational feasibility.
Sure, but IIUC (I've just skimmed the paper), in order to make the comparison to Kolmogorov complexity, they consider arbitrary Turing machines as their hypotheses, which makes the analysis uncomputable.
How do you use Occam's Razor without an appeal a formal notion that grounds your intuition? What does your intuition rest on, if not information theory?
I think that's still an open problem. Solomonoff induction is cer...
r/HPMOR readers on heroic responsibility - not the OP, the comments. Holy snorkels this is good.