This essay claims to refute a popularized understanding of Occam's Razor that I myself adhere to. It is confusing me, since I hold this belief at a very deep level that it's difficult for me to examine. Does anyone see any problems in its argument, or does it seem compelling? I specifically feel as though it might be summarizing the relevant Machine Learning research badly, but I'm not very familiar with the field. It also might be failing to give any credit to simplicity as a general heuristic when simplicity succeeds in a specific field, and it's unclear whether such credit would be justified. Finally, my intuition is that situations in nature where there is a steady bias towards growing complexity are more common than the author claims, and that such tendencies are stronger for longer. However, for all of this, I have no clear evidence to back up the ideas in my head, just vague notions that are difficult to examine. I'd appreciate someone else's perspective on this, as mine seems to be distorted.
Essay: http://bruce.edmonds.name/sinti/
Occam's Razor is a heuristic, that is, a convenient rule of thumb that usually gives acceptable results. It is not a law of nature or a theorem of mathematics (no, not an axiom either). It basically tells you what humans are likely to find more useful. It does not tell you -- and does not even pretend to tell you -- what is more likely to be true.
But humans define "likely" in part via considerations of simplicity. This doesn't guarantee that the long run average of percentage correct conclusions will decline with complexity. But it makes it more likely.