Looking at the machine learning section of the essay, and the paper it mentions, I believe the author to be making a bit too strong a claim based on the data. When he says:
"In some cases the simpler hypotheses were not the best predictors of the out-of-sample data. This is evidence that on real world data series and formal models simplicity is not necessarily truth-indicative."
... he fails to take into account that many more of the complex hypotheses get high error rates than the simpler hypotheses (despite a few of the more complex hypotheses getting the smallest error rates in some cases), which still says that when you have a whole range of hypotheses, you're more likely to get higher error rates when choosing a single complex one than a single simple one. It sounds like he says Occam's Razor is not useful just because the simplest hypothesis isn't ALWAYS the most likely to be true.
Similarly, when he says:
"In a following study on artificial data generated by an ideal fixed 'answer', (Murphy 1995), it was found that a simplicity bias was useful, but only when the 'answer' was also simple. If the answer was complex a bias towards complexity aided the search."
This is not actually relevant to the discussion of whether simple answers are more likely to be fact than complex answers, for a given phenomenon. If you say "It turns out that you're more likely to be wrong with a simple hypothesis when the true answer is complex", this does not affect one way or the other the claim that simple answers may be more common than complex answers, and thus that simple hypotheses may be, all else being equal, more likely to be true than complex hypotheses when both match the observations.
That being said, I am sympathetic to the author's general argument. While complexity (elaboration), when humans are devising theories, tends to just mean more things which can be wrong when further observations are made, this does not necessarily point to whether natural phenomena is generally 'simple' or not. If you observe only a small (not perfectly representative) fraction of the phenomenon, then a simple hypothesis produced at this time is likely to be proven wrong in the end. I'm not sure if this is really an interesting thing to say, however - when talking about the actual phenomena, they are neither really simple nor complex. They have a single true explanation. It's only when humans are trying to establish the explanation based on limited observation that simplicity and complexity come into it.
Did you look up the papers he referenced, then? Or are you speaking just based on your impression of his summaries? I too thought that his summaries were potentially misleading, but I failed to track down the papers he mentioned to verify that for certain.
...I'm not sure if this is really an interesting thing to say, however - when talking about the actual phenomena, they are neither really simple nor complex. They have a single true explanation. It's only when humans are trying to establish the explanation based on limited observation that simplicity and c
This essay claims to refute a popularized understanding of Occam's Razor that I myself adhere to. It is confusing me, since I hold this belief at a very deep level that it's difficult for me to examine. Does anyone see any problems in its argument, or does it seem compelling? I specifically feel as though it might be summarizing the relevant Machine Learning research badly, but I'm not very familiar with the field. It also might be failing to give any credit to simplicity as a general heuristic when simplicity succeeds in a specific field, and it's unclear whether such credit would be justified. Finally, my intuition is that situations in nature where there is a steady bias towards growing complexity are more common than the author claims, and that such tendencies are stronger for longer. However, for all of this, I have no clear evidence to back up the ideas in my head, just vague notions that are difficult to examine. I'd appreciate someone else's perspective on this, as mine seems to be distorted.
Essay: http://bruce.edmonds.name/sinti/