Is simplicity truth indicative?

27chaos

This essay claims to refute a popularized understanding of Occam's Razor that I myself adhere to. It is confusing me, since I hold this belief at a very deep level that it's difficult for me to examine. Does anyone see any problems in its argument, or does it seem compelling? I specifically feel as though it might be summarizing the relevant Machine Learning research badly, but I'm not very familiar with the field. It also might be failing to give any credit to simplicity as a general heuristic when simplicity succeeds in a specific field, and it's unclear whether such credit would be justified. Finally, my intuition is that situations in nature where there is a steady bias towards growing complexity are more common than the author claims, and that such tendencies are stronger for longer. However, for all of this, I have no clear evidence to back up the ideas in my head, just vague notions that are difficult to examine. I'd appreciate someone else's perspective on this, as mine seems to be distorted.

Essay: http://bruce.edmonds.name/sinti/

I notice I am confused. I have updated my understanding of Occam's razor numerous times in the past.

In this case I am helped by wikipedia; " The principle states that among competing hypotheses that predict equally well, the one with the fewest assumptions should be selected. Other, more complicated solutions may ultimately prove to provide better predictions, but—in the absence of differences in predictive ability—the fewer assumptions that are made, the better." https://en.wikipedia.org/wiki/Occam%27s_razor

That statement would suggest that the essay is entirely not about Occham's Razor but about "simple theories".

Whenever I explain Occham today I try to describe. Imagine if you will; a toaster. Bread goes into the "MAGICAL BLACK TOASTER BOX"[1], and toast comes out. As a theory for how a toaster works; its fine; except that it relies on Magic and an assumption of what that is. For practical purposes, knowing that a toaster just works might be more efficient in life than the real explanation of how a toaster works, (electricity passes through coils which heat up due to electrical resistance which cause enough heat to cook a piece of bread and turn it into toasted bread[2]). However if you compare the two explanations; one doesn't really explain how a toaster works; and the other leaves a lot less unexplained. (sure electricity is unexplained but thats a lot less than MAGIC TOASTING BOX). So Occam would suggest that [2] is more likely to be true than [1].

Does this make sense? Have I got my understanding of toasting magic and Occam correct?

I am not sure about this, but it seems to me that the electricity explanation starts to win more clearly when you do multiple experiments. (Also, "simplicity of the explanation" is in the mind, because it means the amount of information in addition to what you already know.)

If you would already know everything about laws of physics, then "magic" would be an implausible explanation; the magical toaster would require you to change your model of the world, and that would be too much work. But let's suppose you know nothing about physics; y... (read more)