= 27d567bc2d48d9f40e9ccc20ea370b15)
orthonormal comments on A Proof of Occam's Razor - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (121)
To expand on cousin_it's point, this is way too weak because it allows for arbitrary encodings, like a complexity value of 1 for "The lady down the street is a witch; she did it."
And if you start with that complexity prior, it's difficult to see how evidence would rescue you from it.
I don't see how you can exclude that encoding without additional stipulations that seem arbitrary to me.
The fact that you assign something a complexity value of 1 doesn't mean you assign it a high probability. I allow for the possibility that some more complex hypotheses will be more probable than some less complex hypotheses. And even if the hypothesis of complexity 1 is the most probable, it may only have a probability of 75% or something. You could easily be rescued from this by ordinary Bayesian evidence. In fact this will still be true even if you assign it a probability of 99.99%. In order to put yourself beyond the possibility of being rescued by evidence, you would have to try very hard.
It's true that other hypotheses better predicting the data would still overcome it in time. But a rigged encoding would still allow for theories which add an epiphenomenal entity to count as less complex; and we definitely don't want Occam's Razor to do that. I know there's no way to guarantee that, say, the universal Turing machine complexity is "fair", but you can't dodge the necessity of putting some restrictions on the encoding.
As I said to cousin it, the proof allows for different Razors depending on the specific definition of complexity. This is actually a good thing because we can learn from experience which definition to use in various contexts.
Of course experience doesn't directly say whether or not epiphenomenal entities exist, but we do discover by experience that the "number of entities" understanding of the Razor often works well, that is, that hypotheses with less entities posited are more probable than ones with more. This would lead us to adopt a Razor that does not count the addition of epiphenomena as less complex.
Would you agree that even if such restrictions are desirable in practice, there is no way to add such restrictions without adding something which is not logically necessary? The original proof is intended to be logically necessary.
At which point, all the difficulty in deciding between theories is transferred to deciding between encodings. Yes we can learn over time which encodings work well in different domains, but at that point, you're really just putting chunks of theory into your encoding choice.
There is a place for this kind of shorthand: I shouldn't have to model people in detail to form a hypothesis explaining the presence of a building in a feild. But this is because I know people exist from vast amounts of other data, so I have a justification for the shorthand of 'a human did it' in my encoding. It should have a high complexity cost, but it comes packaged with tons of evidence so in most cases it's practically free.