= 27d567bc2d48d9f40e9ccc20ea370b15)
Plasmon comments on Open thread, Dec. 29, 2014 - Jan 04, 2015 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (164)
Solomonoff induction justifies this : optimal induction uses a prior which weights hypotheses by their simplicity.
Let me clarify my question. Why do you and iarwain1 think there are absolutely no other methods that can be used to arrive at the truth, even if they are sub-optimal ones?
The prior distribution over hypotheses is distribution over programs, which are bit strings, which are integers. The distribution must be normalizable (its sum over all hypotheses must be 1). All distributions on the integers go to 0 for large integers, which corresponds to having lower probability for longer / more complex programs. Thus, all prior distributions over hypotheses have a complexity penalty.
You could conceivably use a criterion like "pick the simplest program that is longer than 100 bits" or "pick the simplest program that starts with 101101", or things like that, but I don't think you can get rid of the complexity penalty altogether.
I know what SI is. I'm not even pushing the point that SI not always the best thing to do - I'm not sure if it is, as it's certainly not free of assumptions (such as the choice of the programming language / Turing machine), but let's not go into that discussion.
The point I'm making is different. Imagine a world / universe where nobody has any idea what SI is. Would you be prepared to speak to them, all their scientists, empiricists and thinkers and say that "all your knowledge is purely accidental, you unfortunately have absolutely no methods for determining what the truth is, no reliable methods to sort out unlikely hypotheses from likely ones - while we, incidentally, do have the method and it's called Solomonoff induction"? Because it looks like what iarwain1 is saying implies that. I'm sceptical of this claim.
You can have more to it than the complexity penalty, but you need a complexity penalty. The number of possibilities increases exponentially with the complexity. If the probability didn't go down faster, the total probability would be infinite. But that's impossible. It has to add to one.