= 27d567bc2d48d9f40e9ccc20ea370b15)
ThisSpaceAvailable comments on Understanding and justifying Solomonoff induction - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (75)
It seems to me that the word "hypothesis" is generally understood to mean something more general than just a program for a UTM. For instance, we can talk about "the hypothesis that this patient has cancer". So perhaps we should have a term specific for this concept? Such as "SIH"? And then what you're saying is that the Universal Prior only applies to SIHs, and not (at least, not directly) to hypotheses such as "this patient has cancer". Since pretty much all practical hypothesis testing involves hypotheses that are not SIHs, doesn't this make SI rather limited as far as practical application? What work has been done as far as how to apply it to practical hypothesis testing?
A good point!
However, if reformulate the hypothesis as "the cause of patients symptoms is cancer", it can be treated by SI. Reductionism says that "the patient has cancer" can be translated to a statement about physical laws and elementary particles. There are problems with such a reduction, but they are of practical nature. In order to apply SI, everything must be reduced to the basic terms. So human-identified, macro-scale patterns like "cancer" must be reduced to biochemical patterns, which in turn must be reduced to molecular dynamics, which in turn must be reduced to quantum mechanics. Doing these reductions is not possible in practice due to computational limitations - even if we did know all the laws for reduction. But in theory they are fine.
A problem of more theoretical nature: whatever evidence we get in the real-world is probabilistic. SI supposes that our observations are always 100% correct.
On the other hand, it's intuitively obvious that if we treat some high-level concepts as irreductible entitities, then a form of Solomonoff induction can be applied directly. E.g. it can be used to a priori prefer "the cause of the symptoms A, B and C is cancer" over ""there are three unrelated causes a, b, and c to the three symptoms A, B, and C".
It seems to me, however, that SI is not very useful if there are other reliable methods of determining the probabilities. For example, "the single cause of the two patient's symptoms is bubonic plague" in the modern world is a hypothesis of low probability even if it is the shortest one, as the empirically-determined a priori probability of having bubonic plague is tiny.
I agree that Solomonoff induction isn't particularly relevant to most practical cases. Solomonoff Induction tries to explain what your prior should be if you didn't know anything about the world. But in any real context, we actually know a lot about the world.
Medical diagnostics is an apt example. Here, we have especially good priors, since we have accurate medical statistics. We can measure the overall incidence rates of cancer in the population overall, and typically we also have statistics for particular age and sex cohorts. There's no reason to start from "how easy is it to write a program to describe 'cancer' in some fixed language."