You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

How do you learn Solomonoff Induction?

1 Post author: aisarka 17 May 2016 05:47PM

I read about a fascinating technique described on Wikipedia as a mathematically formalized combination of Occam's razor and the Principle of Multiple Explanations. I want to add this to my toolbox. I'm dreaming of a concise set of actionable instructions for using Solomonoff induction. I realize this wish might be overly idealistic. I'm willing to peruse a much more convoluted tome and will consider making time for any background knowledge or prerequisites involved.

If anyone knows of a good book on this, or can tell me what set of information I need to acquire, please let me know. It would be much appreciated!

Comments (6)

Comment author: Anders_H 17 May 2016 06:21:35PM *  8 points [-]

Solomonoff Induction is uncomputable, and implementing it will not be possible even in principle. It should be understood as an ideal which you should try to approximate, rather than something you can ever implement.

Solomonoff Induction is just bayesian epistemology with a prior determined by information theoretic complexity. As an imperfect agent trying to approximate it, you will get most of your value from simply grokking Bayesian epistemology. After you've done that, you may want to spend some time thinking about the philosophy of science of setting priors based on information theoretic complexity.

Comment author: Manfred 17 May 2016 06:40:47PM 5 points [-]

The classic textbook is Li and Vitanyi's An Introduction to Kolmogorov Complexity and Its Applications.

Comment author: bogus 17 May 2016 07:20:17PM *  4 points [-]

Solomonoff induction is uncomputable, thus, as a direct consequence, it cannot be learned. Some approximations to it which are of practical interest: Occam learning and probably approximately correct learning. As a general matter, these questions are addressed by computational learning theory.

Comment author: Daniel_Burfoot 17 May 2016 08:53:01PM 2 points [-]

convoluted tome

My book describes a philosophy of science based on large scale lossless data compression. It is not going to give you a toolbox for using SI; as others have observed, SI is of primarily theoretical importance, since it can't be computed. However, different aspects of the book might help expand your worldview in this area.

Comment author: ike 17 May 2016 06:39:39PM 3 points [-]
Comment author: Houshalter 18 May 2016 05:01:22AM 0 points [-]

Also Yudkowsky's article on Occam's Razor describes the Occam's razor/ simplicity prior OP was interested in.