lukeprog comments on Intuitive Explanation of Solomonoff Induction - Less Wrong

13 Post author: lukeprog 01 December 2011 06:56AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (31)
Tags: induction

Comments (31)

You are viewing a single comment's thread. Show more comments above.

Comment author: lukeprog 01 December 2011 06:15:38PM 3 points [-]

You don't need SI to find the exact truth if you have all information, but without all information, SI finds only the probable.

Yes of course. We are always engaged in a probabilistic search for truth.

But even then it's based on Kolmogorov complexity which is not computable, so SI is not computable, so "exactness" isn't ideal. Isn't it just that other approaches do worse on average?

The ideal, if we had infinite computing power, would be to discover what is most probably true. But the ideal is uncomputable, so we seek out approximations of that ideal that are computable, especially ones that are computable within a certain domain of problems, like the Bayesian information criterion.

Comment author: hairyfigment 02 December 2011 06:04:34AM 0 points [-]

We are always engaged in a probabilistic search for truth.

And a good thing, too, if I understand Tarski's undefinability theorem correctly.

While you don't have to mention Tarski, you do need to remember that some smart people will object to the claim that (in principle) there exists a recipe for truth.

Comment author: lessdazed 01 December 2011 06:53:19PM 0 points [-]

As it turns out, there is a best recipe for finding truth from uncertainty (although it won't outperform recipes that do things like call for the truth as an ingredient and say not to cook anything). And a recipe that's almost as good for every problem was discovered in the 1960s. It was also discovered that no one could find a better one and know that they had discovered it.

To find a slightly better one and know you had would require you to follow one step after another for an infinite amount of time. That's impossible, so the best you can do in theory is to follow almost as good recipe we have.

The problem is that you don't have time to follow this recipe either. To find the truth to even a simple question using this recipe would require you to follow one step after another until long after the heat death of the universe. While this is less than an infinite amount of time, you can't wait that long either.

Comment author: lukeprog 01 December 2011 07:37:48PM 1 point [-]

I assume some of this, the parts I didn't write, are not meant to be blockquoted?

Comment author: lessdazed 01 December 2011 10:37:44PM 0 points [-]

It was meant as a draft of an alternative, based on my limited understanding. I see two thresholds where I think you see one. The recipe is uncomputable, so it would take longer than "long after the heat death of the universe" or any other finite amount of time to finish. Also, the computable functions most similar to it would take more steps than there is time to do.

Comment author: lukeprog 01 December 2011 10:42:38PM 1 point [-]

Yes, approximations of Solomonoff Induction need to be not just computable but also tractable.

Comment author: lessdazed 03 December 2011 01:06:57AM 0 points [-]

It's a good sign our understandings match, but consider that after simply reading the explanation I thought you meant something other than what you did.

Powered by Reddit Powered by Reddit