SilasBarta comments on Open Thread: October 2009 - Less Wrong

5 Post author: gwern 01 October 2009 12:49PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (425)
Tags: open_thread

Comments (425)

You are viewing a single comment's thread. Show more comments above.

Comment author: SilasBarta 01 October 2009 06:09:33PM *  0 points [-]

You can't "speed up" an uncomputable non-algorithm.

Okay, we're going in circles. You had just mentioned possible computable algorithms that approximate Solomonoff induction.

it's not obvious that there are no algorithms that approximate Solomonoff induction well enough in practical cases. [emphasis added -- SB]

So, we were talking about approximating algorithms. The point I was making, in response to this argument that "well, we can have working algorithms that are close enough to S/I", was that to do so, you have to bring in knowledge of the distribution gained some other way, at which point it is no longer universal. (And, in which case talk of "speeding up" is meaningful.)

Demonstrating my point that universal intelligence has its limits and must combine with intelligence in a different sense of the term.

Comment author: Vladimir_Nesov 01 October 2009 07:30:12PM *  0 points [-]

You introduce operations on the approximate algorithms (changing the algorithm by adding data), something absent from the original problem. What doesn't make sense is to compare "speed" of non-algorithmic specification with the speed of algorithmic approximations. And absent any approximate algorithms, it's also futile to compare their speed, much less propose mechanisms for their improvement that assume specific structure of these absent algorithms (if you are not serious about exploring the design space in this manner to obtain actual results).

Comment author: SilasBarta 02 October 2009 02:35:56PM 0 points [-]

You introduce operations on the approximate algorithms (changing the algorithm by adding data), something absent from the original problem.

What you call "the original problem" (pure Solomonoff induction) isn't. It's not a problem. It can't be done, so it's a moot point.

What doesn't make sense is to compare "speed" of non-algorithmic specification with the speed of algorithmic approximations

Sure it does. The uncomputable Solomonoff induction has a speed of zero. Non-halting approximations have a speed greater than zero. Sounds comparable to me for the purposes of this discussion.

And absent any approximate algorithms, it's also futile to compare their speed, much less propose mechanisms for their improvement that assume specific structure of these absent algorithms (if you are not serious about exploring the design space in this manner to obtain actual results).

There are approximate algorithms. Even Bayesian inference counts. And my point is that any time you add something to modify Solomonoff induction to make it useful is, directly or indirectly, introducing a prior unique to the search space -- cleary showing the distinctness of type 2 intelligence.

Comment author: Vladimir_Nesov 02 October 2009 03:08:22PM *  1 point [-]

To wrap up (as an alternative to not replying):

  • I don't understand why you'd continue arguing definitions about speed of Solomonoff induction or it being "the original problem". It's clear what we both mean.
  • I believe you are wrong about general statements about what needs to be done to implement approximate Solomonoff induction. Since we don't technically define in what sense this approximation has to be general, there remain possibilities for a good technical definition that preserves "generality" in an approximate implementation.
Comment author: SilasBarta 02 October 2009 03:18:27PM 0 points [-]

don't understand why you'd continue arguing definitions about speed of Solomonoff induction or it being "the original problem". It's clear what we both mean.

A better question would be why you brought up the issue. We both knew what the other meant before that, but you kept bringing it up.

I believe you are wrong ... there remain possibilities for a good technical definition that preserves "generality" in an approximate implementation.

Okay, well, I'll believe it when I see it. In the mean time, I suspect it will be far more productive to exploit whatever regularity we already know about the environment, and work on building that into the inference program's prior. (Arguably, even the Occamian prior does that by using our hard-won belief in the universe's preference for simplicity!)

Powered by Reddit Powered by Reddit