= 27d567bc2d48d9f40e9ccc20ea370b15)
Eliezer_Yudkowsky comments on Open Thread: October 2009 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (425)
NFL theorems are about max-entropy worlds. Solomonoff induction works on highly lawful, simplicity-biased, low-entropy worlds.
If you could actually do Solomonoff induction, you would become at least as smart as a human baby in roughly 0 seconds (some rounding error may have occurred).
The same (or a similar) point applies. If you limit yourself to the set of lawful worlds and use an Occamian prior, you will start off much worse than an algorithm that implictly assumes a prior that's close to the true distribution. As Solomonoff induction works its way up through longer algorithms, it will hit some that run into an infinite loop. Even if you program a constraint that gets it past or out of these, the optimality is only present "after a long time", which, in practice, means later than we need or want the results.
What else can you tell us about the implications of being able to compute uncomputable functions?
You are arguing against a strawman: it's not obvious that there are no algorithms that approximate Solomonoff induction well enough in practical cases. Of course there are silly implementations that are way worse than magical oracles.
Right, but any such approximation works by introducing a prior about which functions it can skip over. And for such knowledge to actually speed it up, it must involve knowledge (gained separately from S/I) about the true distribution.
But at that point, you're optimizing for a narrower domain, not implementing universal intelligence. (In my naming convention, you're bringing in type 2 intelligence.)
It introduces a prior, period. Not a prior about "skipping over". Universal induction doesn't have to "run" anything in a trivial manner.
You can't "speed up" an uncomputable non-algorithm.
Okay, we're going in circles. You had just mentioned possible computable algorithms that approximate Solomonoff induction.
So, we were talking about approximating algorithms. The point I was making, in response to this argument that "well, we can have working algorithms that are close enough to S/I", was that to do so, you have to bring in knowledge of the distribution gained some other way, at which point it is no longer universal. (And, in which case talk of "speeding up" is meaningful.)
Demonstrating my point that universal intelligence has its limits and must combine with intelligence in a different sense of the term.
You introduce operations on the approximate algorithms (changing the algorithm by adding data), something absent from the original problem. What doesn't make sense is to compare "speed" of non-algorithmic specification with the speed of algorithmic approximations. And absent any approximate algorithms, it's also futile to compare their speed, much less propose mechanisms for their improvement that assume specific structure of these absent algorithms (if you are not serious about exploring the design space in this manner to obtain actual results).
What you call "the original problem" (pure Solomonoff induction) isn't. It's not a problem. It can't be done, so it's a moot point.
Sure it does. The uncomputable Solomonoff induction has a speed of zero. Non-halting approximations have a speed greater than zero. Sounds comparable to me for the purposes of this discussion.
There are approximate algorithms. Even Bayesian inference counts. And my point is that any time you add something to modify Solomonoff induction to make it useful is, directly or indirectly, introducing a prior unique to the search space -- cleary showing the distinctness of type 2 intelligence.
To wrap up (as an alternative to not replying):
A better question would be why you brought up the issue. We both knew what the other meant before that, but you kept bringing it up.
Okay, well, I'll believe it when I see it. In the mean time, I suspect it will be far more productive to exploit whatever regularity we already know about the environment, and work on building that into the inference program's prior. (Arguably, even the Occamian prior does that by using our hard-won belief in the universe's preference for simplicity!)