= 27d567bc2d48d9f40e9ccc20ea370b15)
timtyler comments on Q&A with Michael Littman on risks from AI - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (88)
A common sentiment. Shane Legg even says something similar:
I think we can now label the need for an environment as a fallacy. Most of the guts of building an intelligent agent involves finding good computable approximations to Solomonoff induction - and you can do that pretty well in a virtual world with an optimisation algorithm and a fitness function based around something like AIQ. This is essentiallly a math problem.
I think it's difficult to label something as a fallacy when there is almost no hard evidence either way about who is right. The vast majority of AI researchers (including myself) don't think that Solomonoff induction will solve AI. It is also possible to construct formal environments where performing better than chance is impossible without constant interaction with the environment (unless P=NP). So, if the problem is essentially a math problem, that would have to depend on specific facts about the world that make it different from such environments.
Solomonoff induction certainly doesn't give you an evaluation function on a plate. It would pass the Turing test, though. If the "vast majority of AI researchers" don't realise that, they should look into the issue further.
So: Occam's razor - the foundation of science - is also needed. We know about that. Other facts about the world might help a little (arguably, some elements relating to locality are implicit in the reference machine) - but they don't seem to be as critical.
I was referring to computational issues, not whether a complexity prior is reasonable or not. It is possible that making inferences about the environment requires you to solve hard computational problems, and that these problems become easier after additional interaction with the environment. I don't see how Occam's razor suggests that our world doesn't look like that (in fact, I currently think that our world does look like that, although my confidence in this is not very high).
Well, of course - but that's learning - which Solomonoff induction models just fine (it is a learning theory).
Or maybe you are suggesting that organisms modify their environment to make their problems simpler. That is perfectly possible - but I don't really see how it is relevant.
You apparently didn't disagree with Solomonoff induction allowing the Turing test to be passed. So: what exactly is your beef with its centrality and significance?
It's possible I misunderstood your original comment. Let me play it back to you in my own words to make sure we're on the same page.
My understanding was that you did not think it would be necessary for an AGI to interact with its environment in order to achieve superhuman intelligence (or perhaps that a limited initial interaction with its environment would be sufficient, after which it could just go off and think). Is that correct, or not?
P.S. I think that I also disagree with the Solomonoff induction -> Turing test proposition; but I'd rather delay discussing that point because I think it is contingent on the others.
Pretty much. Virtual environments are fine, contain lots of complexity (chaos theory) and have easy access to lots of interesting and difficult problems (game of go, etc). Virtual worlds permit the development of intelligent agents just like the "real" world does. A good job too - since we have no terribly good way of telling whether our world exists under simulation or not.
The Solomonoff induction -> Turing test proposition is detailed here.
Sorry for the delayed response, it took me a while to get through the article and corresponding Hutter paper. Do you know of any sources that present the argument for why the Kolmogorov complexity of the universe should be relatively low (i.e. not proportional to the number of atoms), or else why Solomonoff induction would perform well even if the Kolmogorov complexity is high? These both seem intuitively true to me, but I feel uneasy accepting them as fact without a solid argument.
The Kolmogorov complexity of the universe is a totally unknown quantity - AFAIK. Yudkowsky suggests a figure of 500 bits here - but there's not much in the way of supporting argument.
Solomonoff induction doesn't depend of the Kolmogorov complexity of the universe being low. The idea that Solomonoff induction has something to do with the Kolmogorov complexity of the universe seems very strange to me.
Instead, consider that Solomonoff induction is a formalisation of Occam's razor - which is a well-established empirical principle.
I don't understand. I thought the point of Solomonoff induction is that its within an additive constant of being optimal, where the constant depends on the Kolmogorov complexity of the sequence being predicted.
Wouldn't it put an upper bound on the complexity of any given piece, as you can describe it with "the universe, plus the location of what I care about"?
Edited to add: Ah, yes but "the location of what I care about" is has potentially a huge amount of complexity to it.