= 27d567bc2d48d9f40e9ccc20ea370b15)
RobbBB comments on The Problem with AIXI - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (78)
I saw your comment; the last section ('Beyond Solomonoff?') speaks to the worry you raised. Somewhere in AIXI's hypothesis space is a reasoner R that is a reductionist about R; AIXI can simulate human scientists, for example. But nowhere in AIXI's hypothesis space is a reasoner that is a native representation of AIXI as 'me', as the agent doing the hypothesizing.
One way I'd put this is that AIXI can entertain every physical hypothesis, but not every indexical hypothesis. Being able to consider all the objective ways the world could be doesn't mean you're able to consider all the ways you could be located in the world.
AIXI's hypothesis space does include experts on AIXI that could give it advice about how best to behave like a naturalist. Here the problem isn't that the hypotheses are missing, but that they don't look like they'll be assigned a reasonable prior probability.
I disagree: Among all the world-programs in AIXI model space, there are some programs where, after AIXI performs one action, all its future actions are ignored and control is passed to a subroutine "AGENT" in the program. In principle AIXI can reason that if the last action it performs damages AGENT, e.g. by dropping an anvil on its head, the reward signal, computed by some reward subroutine in the world-program, won't be maximized anymore.
Of course there are the usual computability issues: the true AIXI is uncomputable, hence the AGENTs would be actually a complexity-weighted mixture of its computable approximations. AIXItl would have the same issue w.r.t. the resource bounds t and l.
I'm not sure this is necessarily a severe issue. Anyway, I suppose that AIXItl could be modified in some UDT-like way to include a quined source code and recognize copies of itself inside the world-programs.
The other issue is how does AIXI learn to assign high weights to these world-programs in a non-ergodic environment? Humans seem to manage to do that by a combination of innate priors and tutoring. I suppose that something similar is in principle applicable to AIXI.
It seems worth saying at this point that I don't have an objection to loading up an AI with true prior information; it's just not clear to me that a Solomonoff approximator would be incapable of learning that it's part of the Universe and that its continued existence is contingent on the persistence of some specific structure in the Universe.