V_V comments on Failures of an embodied AIXI - LessWrong

29 Post author: So8res 15 June 2014 06:29PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (45)

You are viewing a single comment's thread. Show more comments above.

Comment author: V_V 10 June 2014 09:02:07PM 1 point [-]

Edit: Come to think of it, your response would probably start with something like "I know that, but...", but I may have addressed some of what your further objections would have been in my replay to Cyan.

...

Counting that as "identifying itself with the approximation" seems overly generous, but if we grant that, I still don't see any reason that AIXI would end up considering such a hypothesis likely, or that it would be likely to understand any mechanism for it to self-modify correctly in terms of its model of modifying an external approximation to itself.

The simple answer is: because these models accurately predict the observations after self-modification actions are performed.
Of course, the problem is not that simple, because of the non-ergodicity issues I've discussed before: when you fiddle with your source code without knowing what you are doing is easy to accidentally 'drop an anvil' on yourself. But this is a hard problem without any simple solution IMHO.

Comment author: AlexMennen 10 June 2014 09:26:34PM 1 point [-]

The simple answer is: because these models accurately predict the observations after self-modification actions are performed.

For that to be true, the environment has to keep sending AIXI the same signals that it sends the approximation even after it stops paying attention to AIXI's output. Even in that case, the fact that this model correctly predicts future observations doesn't help at all. Prior to self-modifying, AIXI does not have access to information about what it will observe after self-modifying.

I agree with you that the non-ergodicity issues don't have any simple solution. I haven't been making a big deal about non-ergodicity because there don't exist any agents that perform optimally in all non-ergodic environments (since one computable environment can permanently screw you for doing one thing, and another computable environment can permanently screw you for doing anything else), so it's not a problem specific to AIXI-like agents, and AIXI actually seems like it should act fairly reasonably in non-ergodic computable environments separated from the agent by a Cartesian barrier, given the information available to it.

Comment author: V_V 11 June 2014 01:04:02AM 1 point [-]

Then I don't think we actually disagree.
I mean, it was well known that the AIXI proof of optimality requred ergodicity, since the original Hutter's paper.