cromulented comments on The Problem with AIXI - Less Wrong

24 Post author: RobbBB 18 March 2014 01:55AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (78)
Tags: aixi solomonoff opfai induction seq_naturalism

Comments (78)

You are viewing a single comment's thread. Show more comments above.

Comment author: matheist 12 March 2014 05:53:10AM 4 points [-]

It's really great to see all of these objections addressed in one place. I would have loved to be able to read something like this right after learning about AIXI for the first time.

I'm convinced by most of the answers to Xia's objections. A quick question:

Yes... but I also think I'm like those other brains. AIXI doesn't. In fact, since the whole agent AIXI isn't in AIXI's hypothesis space — and the whole agent AIXItl isn't in AIXItl's hypothesis space — even if two physically identical AIXI-type agents ran into each other, they could never fully understand each other. And neither one could ever draw direct inferences from its twin's computations to its own computations.

Why couldn't two identical AIXI-type agents recognize one another to some extent? Stick a camera on the agents, put them in front of mirrors and have them wiggle their actuators, make a smiley face light up whenever they get rewarded. Then put them in a room with each other.

Lots of humans believe themselves to be Cartesian, after all, and manage to generalize from others without too much trouble. "Other humans" isn't in a typical human's hypothesis space either — at least not until after a few years of experience.

Comment author: cromulented 12 March 2014 10:05:46PM *  4 points [-]

Why couldn't two identical AIXI-type agents recognize one another to some extent? Stick a camera on the agents, put them in front of mirrors and have them wiggle their actuators, make a smiley face light up whenever they get rewarded. Then put them in a room with each other.

If you're suggesting this as a way around AIXI's immortality delusion, I don't think it works. AIXI "A" doesn't learn of death even if it witnesses the destruction of its twin, "B", because the destruction of B does not cause A's input stream to terminate. It's just a new input, no different in kind than any other. If you're considering AIXI(tl) twins instead, there's also the problem that an full model of an AIXI(tl) can't fit into its own hypothesis space, and thus a duplicate can't either.

Lots of humans believe themselves to be Cartesian, after all, and manage to generalize from others without too much trouble. "Other humans" isn't in a typical human's hypothesis space either — at least not until after a few years of experience.

AIXI doesn't just believe it's Cartesian. It's structurally unable to believe otherwise. That may not be true of humans.

Comment author: matheist 13 March 2014 02:57:12AM 0 points [-]

Let me try to strengthen my objection.

Xia: But the 0, 0, 0, ... is enough! You've now conceded a case where an endless null output seems very likely, from the perspective of a Solomonoff inductor. Surely at least some cases of death can be treated the same way, as more complicated series that zero in on a null output and then yield a null output.

Rob: There's no reason to expect AIXI's whole series of experiences, up to the moment it jumps off a cliff, to look anything like 12, 10, 8, 6, 4. By the time AIXI gets to the cliff, its past observations and rewards will be a hugely complicated mesh of memories. In the past, observed sequences of 0s have always eventually given way to a 1. In the past, punishments have always eventually ceased. It's exceedingly unlikely that the simplest Turing machine predicting all those intricate ups and downs will then happen to predict eternal, irrevocable 0 after the cliff jump.

Put multiple AIXItI's in a room together, and give them some sort of input jack to observe each other's observation/reward sequences. Similarly equip them with cameras and mirrors so that they can see themselves. Maybe it'll take years, but it seems plausible to me that after enough time, one of them could develop a world-model that contains it as an embodied agent.

I.e. it's plausible to me that an AIXItI under those circumstances would think: "the turing machines with smallest complexity which generate BOTH my observations of those things over there that walk like me and talk like me AND my own observations and rewards, are the ones that compute me in the same way that they compute those things over there".

After which point, drop an anvil on one of the machines, let the others plug into it and read a garbage observation/reward sequence. AIXItI thinks, "If I'm computed in the same way that those other machines are computed, and an anvil causes garbage observation and reward, I'd better stay away from anvils".

Powered by Reddit Powered by Reddit