Gunnar_Zarncke comments on The Problem with AIXI - Less Wrong

24 Post author: RobbBB 18 March 2014 01:55AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (78)

You are viewing a single comment's thread. Show more comments above.

Comment author: Gunnar_Zarncke 19 March 2014 08:34:44AM 0 points [-]

You are mentioning some aspects keeping from or motivating for suicide. This is the whole point. Suicide is a thinkable option. It just doesn't happen so often because - no wonder - it is heavily selected against. There are lots of physical, psychical and social feedbacks in place that ensure it happens seldom. But that is nothing different from providing comparable training to AIXI.

And it appears that depite of all these checks it is still possible to navigate people out of these checks (which is not much differnt from AIXI deriving solutions evading checks) to commit suicide. I e.g. remember a news story (disclaimer!) where a cultist fraudster convinced unhappy people to gift their wellth to some other person and commit suicide with the cultistly embellished promise that they'd awake in the body of the other person at another place. Now that wouldn't convince me, but could it convince AIXI? ("questions ending with a '?' mean no")

Comment author: V_V 19 March 2014 02:18:29PM *  0 points [-]

You are mentioning some aspects keeping from or motivating for suicide. This is the whole point. Suicide is a thinkable option.

Yes, but people generally know what it entails.
We don't want an AI agent to be completely incapable of destroying itself. We don't want it do destroy itself without a good cause. Crashing with its spaceship on an incoming asteroid to deflect it away from Earth would be a good cause, for instance.

a cultist fraudster convinced unhappy people to gift their wellth to some other person and commit suicide with the cultistly embellished promise that they'd awake in the body of the other person at another place. Now that wouldn't convince me, but could it convince AIXI?

If AIXI had a sufficient amount of experience of the world, I think it couldn't.