= 27d567bc2d48d9f40e9ccc20ea370b15)
Gunnar_Zarncke comments on The Problem with AIXI - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (78)
If human adults didn't grasp death any better than AIXI does, they'd routinely drop anvils on their heads literally, not 'so to speak'.
What do you mean? What would be the alternative to 'symbolic reasoning'?
If a smart AI values things about the world outside its head, it won't deliberately hack itself (e.g., it won't alter its hardware to entertain happy delusions), because it won't expect a policy of self-hacking to make the world actually better. It's the actual world it cares about, not its beliefs about, preferences over, or enjoyable experiences of the world.
The problem with AIXI isn't that it lacks the data or technology needed to self-modify. It's that it has an unrealistic prior. These aren't problems shared by humans. Humans form approximately accurate models of how new drugs, food, injuries, etc. will affect their minds, and respond accordingly. They don't always do so, but AIXI is special because it can never do so, even when given unboundedly great computing power and arbitrarily large supplies of representative data.
I disagree.
If you coerce AIXI with sufficiently tricky rewards (and nothing else is our elvolved body doing with our developing brain) ro form 'approximately accurate models' AIXI will also respond accordingly. Except
When it doesn't do so either because it has learned that it can get around this coercing. Same with humans which may also may come to think that they can get around their body and go to heaven, take drug...
AIXI wouldn't either if you coerced it like our body (and society) does us.
I don't say that there is an alternative. It means that symbolic reasoning needs some base. Axioms, goals states. Where do you get these from? In the human brain these form stabilizing neural nets thus representing approximations of vague interrelated representations of reality. But you have no cognitive access to this fuzzy-to-symbolic-relation, only to its mentalese correlate - the symbols you reason with. Whatever you derive from the symbols is in the same way separate from reality as the cartesian barrier of AIXI.
Added: See http://lesswrong.com/lw/ii5/baseline_of_my_opinion_on_lw_topics/