= 27d567bc2d48d9f40e9ccc20ea370b15)
ShardPhoenix comments on Probabilistic Löb theorem - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (39)
I'm still a bit vague on this Löb business. Is it a good thing or a bad thing (from a creating AI perspective) that "Löb's theorem fails" for the probabilistic version?
edit: The old post linked suggests that it's good that it doesn't apply here.
It's a good thing. Löb's theorem is an obstacle (the "Löbstacle," if you will).
Löb's theorem's means an agent cannot trust future copies of itself, or simply identical copies of itself, to only prove true statements.
Er I don't think this is right. Lob's theorem says that an agent cannot trust future copies of itself, unless those future copies use strictly weaker axioms in their reasoning system.
The "can" has now been changed into "cannot". D'oh!
Good for creating AGI, maybe bad for surviving it. Hopefully the knowledge will also help us predict the actions of strong self-modifying AI.
It does seem promising to this layman, since it removes the best reason I could imagine for considering that last goal impossible.