= 27d567bc2d48d9f40e9ccc20ea370b15)
cousin_it comments on Example decision theory problem: "Agent simulates predictor" - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (76)
If the outlined proof is less than N symbols long (which is true if N is large enough), the predictor will find it because it enumerates all proofs up to that length. Since the predictor's proof system is consistent, it won't find any other proofs contradicting this one.
The N < M is necessary to guarantee that the agent predicts the predictor's proof, right?
What happens if the outlined proof is more than N symbols long?
Yeah. Actually, N must be exponentially smaller than M, so the agent's proofs can completely simulate the predictor's execution.
No idea. :-) Maybe the predictor will fail to prove anything, and fall back to filling only one box, I guess? Anyway, the outlined proof is quite short, so the problem already arises for not very large values of N.