= 27d567bc2d48d9f40e9ccc20ea370b15)
atucker comments on Example decision theory problem: "Agent simulates predictor" - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (76)
Thanks for the response, that was fast.
I can parse it, but I don't really think that I understand it in a mathematical way.
A is a statement that makes sense to me, and I can see why the predictor needs to know that the agent's proof system is consistent.
What I don't get about it is why you specify that the predictor computes proofs up to length N, and then just say how the predictor will do its proof.
Basically, I have no formal mathematics education in fields that aren't a direct prerequisite of basic multivariable calculus, and my informal mathematics education consists of Godel, Escher, Bach. And a wikipedia education in game theory.
Do you have any suggestions on how to get started in this area? Like, introductory sources or even just terms that I should look up on wikipedia or google?
If the outlined proof is less than N symbols long (which is true if N is large enough), the predictor will find it because it enumerates all proofs up to that length. Since the predictor's proof system is consistent, it won't find any other proofs contradicting this one.
The N < M is necessary to guarantee that the agent predicts the predictor's proof, right?
What happens if the outlined proof is more than N symbols long?
Yeah. Actually, N must be exponentially smaller than M, so the agent's proofs can completely simulate the predictor's execution.
No idea. :-) Maybe the predictor will fail to prove anything, and fall back to filling only one box, I guess? Anyway, the outlined proof is quite short, so the problem already arises for not very large values of N.