You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Manfred comments on Example decision theory problem: "Agent simulates predictor" - Less Wrong Discussion

23 Post author: cousin_it 19 May 2011 03:16PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (76)

You are viewing a single comment's thread.

Comment author: Manfred 19 May 2011 06:05:03PM *  0 points [-]

If your proof works, I would expect that Omega also knows the agent is consistent and can follow the same logic, and so the UDT agent two-boxes on Newcomb's problem. Unless you use a version of UDT that (effectively) optimizes over decisions rather than actions (like TDT), which would solve both problems.

EDIT: On solving both problems: my understanding of UDT comes from AlephNeil's post. If you look at his "generalization 2," it is exactly what I mean by a problem where you need to optimize over decisions rather than actions - and he claims that a UDT agent does so 5 diagrams later - that is, treats the action as also "controlling telepathic robots."

So going by that understanding of UDT, cousinIt's proof is incorrect. If we can find proofs shorter than N, we can treat the predictor as a "robot," and so two-boxing is regarded as worse than one-boxing if it "controls the robot" into not filling the box. So to predict the agent's action we probably need to fully define the problem - what does the predictor do in cases with no proofs and in cases where the agent's action depends on the predictor's?

Comment author: wedrifid 19 May 2011 06:09:02PM 1 point [-]

If your proof works, I would expect that Omega also knows the agent is consistent and can follow the same logic

Omega knows everything but unfortunately he isn't available right now. We are stuck with a far more limited predictor.

Comment author: Manfred 19 May 2011 06:16:15PM *  0 points [-]

The reference to Omega comes from contrasting this post with prior claims that a UDT agent will one-box on newcomb's problem.