# loup-vaillant comments on Problematic Problems for TDT - Less Wrong

34 29 May 2012 03:41PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Sort By: Best

Comment author: 22 June 2012 12:54:50AM 0 points [-]

The simulated problem and the actual problem don't have to actually be the same - just indistinguishable from the point of view of the agent.

Omega avoids infinite regress because the actual contents of the boxes are irrelevant for the purposes of the simulation, so no sub-simulation is necessary.

Comment author: 22 June 2012 09:15:47AM 0 points [-]

Okay. So, what specific mistake TDT does that would prevent it to distinguish the two problems? What does it lead it to think "If I precommit X in problem 1, I have to precommit X in problem 2 as well".

(If the problems aren't the same, of course Omega can avoid infinite regress. And if there is unbounded regress, we may be able to find a non-infinite solution by looping the regress over itself. But then the problems (simulated an real) are definitely the same.)

Comment author: 22 June 2012 11:51:57AM 0 points [-]

In the simulated problem the simulated agent is presented with the choice but never gets the reward; for all it matters both boxes can be empty. This means that Omega doesn't have to do another simulation to work out what's in the simulated boxes.

The infinite regress is resolvable anyway - since each TDT agent is facing the exact same problem, their decisions must be identical, hence TDT one-boxes and Omega knows this.

Comment author: 22 June 2012 12:20:19PM 0 points [-]

The infinite regress is resolvable anyway - since each TDT agent is facing the exact same problem, their decisions must be identical, hence TDT one-boxes and Omega knows this.

Now there's still the question of the perceived difference between the simulated problem and the real one (I assume here that you should 1 box in the simulation, and 2 box in the real problem). There is a difference, how come TDT does not see it? A Rational Decision Theory would —we humans do. Or if it can see it, how come can't it act on it? RDT could. Do you concede that TDT does and can, or do you still have doubts?

Comment author: 23 June 2012 12:19:36AM 1 point [-]

Due to how the problem is set up, you can't notice the difference until after you've made your decision. The only reason other decision theories know they're not in the simulation is because the problem explicitly states that a TDT agent is simulated, which means it can't be them.

Comment author: 24 June 2012 08:04:56PM *  0 points [-]

The only reason other decision theories know they're not in the simulation is because the problem explicitly states that a TDT agent is simulated, which means it can't be them.

That's false. Here is a modified version of the problem:

Omega presents the usual two boxes A and B and announces the following. "Before you entered the room, I ran a simulation of Newcomb's problem as presented to you. If your simulated twin 2-boxed then I put nothing in Box B. If your simulated twin 1-boxed, I put \$1 million in Box B. In any case, I put \$1000 in Box A. Now please 1-box or 2-box."

Even if you're not running TDT, the simulated agent is running the same decision algorithm as you are. If that was the reason why TDT couldn't tell the difference, well, now no one can. However you and I can make the difference. The simulated problem is obviously different:

Omega presents the usual two boxes A and B and announces the following. "I am subjecting you to Newcomb's problem. Now please 1-box or 2-box".

Really, the subjective difference between the two problems should be obvious to any remotely rational agent.

(Please let me know if you agree up until that point. Below, I assume you do.)

I'm pretty sure the correct answers for the two problems (my modified version as well as the original one) are 1-box in the simulation, 2-box in the real problem. (Do you still agree?)

So. We both agree that RDT (Rational Decision Theory) 1-boxes in the simulation, and 2-boxes in the real problem. CDT would 2-box in both, and TDT would 1-box in the simulation while in the real problem it would…

• 2-box? I think so.
• 1-box? Supposedly because it can't tell simulation from reality. Or rather, it can't tell the difference between Newcomb's problem and the actual problem. Even though RDT does. (riiight?) So again, I must ask, why not? I need a more specific answer than "due to how the problem is set up". I need you to tell me what specific kind of irrationality TDT is committing here. I need to know its specific blind spot.
Comment author: 24 June 2012 11:04:10PM *  1 point [-]

In your problem, TDT does indeed 2-box, but it's quite a different problem from the original one. Here's the main difference:

I ran a simulation of this problem

vs

I ran a simulation of Newcomb's problem

Comment author: 25 June 2012 06:41:25AM 0 points [-]

Oh. So this is indeed a case of "If you're running TDT, I screw you, otherwise you walk free". The way I understand this, TDT one boxes because it should.

If TDT cannot perceive the difference between the original problem and the simulation, this is because it is actually the same problem. For all it knows, it could be in the simulation (the simulation argument would say it is). There is an infinite regress, solved by the fact that all agents in all simulation level will have taken the same decision, because they ran the same decision algorithm. If they all 2-box, they get \$1000, while if they all 1-box, they get the million (and no more).

Now, if you replace "an agent running TDT" by "you" (like, a fork of you started from a snapshot taken 3 seconds ago), the correct answer is always to 1-box, because then the problem is equivalent to the actual Newcomb's problem.

Comment author: 24 June 2012 08:59:33PM 0 points [-]

Well, in the problem you present here TDT would 2-box, but you've avoided the hard part of the problem from the OP, in which there is no way to tell whether you're in the simulation or not (or at least there is no way for the simulated you to tell), unless you're running some algorithm other than TDT.

Comment author: 24 June 2012 09:19:05PM *  0 points [-]

I see no such hard part.

To get back to the exact original problem as stated by the OP, I only need to replace "you" by "an agent running TDT", and "your simulated twin" by "the simulated agent". Do we agree?

Assuming we do agree, are you telling me the hard part is in that change? Are you telling me that TDT would 1-box in the original problem, even though it 2-boxes on my problem?

WHYYYYY?

in which there is no way to tell whether you're in the simulation or not

Wait a minute, what exactly do you mean by "you"? TDT? or "any agent whatsoever"? If it's TDT alone why? If I read you correctly, you already agree that's it's not because Omega said "running TDT" instead of "running WTF-DT". If it's "any agent whatsoever", then are you really sure the simulated and real problem aren't actually the same? (I'm sure they aren't, but, just checking.)

Comment author: 24 June 2012 10:08:30PM 0 points [-]

Wait a minute, what exactly do you mean by "you"? TDT? or "any agent whatsoever"? If it's TDT alone why? If I read you correctly, you already agree that's it's not because Omega said "running TDT" instead of "running WTF-DT". If it's "any agent whatsoever", then are you really sure the simulated and real problem aren't actually the same? (I'm sure they aren't, but, just checking.)

Well, no, this would be my disagreement: it's precisely because Omega told you that the simulated agent is running TDT that only TDT could or could not be the simulation; the simulated and real problem are, for all intents and purposes, identical (Omega doesn't actually need to put a reward in the simulated boxes, because he doesn't need to reward the simulated agent, but both problems appear exactly the same to the simulated and real TDT agents).

Comment author: 25 June 2012 06:51:30AM 1 point [-]

This comment from lackofcheese finally made it click. Your comment also make sense.

I now understand that this "problematic" problem just isn't fair. TDT 1-boxes because it's the only way to get the million.