LESSWRONG
LW

Comment Permalink

Considering something impossible is necessary to decision theory.

Paraphrasing Drescher: Since I am rational, it is impossible that I will walk into the street while I can see traffic. I have failed to do so 100/100 times before. Therefore, it is impossible that I get hit by traffic, therefore I should cross the street. BLAM

Randaly15y00

See in context

1 “Fake Options” in Newcomb’s Problem

by Normal_Anomaly

10th Dec 2010

1 min read

1

This is an exploration of a way of looking at Newcomb’s Problem that helped me understand it. I hope somebody else finds it useful. I may add discussions of other game theory problems in this format if anybody wants them.

Consider Newcomb’s Problem:: Omega offers you two boxes, one transparent and containing $1000, the other opaque and containing either $1 million or nothing. Your options are to take both boxes, or only take the second one; but Omega has put money in the second box only if it has predicted that you will only take 1 box. A person in favor of one-boxing says, “I’d rather have a million than a thousand.” A two-boxer says, “Whether or not box B contains money, I’ll get $1000 more if I take box A as well. It’s either $1001000 vs. $1000000, or $1000 vs. nothing.” To get to these different decisions, the agents are working from two different ways of visualising the payoff matrix. The two-boxer sees four possible outcomes and the one-boxer sees two, the other two having very low probability.

The two-boxer’s payoff matrix looks like this:

Box B

|Money | No money|

Decision 1-box| $1mil | $0 |

2-box | $1001000| $1000 |

The outcomes $0 and $1001000 both require Omega making a wrong prediction. But as the problem is formulated, Omega is superintelligent and has been right 100 out of 100 times so far. So the one-boxer, taking this into account, describes the payoff matrix like this:

Box B

|Money | No money|

Decision 1-box| $1mil | not possible|

2-box | not possible| $1000 |

If Omega is really a perfect (nearly perfect) predictor, the only possible (not hugely unlikely) outcomes are $1000 for two-boxing and $1 million for one-boxing, and considering the other outcomes is an epistemic failure.

Personal Blog

1

“Fake Options” in Newcomb’s Problem

New Comment

22 comments, sorted by

top scoring

Click to highlight new comments since: Today at 4:09 PM

[-]Will_Sawin15y60

Considering something impossible is necessary to decision theory.

[-]Randaly15y00

It is, however, worth making a distinction between something impossible due to a known fact about the world not including from you, and something impossible because it results from an action you'll never take. The latter leads to paradoxes like the above; since, obviously, if I do in fact walk into the street when I can see traffic, then it's not in fact impossible for me to walk into the street when I can see traffic. This results from the agents conditioning on incorrect evidence (whose incorrectness results from the agent's decision, which results in part from the evidence...); the way this is dealt with in Ambient Decision Theory is to ban considering the results of an agents actions as part of the world, and instead requiring that they be reconstructed each time they are called (see the second and third sections of this post).

While it's technically possible for a similar process to occur in the world, it's fairly clear that it's not occurring here: while the initial state of the world does depend on the agent's decision, the problem explicitly states that it's impossible for such a paradox to result, as the decision Omega is conditioning on is always the same as the one that the agent eventually makes.

[-]Vladimir_Nesov15y00

the way this is dealt with in Ambient Decision Theory is to ban considering the results of an agents actions as part of the world, and instead requiring that they be reconstructed each time they are called

I wouldn't say it like that, since the actions must already be part of the world (its properties following from the definition), the agent just can't figure out what they are, can't resolve that particular logical uncertainty. See "Against counterfactuals": everything should be already determined, as variations of the decision problem have nothing to do with the actual decision problem.

[-]FAWS15y10

That's not the right reasoning for 1-boxing.

Consider Newcomb's problem with both boxes transparent. Suppose that you see that box 1 is empty. Following your reasoning 1-boxing and receiving $1,000,000 would seem to be a fake option. If your conclusion is that the rational thing is to take box 2 and the $1000 Omega will be justified in not having placed $1,000,000 in box 1. So the reason you see the best outcome eliminated as a fake option is because you are willing to eliminate that fake option. The correct solution is to only take box 1, whether it it empty or not. And this means it will not be empty. But for that outcome you need the counterfactual of what would have happened if the box had been empty to be favorable, which requires actually being prepared to act in the favorable way in the counterfactual. The only way to ensure that is to be prepared to act that way in reality when reality looks identical to the counterfactual.

[-]Manfred15y30

Oh dear, you seem to be running into contradictory assumptions. If Omega can change the state of the box, it's not Newcomb's problem, and if an apparently empty box contains money, it's not a transparent box. I'm excluding ad-hoc hypotheses like "you could be hallucinating," because it's stupid and inelegant - if Omega can exist, it is also possible for me to be an ideal agent. If it's impossible to be an ideal agent, why the hell are we talking about a problem where Omega can exist?

[-]FAWS15y00

If Omega can change the state of the box, it's not Newcomb's problem,

Of course Omega can't afterwards. At the moment you see the content the box either is empty or it isn't, and that doesn't change.

and if an apparently empty box contains money,

Of course it doesn't.

I'm excluding ad-hoc hypotheses like "you could be hallucinating,"

Sure, no hallucinating was assumed, though that doesn't actually change anything important.

I don't see what's so difficult to understand about this. If you counterfactually two-box if you see an empty box-1 you will see an empty box 1 and two-box, in actuality, because that's what two-boxers like you get to see in the transparent box Newcomb problem (assuming Omega defaults to an empty box-1 when either state is self-fulfilling). If you counterfactually one-box even if you see an empty box-1 you will never actually see an empty box-1, because one-boxers like you don't. If (hypothetically speaking) you somehow end up observing an empty box-1 even though you know you are a one-boxer and what you see is impossible, one-box anyway. Because if you two-box you counterfactually two-box (and thus are a two-boxer), and what you want is to counterfactually one-box. You can't counterfactually act one way and actually act another when the situations are identical down to the smallest detail.

It doesn't really matter what you assume when observing an empty box-1, (e. g. Omega is wrong for once, you are hallucinating, you are a simulation, you exist in the place the truth value of counterfactuals is located, you are a free-floating counterfactual and don't actually exist, etc) the important part is that you still one-box. And if you can manage that part properly you never actually have to.

What you should definitely not do is assume that something you believed is wrong and change your behavior accordingly.

[-]Manfred15y-10

Oh, okay, so you meant "The correct solution is to only take box 1, whether it it empty or not. And this means it will not be empty" in some non-standard sense, then?

What you should definitely not do is assume that something you believed is wrong and change your behavior accordingly.

I think that, instead, changing your behavior in response to new evidence is quite important.

It doesn't really matter what you assume when observing an empty box-1, [...] the important part is that you still one-box. And if you can manage that part properly you never actually have to.

Part of the definition of Newcomb's problem is that Omega is never wrong. And so one-boxing with transparent boxes creates a contradiction, because you're doing what Omega thought you wouldn't! This isn't a situation where you should go "oh, I guess it's no biggie that the infallible agent was wrong," this is a situation where you should go "holy shit, the supposedly infallible agent was wrong!"

As for "never actually have to:" if you precommit to one-boxing no matter what, then yes, Omega should never show you box 1 empty. But box 1 is empty. The "never" has already happened. If we stick with Omega's infallibility, you must take both boxes. If we throw away infallibility, you should still take box 2, because you should update on new evidence.

I think your procedure also might be trying to maximize the utility of every possible present you, as if every you had to act the same. But once you let yourself update on new evidence, suddenly you can do better, not just in each possibility, but in the aggregate as well - the trick with Omega is just that omniscient future-predictors create an infinite loop, which is equivalent to infinite iterations of the game.

[-]ata15y30

Part of the definition of Newcomb's problem is that Omega is never wrong. And so one-boxing with transparent boxes creates a contradiction, because you're doing what Omega thought you wouldn't! This isn't a situation where you should go "oh, I guess it's no biggie that the infallible agent was wrong," this is a situation where you should go "holy shit, the supposedly infallible agent was wrong!"

One-boxing with transparent boxes should cause you to believe that you are the result of Omega simulating what you would do if the box were empty, in which case you should take the empty box and expect to pop out of existence a moment later (content in the knowledge that the real(er) you will be getting a million dollars soon).

(Well, you shouldn't literally expect to pop out of existence (I'm not sure what that would be like), but that is what the situation would look like from the outside. In order for you to win on the actual problem, you will have to be the sort of person who would take the one empty box, and in order for Omega to correctly determine that, some amount of you will have to actually do that. It's possible that Omega doesn't need to simulate you as a full conscious being in order to correctly predict your decision, but it'll at least need to simulate you as a decision algorithm that thinks it's a full conscious being.)

[-]Manfred15y00

Hm, interesting take on it. sounds reasonable to me. But at the expense of making the whole problem sound more ridiculous :D - arguments dependent on a particular nature of reality highlight the unphysical-ness of Omega.

Also, it tells me an interesting strategy if presented with box 1 empty - delay deciding as long as possible!

[-]FAWS15y00

No, absolutely not. You haven't actually observed anything odd happening until after after you have irrevocably one-boxed, and then by definition it is already too late.

as if every you had to act the same.

Every identical you that makes exactly the same observations does have to act the same, unless you act at random.

[-]Manfred15y10

You haven't actually observed anything odd happening until after after you have irrevocably one-boxed, and then by definition it is already too late.

If you can correctly predict that something odd happens, that's equivalent to something odd happening, with your prediction mechanism if not the world. Since Newcomb's problem is so black and white, it is pretty clear that the problem (or more specifically, the bunch o' results proven from the assumptions of the problem) is broken if you act against Omega's prediction.

Every identical you

Which is why I didn't specify "identical." My claim is that you are behaving, when box 1 is empty, as if you wanted to maximize the utility of all the yous with box 1 not necessarily empty.

[-]FAWS15y00

If you can correctly predict that something odd happens, that's equivalent to something odd happening, with your prediction mechanism if not the world.

You can only correctly predict that something odd happens if you know you will still one-box, and you can only know you will still one-box if you are still going to one-box. As long as you model the problem as you still having the choice to two-box you haven't observed anything odd happening.

Which is why I didn't specify "identical."

All yous observing an empty box behave the the same unless there is something else differentiating them, which (in the scenario considered) there is not unless you incorporate sufficient randomness in your decision making process, which you have no reason to want to do. The only way for the counterfactual you in case of an empty box that determines the state of the box to one-box so that the real you can get the $1,000,000 is for the real you to also one-box in a hypothetical encounter with the empty box. The only way you could actually encounter the empty box is if you two-box after encountering it, which you should not want to do.

I'm not assuming the actual existence of more than one you. Just the existence of at least one real you that matters. If you break your precommitment and two-box just because you see an empty box the real you is losing out on the $1,000,000. It doesn't matter how you reconcile the apparent existence of the situation, the apparent emptiness of the box, Omega's infailability and your precommitment as long as that reconciliation doesn't lead to breaking the precommitment, you can worry about that afterwards (personally I'm leaning towards assuming you don't exist and are just a couterfactual).

[-]Manfred15y00

As long as you model the problem as you still having the choice to two-box you haven't observed anything odd happening.

You don't count a contradiction (perfect predictor being wrong) as "odd?"

[rest of your post]

Oh, okay. So when you said "imagine box 1 is empty," you didn't actually mean to treat box 1 as empty - that wasn't supposed to be "real," and you agree that if it was real, the logic of the problem would compel you to take box 2. Rather than treating it like a normal hypothetical, your intent is to precommit to one-boxing even if box 1 is empty so that it won't be. Does that sound right?

[-]FAWS15y00

You don't count a contradiction (perfect predictor being wrong) as "odd?"

As I argued above you haven't actually observed a perfect predictor being wrong at that point.

Oh, okay. So when you said "imagine box 1 is empty," you didn't actually mean to treat box 1 as empty - that wasn't supposed to be "real,"

Not quite, as repeatedly said it doesn't matter what you think (or even if you think anything), just that the mere reality of the situation should not change how you act. If the only way you can manage that is to pretend it's not real then so be it.

and you agree that if it was real, the logic of the problem would compel you to take box 2.

No, it doesn't. The logic of the problem merely predicts that will happen because you are a two-boxer only pretending to be a one-boxer. You still can (and should) choose to one-box, and there is (as stated) no outside force compelling you. You shouldn't be very surprised when you do find an outside force compelling you, but it won't be the logic of the problem, unless you let it (and you shouldn't).

Rather than treating it like a normal hypothetical, your intent is to precommit to one-boxing even if box 1 is empty so that it won't be.

If you want to put it that way. Anyone who wants the $1,000,000 in a transparent box Newcomb problem has to be prepared to do the same.

[-]Manfred15y20

No, it doesn't. The logic of the problem merely predicts that will happen because you are a two-boxer only pretending to be a one-boxer. You still can (and should) choose to one-box

See, this is what I find unusual. You predict that you will one-box and you also predict that this would cause a contradiction with the assumptions of the problem. This is like saying "I predict I will prove that 2=3 at noon tomorrow," and yet you don't see the oddness. Again, the fact that a proof exists (of something like "this formulation of newcomb's problem with transparent boxes is inconsistent") is as good as the proof itself.

[-]FAWS15y00

No. Not at all. The only reason we are even having this discussion is because of the highly defective way the human brain usually models choice, among other things inappropriately equating capability to make a certain choice with a material, ill-defined possibility of that choice happening. Those are two entirely different things, even though I'm afraid this all just sounds like nonsense to you.

[-]Manfred15y00

Not just ability, you're actually expecting to make that choice, which I most certainly associate with calculating a probability.

[-]FAWS15y00

Not just ability, you're actually expecting to make that choice

I never said that! I said repeatedly that it doesn't matter what you you think (inside hypothetical case), only that you one-box. Sure, if you absolutely have to make predictions, and if assuming that Omega will turn out wrong does not change your resolve to one-box that's one possible way to deal with that problem, but I already said that personally I'm leaning towards thinking that none of that is actually happening, and as long as thinking you are going to fail and two-box doesn't impede you one-boxing that works, too (as implied above). Or anything else that doesn't stop you from one-boxing.

[-]Manfred15y00

So your tentative solution is to break the problem in the same way as ata, by saying "well, what the problem really means is that you see someone who looks just like Omega pose you the problem, but it might be a simulation." (Note that Omega cannot simulate Omega for this to work, so the problem is genuinely different. If Omega could simulate Omega, it would have no need to simulate you with any uncertainty).

Let's see if I understand your more general statement - in this formulation of Newcomb's problem, it would be better if you picked box 1 even when it was empty. Therefore you should do something (anything) so that you will pick box 1 even if it is empty. Am I getting closer to what you think?

[-]FAWS15y00

So your tentative solution is to break the problem in the same way as ata, by saying "well, what the problem really means is that you see someone who looks just like Omega pose you the problem, but it might be a simulation."

No, simulation is just one of the possibilities I listed way up-thread:

(e. g. Omega is wrong for once, you are hallucinating, you are a simulation, you exist in the place the truth value of counterfactuals is located, you are a free-floating counterfactual and don't actually exist, etc)

But it's not my favored conclusion, because it leads to doing silly things like holding off deciding so you are simulated for a longer time and exist longer, as you suggested. My favored one is the last one, that you don't exist, at all, not even inside a simulation or a Tegmark IV type of thing. After one-boxing you'd (hypothetically) switch to the Tegmark IV version of course (or Omega just being wrong, nothing differentiating those).

Let's see if I understand your more general statement - in this formulation of Newcomb's problem, it would be better if you picked box 1 even when it was empty. Therefore you should do something (anything) so that you will pick box 1 even if it is empty. Am I getting closer to what you think?

I don't specifically disagree with anything in particular here, but you sound as if you would draw conclusions from that I wouldn't draw.

[-]Manfred15y00

Well, the possibilities listed up-thread other than "you don't exist" make the problem no longer exactly Newcomb's problem, unless you two-box. So I like your favorite, although I'm probably thinking of a stricter version of "don't exist" that makes it more nonsensical to talk about "what would you (who don't exist) do?"

E.g. if carrots didn't exist, what would the carrots that don't exist taste like? :D

[-]Bongo15y00

You don't expect anything contradictory to actually happen. Because you would one-box no matter what you see, you will never end up seeing an empty box.

Moderation Log

Curated and popular this week

22Comments