Comment Permalink

Vladimir_Nesov14y20

By crafting an appropriate dependency graph, you can make TDT agent agree to any UDT decision. Even in CM, if you model Omega in more detail as depending on your decision, you can get TDT agent to comply, but this is not the point: TDT doesn't get this answer naturally without external introduction of compensating explicit dependence bias, and neither does it in this case.

JGWeissman14y20

I would like to see the dependency graph that compels TDT to pay in a counterfactual mugging.

See in context

4 Counterfactual mugging: alien abduction edition

by Emile

28th Sep 2010

1 min read

4

Omega kidnapps you and an alien from FarFarAway Prime, and gives you the choice: either the alien dies and you go home with your memory wiped, or you lose an arm, and you both go home with your memories wiped. Nobody gets to remember this. Oh and Omega flipped a coin to see who got to choose. What is your choice?

As usual, Omega is perfectly reliable, isn't hiding anything, and goes away afterwards. You also have no idea what the alien's values are, where it lives, what it would choose, nor what is the purpose of that organ that pulsates green light.

(This is my (incorrect) interpretation of counterfactual mugging, which we were discussing on the #lesswrong channel; Boxo pointed out that it's Prisonner's Dilemma where a random player is forced to cooperate, and isn't that similar to counterfactual mugging.)

Counterfactual MuggingCounterfactuals

Personal Blog

4

Counterfactual mugging: alien abduction edition

New Comment

18 comments, sorted by

top scoring

Click to highlight new comments since: Today at 11:05 PM

[-]Sniffnoy14y70

How is this a counterfactual mugging? You're not making any counterfactual decision.

[-]Emile14y30

Agreed, it isn't counterfactual mugging. It would only be counterfactual mugging if Omega also told you the alien would decide in exactly the same way you would if it had won the coin-flip.

[-]Bongo14y30

Here's another CFM-inspired game:

Omega flips a coin. Then he asks you to choose to cooperate or defect in a game of PD against the counterfactual version of you that got the other side of the coin. He knows what that version would have chosen. Do you C or D?

I know the answer is clear, but I think this is a good intuition pump for CFM. Like I said on IRC, CFM is all about not defecting against your counterfactual cousins just because they're not real.

[-][anonymous]14y00

The obvious TDT/UDT answer would be C and the CDT answer would probably be D. The counterfactual component doesn't (or shouldn't) change your strategy much in this case.

[-]Nick_Tarleton14y00

The obvious UDT answer is C, but I think TDT defects, for the exact same reason it doesn't pay the counterfactual mugger (can't improve any aspect of the 'real' situation it finds itself in).

Edit: Wrong, see reply.

[-]JGWeissman14y20

TDT cooperates. The node representing the output of TDT affects the counterfactual TDT agent, which in turn affects Omega's "real" prediction of the counterfactual TDT.

[-]Vladimir_Nesov14y20

[-]JGWeissman14y20

I would like to see the dependency graph that compels TDT to pay in a counterfactual mugging.

[-]Vladimir_Nesov14y00

Not if it expresses what's real, but surely if it expresses what the agent cares about, basically the counterfactual world explicitly included.

[-]Douglas_Knight14y00

Are you saying that it's easier to get TDT to comply to CM if it's ontologically fundamental randomness than if it's logical uncertainty? (but you think it can be made to comply then, too)

[-]JGWeissman14y00

In the least convenient possible world, the TDT agent doesn't care intrinsically about any counterfactual process, only about the result on the real world.

Saying you can get an agent with one DT to follow the output of another DT by changing its utility function is not interesting.

[-]Vladimir_Nesov14y00

Saying you can get an agent with one DT to follow the output of another DT by changing its utility function is not interesting.

If the mapping is natural enough, it establishes relative expressive power of the decision theories, perhaps even allowing to get the same not-a-priori-obvious conclusions from studying one theory as the other. But granted, as I described in this post, the step forward made in UDT/ADT, as compared to TDT, is that causal graph doesn't need to be given as part of problem statement, dependencies get inferred from utility/action definition.

[-]JGWeissman14y00

If the mapping is natural enough,

Ok, so show me an actual example of a mapping that is "natural enough", and causes TDT to pay of in CM.

[-]Vladimir_Nesov14y00

I argued with your argument, not your conclusion.

[-]JGWeissman14y00

I am not following your abstract argument, and would like to see an example of how a "natural enough" mapping can establish "relative expressive power of the decision theories".

[-]Nick_Tarleton14y00

I think you're right.

[-]Vladimir_Nesov14y00

If you accept updateless decision-making (caring about alternative possibilities, or equivalently deciding on your strategies in advance), this is equivalent to PD, with payoff depending of your and alien's strategies, computed as expected utility through the uncertainty of the coin.

[-][anonymous]14y00

Why are you even using Omega, if it's not actually using its superintelligence to simulate you (or anyone else)? It would be less misleading it if you just said it was a random truthful jerk.

Moderation Log