= 27d567bc2d48d9f40e9ccc20ea370b15)
shminux comments on Causal decision theory is unsatisfactory - LessWrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (158)
Imagine the following game: You give me a program that must either output "give" or "keep". I play it in a token trade (as defined above) against itself. I give you the money from only the first instance (you don't get the wealth that goes to the second instance).
Would you be willing to pay me $150 to play this game? (I'd be happy to pay you $150 to give me this opportunity, for the same reason that I cooperate with myself on a one-shot PD.)
This is broken counterfactual reasoning. It assumes that your action is independent of your clone's just because your action does not influence your clone's. According to the definition of the game, the clone will happen to defect if you defect and will happen to cooperate if you cooperate. If you respect this logical dependence when constructing your counterfactuals, you'll realize that reasoning like "regardless of what my clone does I..." neglects the fact that you can't defect while your clone cooperates.
Assume it's not a perfect clone, it can defect with probability p even if you cooperate. Then apply CDT. You get "defect" for any p>0. So it is reasonable to implicitly assume continuity and declare that CDT forces you to defect when p=0. However, if you apply CDT for the case p=0 directly, you get "cooperate" instead.
In other words, the conterfactual reasoning gets broken when the map CDT(p, PD) is not continuous at the point p=0.
It's not entirely clear what you're saying, but I'll try to take the simplest interpretation. I'm guessing that:
- If you're going to defect, your clone always defects.
- If you're going to cooperate, your clone cooperates with probability 1-p and defects with probability p
In that case, I don't see how it is that you get "defect" for p>0; the above formulation gives "cooperate" for 0<=p<0.5.
I disagree. If the agent has a 95% probability of doing the same thing as me and a 5% chance of defecting, I still cooperate. (With 95% probability, most likely, because you gotta punish defectors.)
Indeed, consider the following game: You give me a program that must either output "give" or "keep". I roll a 20 sided die. On a 20, I play your program against a program that always keeps its token. Otherwise, I play your program against itself. I give you the money that (the first instance of) your program wins. Are you willing to pay me $110 to play? I'd be happy to pay you $110 for this opportunity.
I don't cooperate with myself because P(TheirChoice=Defect)=0, I cooperate with myself because I don't reason as if p is independent from my action.
Suppose you have to submit the source code of a program X, and I will play Y = “run X, then do what X did with probability 0.99 and the reverse with probability 0.01” against Y' which is the same as Y but with a different seed for the RNG, and pay you according to how Y does.
Then “you” (i.e. Y) are not a perfect clone of your opponent (i.e. Y').
What do you do?