= 27d567bc2d48d9f40e9ccc20ea370b15)
Vladimir_Nesov comments on The True Epistemic Prisoner's Dilemma - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (70)
With this attitude, you won't be able to convince him. He'll expect you to defect, no matter what you say. It's obvious to you what you'll do, and it's obvious for him. By refusing to save a billion people, and instead choosing the meaningless alternative option, you perform an instrumental action that results in your opponent saving 2 billion people. You control the other player indirectly.
Choosing the option other than saving 1 billion people doesn't have any terminal value, but it does have instrumental value, more of it than there is in directly saving 1 billion people.
This is not to say that you can place this kind of trust easily, for humans you may indeed require making a tangible precommitment. Humans are by default broken, in some situations you don't expect the right actions from them, the way you don't expect the right actions from rocks. An external precommitment is a crutch that compensates for the inborn ailments.
What makes us assume this? I get why in examples where you can see each others' source code this can be the case, and I do one-box on Newcomb where a similar situation is given, but I don't see how we can presume that there is this kind of instrumental value. All we know about this person is he is a flat earther, and I don't see how this corresponds to such efficient lie detection in both directions for both of us.
Obviously if we had a tangible precommitment option that was sufficient when a billion lives were at stake, I would take it. And I agree that if the payoffs were 1 person vs. 2 billion people on both sides, this would be a risk I'd be willing to take. But I don't see how we can suppose that the correspondance between "he thinks I will choose C if he agrees to choose C, and in fact then chooses C" and "I actually intend to choose C if he agrees to choose C" is not all that high. If the flat Earther in question is the person on whom they based Dr. Cal Lightman I still don't choose C because I'd feel that even if he believed me he'd probably choose D anyway. Do you think mosthumans are this good at lie detection (I know that I am not), and if so do you have evidence for it?
What does the source code really impart? Certainty in the other process' workings. But why would you need certainty? Is being a co-operator really so extraordinary a claim that to support it you need overwhelming evidence that leaves no other possibilities?
The problem is that there are three salient possibilities for what the other player is:
Between co-operator and deceiver, all else equal, you should expect the evidence given by co-operator to be stronger than evidence given by deceiver. Deceiver has to support a complex edifice of his lies, separate from reality, while co-operator can rely on the whole of reality for support of his claims. As a result, each argument a co-operator makes should on average bring you closer to believing that he really is a co-operator, as opposed to being a deceiver. This process may be too slow to shift your expectation from the prior of very strongly disbelieving in existence of co-operators to posterior of believing that this one is really a co-operator, and this may be a problem. But this problem is only as dire as the rarity of co-operators and the deceptive eloquence of deceivers.
We clearly disagree strongly on the probabilities here. I agree that all things being equal you have a better shot at convincing him than I do, but I think it is small. We both do the same thing in the Defector case. In the co-operator course, he believes you with probability P+Q and me with probability P. Assuming you know if he trusts you in this case (we count anything else as deceivers) you save (P+Q) * 2 +(1-P-Q) *1, I save (P) * 3+(1-P) * 1, both times the percentage of co-operators R. So you have to be at least twice as successful as I am even if there are no deceivers on the other side. Meanwhile, there's some percentage A who are decievers and some probability B that you'll believe a deceiver, or just A and 1 if you count anyone you don't believe as a simple Defector.
You think that R * (P+Q) * 2 + R * (1-P-Q) * 1 > R * P * 3 + R * (1-P) * 1 + A * B * 1. I strongly disagree. But if you convinced me otherwise, I would change my opinion.
Here's an older thread about this
That may be for one step, but my point is that the truth ultimately should win over lies. If you proceed to the next point of argument, you expect to distinguish Cooperator from Defector a little bit better, and as the argument continues, your ability to distinguish the possibilities should improve more and more.
The problem may be that it's not a fast enough process, but not that there is some fundamental limitation on how good the evidence may get. If you study the question thoroughly, you should be able to move long way away from uncertainty in the direction of truth.
How does it to that, please? How does my action affect his?
Maybe it's not enough, maybe you need to do more than just doing the right thing. But it you actually plan to defect, you have no hope of convincing the other player that you won't. (See the revised last paragraph of the above comment.)
Why? My opponent is not a mind-reader.
Yes, if we can both pre-commit in a binding way, that's great. But what if we can't?
I feel that this is related to the intuitions on free will. When a stone is thrown your way, you can't change what you'll do, you'll either duck, or you won't. If you duck, it means that you are a stone-avoider, a system that has a property of avoiding stones, that processes data indicating the fact that a stone is flying your way, and transforms it into the actions of impact-avoiding.
The precommitment is only useful because [you+precommitment] is a system with a known characteristic of co-operator, that performs cooperation in return to the other co-operators. What you need in order to arrange mutual cooperation is to signal the other player that you are a co-operator, and to make sure that the other player is also a co-operator. Signaling the fact that you are a co-operator is easy if you attach a precommitment crutch to your natural decision-making algorithm.
Since co-operators win more than mutual defectors, being a co-operator is rational, and so it's often just said that if you and your opponent are rational, you'll cooperate.
There is a stigma of being just human, but I guess some kind of co-operator certification or a global meta-commitment of reflective consistency could be arranged to both signal that you are now a co-operator and enforce actually making co-operative decisions.
He is no fool either.
I don't understand.
You need to make it clear how my intention to defect or my intention to cooperate influences the other guy's actions, even if what I say to him is identical in both cases. Assume I'm a good liar.
Um... are you asserting that deception between humans is impossible?