You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

EngineerofScience comments on Cooperating with agents with different ideas of fairness, while resisting exploitation - Less Wrong Discussion

38 Post author: Eliezer_Yudkowsky 16 September 2013 08:27AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (44)

You are viewing a single comment's thread.

Comment author: EngineerofScience 04 August 2015 02:16:04PM *  -2 points [-]

I would say that according to rationality and game theory cooperating is the best choice. I will show my logic as if both people were thing the same thing.

If I defect, than they will too, and that will give a result of 2,2

If I cooperate, than they will too, and that will give a result of 3,3

I could defect and hope they use the logic above and get a gain of 5,0 but if they use this logic too, then we end up back at the nash equilibrium of getting a result of 2,2.

If I cooperate then I am giving the opponent an oppurtunity to defect but if both people are using this logic than I should cooperate and will end up at the pareto boundry and end up with a result of 3,3 but it is unrealistic to try to achieve a better score so I should just cooperate

And so, both people cooperate.

Comment author: Lumifer 04 August 2015 05:24:11PM 2 points [-]

And so, both people cooperate.

Both people who are identical and know they are identical cooperate.

Now do the exercise for two people who are different.

Comment author: EngineerofScience 05 August 2015 01:07:51PM *  -1 points [-]

Both people who are identical and know they are identical cooperate.

I see your point, but according to game theory in this scenario you assume that your opponent will make the same move as you will, because if both of you are in the same situation then assuming you both are using "perfect" logic then you will reach the same decision.

Comment author: Lumifer 05 August 2015 02:31:27PM 1 point [-]

according to game theory

How about according to reality?

And, by the way, what is the fate of theories which do not match reality? X-)

Comment author: EngineerofScience 06 August 2015 12:10:03PM *  -1 points [-]

I see your point. According to game theory you should cooperate( as I stated above). However, I will show what my thinking would be in reality...

If I cooperate, they could to, and if that happened we would at up at a payoff of 12,12. However, if they defect then I will loose points.

If I defect, I would have a chance of getting a payoff of 5,0 or a payoff of 2,2. This is the only way to get more than 12 points, and the only way to be give at least two points every time.

Then, you defect every time. If your oppponent also defects every time, you end up at the pareato boundry with a total payoff of 8,8.

Comment author: Lumifer 06 August 2015 03:02:14PM 1 point [-]

So is the game theory just wrong, then? :-)

Comment author: EngineerofScience 08 August 2015 08:46:39PM 0 points [-]

No. In this case, game theory says that if both people are using the same logic and they know that, then what I showed above is correct: cooperating is the best choice. However, that is not always the case in reality.

Comment author: Lumifer 10 August 2015 03:24:14PM 1 point [-]

Is it ever the case in reality?

Comment author: Tem42 13 August 2015 06:29:33PM *  0 points [-]

In this case, game theory says that if both people are using the same logic and they know that, then what I showed above is correct

and

Is it ever the case in reality?

It seems so, yes. We don't have absolutely certain frameworks, but we do have contracts that are enforceable by law, and we have strong trust-based networks.

It is worth pointing out that even in fairly sloppy situations, we can still use "if both people are using the same logic and they know that" rule of thumb. For example, I would never decide to carpool if I though that I could not trust the other person to be on time (but I might frequently be late if there was no cost to doing so). When all members of the carpool make this calculation, even a limited amount of evidence that we all agree that that this calculation makes it worth showing up on time is likely to keep the carpool going; that is, if it works well for two days and on the third day Bob shows up late but has a good excuse and is apologetic, we will probably be willing to pick Bob up on the fourth day.

[Edits; I have no clue how to separate two blocks of quoted text.] [Edit: figured it out].