Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

# Douglas_Knight comments on Decision Theory Paradox: PD with Three Implies Chaos? - Less Wrong

19 27 August 2011 07:22PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

## Comments (56)

Sort By: Best

You are viewing a single comment's thread. Show more comments above.

Comment author: 29 August 2011 11:41:40PM 0 points [-]

Your restatement of my example is correct, though the utility functions being long-term is not supposed to distinguish my example from yours. In a generational model where all the children happen at a predetermined time, discounting is not an issue. Your example seems to say that the agents care about the number of children (not descendents) that they have, regardless of when they occur. If they care about their infinite descendents they need discounting to get a finite number. If they care about exponentially discounted children, even with your payoffs and linear utility, they might choose to defect if the discount rate is high enough.

But in the second case, an agent might encounter someone else's child when it plays the game, so it does care about the proportion of population.

I don't understand what you mean by this.

My version is exaggerated so that the question is whether an agent gets to play the game at all. With the original constant payoffs, every agent gets put in some triple (with probability 1), so a homogeneous population of TDT agents cooperate. Now go back to your payoffs and consider a mix of TDT and defectors with Omega doing the triples one at a time. The only interesting strategy question for the TDT agent is whether to cooperate against 1 defector or to act cliquey. To decide the expected utility of a strategy, the agent must compute the probability of the different triples it might encounter. These depend on the population when it gets picked, which depend on the strategy it chose. In order to answer exercise 1, it must do exercises 2-4 and more. It is unlikely to encounter the asymptotic population, but it is also unlikely to encounter the original population.

Here's a rough calculation. Let's start with 1 defector. Let's assume that it encounters 2 of the original agents, not their descendents. Let's further assume its immediate children encounter original agents, not descendents (a quite unreasonable assumption); and that they don't get picked together, but face 2 TDT agents. Then the difference between cooperating and acting cliquey is 3 encounters with a defector against 9 such encounters. The number of children lost to such an encounter (compared to 3 TDTs) is 5 or 6, depending on cooperating or acting cliquey, which is quite large compared to the 1 that is gained by cooperating rather than acting cliquey. The assumption is ridiculous and the answer with linear utility is probably to cooperate, but it requires calculation and depends on the utility function (and maybe the initial population).