You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

gjm comments on Agent-Simulates-Predictor Variant of the Prisoner's Dilemma - Less Wrong Discussion

5 Post author: Gram_Stone 15 December 2015 07:17AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (31)

Comments (31)

You are viewing a single comment's thread.

Comment author: gjm 15 December 2015 03:11:36PM 3 points [-]

I'm not sure it's really more counterintuitive that (known) ability to predict can be a disadvantage than it is that (known) having more options can be a disadvantage.

In this case, predictability is an advantage because it allows you to make binding commitments; in other words, to visibly eliminate options that would otherwise be available to you. And (see, e.g., Schelling) the ability to visibly eliminate some of your own options is very often valuable, because those options might be ones whose possibility gives the other player reason to do something that would be bad for you.

In this case, A's predictability effectively takes the possibility that A might cooperate out of the picture for B, which means that B no longer has reason to defect.

(The examples in Schelling, IIRC, tend to be of an opposite kind, more like the PD, where the ability to assure the other player that you won't defect is advantageous for both parties.)

Comment author: Lumifer 15 December 2015 07:58:48PM 1 point [-]

In this case, predictability is an advantage because it allows you to make binding commitment

In this case the human did not make any binding commitments.

Not to mention that it's UDP that is predictable.

Comment author: gjm 15 December 2015 11:29:10PM 0 points [-]

the human did not make any binding commitments.

The human did something functionally equivalent to making binding commitments.

it's UDP that is predictable

Nope, TCP :-).

(If you meant UDT or something of the kind: the claim being made in the OP is that a good implementation of UDT will be very good at predicting, not that it will be very predictable.)

Comment author: Lumifer 16 December 2015 01:30:21AM 0 points [-]

The human did something functionally equivalent to making binding commitments.

Unless you want to screw around with terminology, making a move in a game is not "making a binding commitment". It's making a move.

the claim being made in the OP

Let's look at the OP:

So if I decide to write down 9, then it will predict this, and it will decide to write 1. Therefore, I can write down 9 without fear.

This is predicting the opponent's response. Since UDT (according to the OP) does write down 1, the prediction is accurate.

UDT looks very predictable to me in this case.

Comment author: gjm 16 December 2015 03:18:25AM -1 points [-]

It isn't making a move that I suggested was equivalent to making a binding commitment. (In this case, it's working out one's best strategy, in the presence of a perfect predictor.) It's equivalent in the sense that both have the effect of narrowing the options the other player thinks you might take. That's not a good notion of equivalence in all contexts, but I think it is here; the impact on the game is the same.

Yes, there are situations in which UDT-as-understood-by-the-OP produces predictable results. That doesn't mean that UDT (as understood etc.) Is consistently predictable, and it remains the case that the OP explicitly characterized the UDT-using agent as a superhumanly effective predictor.

Comment author: Gram_Stone 16 December 2015 01:40:13AM *  0 points [-]

I'm not sure it's really more counterintuitive that (known) ability to predict can be a disadvantage than it is that (known) having more options can be a disadvantage.

I wonder if it might be fruitful to think generally about decision theories in terms of their ability to rule out suboptimal decisions, as opposed to their ability to select the optimal decision.

I also wanted you to read something I wrote below:

When described in this way, I am reminded that I would be very interested to see this sort of problem examined in the modal agents framework. I have to flag that I lack a technical understanding of this sort of thing, but it seems like we can imagine the agents as formal systems, with B stronger than A, and A forcing B to prove that A defects by making it provable in A that A defects, and since B is stronger than A, it is also provable in B that A defects.

Also, there are variants with imperfect predictors:

It can be shown that A behaves precisely as an agent would if it cooperates with an arbitrary agent C if P(C predicts A defects | A defects) is less than 0.5, is indifferent if that probability equals 0.5, and defects if that probability is greater than 0.5.

Suppose that B's predictive accuracy is greater than 50 percent. Then the expected utility of A defecting is 2p + 0(1 - p), and the expected utility of A cooperating is 1p + 1(1 - p), and the expected utility of defection is greater than that of cooperation. Plug in numbers if you need to. There are similar proofs that if B's predictions are random, then A is indifferent, and if B's predictive ability is less than 50 percent, then A cooperates.

Powered by Reddit Powered by Reddit