Cyan comments on Teachable Rationality Skills - Less Wrong

52 Post author: Eliezer_Yudkowsky 27 May 2011 09:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (257)

You are viewing a single comment's thread. Show more comments above.

Comment author: PhilGoetz 30 May 2011 03:39:39PM *  4 points [-]

This is <EDIT>probably</EDIT> a myth. The Aumann agreement theorem does not apply to real life. Here are three reasons why:

  1. It requires that the two rational people already share a partition function. [EDIT: No! Major mistake on my part. It requires that the two people have common knowledge of each others' partition functions.] The range of the partition function is the set of sets of states of the world that an agent can't distinguish between. That implies that, <EDIT>for all possible sets of observations, each agent knows what the other agent will infer. You could say it requires that the agents query each other endlessly about their beliefs, until they each know everything that the other agent believes.</EDIT>

  2. Interpreting Aumann’s theorem to mean what Aumann said it means, requires saying that “The meet at w of the partitions of X and Y is a subset of event E” means the same as the English phrase “X knows that Y knows event E” means. That is wrong. To expland this language a little bit: Aumann claims: To say that agent 1 knows that agent 2 knows E means that E includes all P2 in N2 that intersect P1. I claim: To say that agent 1 knows that agent 2 knows E , means that E includes P1(w), and that E includes P2(w). Agent 1 can conclude that E includes P1 union P2, for some P2 that intersects P1. Not for all P2 that intersect P1. That is a fine semantic error buried deep within the English interpretation, but it makes the entire theorem worthless.

  3. Even if you still believe that the Aumann agreement theorem applies in the way James states above, it relies on all agents being perfectly honest with each other, and (probably, tho I'd have to check this) on having mutual knowledge that they are being honest with each other.

Comment author: Cyan 30 May 2011 04:08:24PM *  0 points [-]

2 . Interpreting Aumann’s theorem to mean what Aumann said it means... That is a fine semantic error buried deep within the English interpretation, but it makes the entire theorem worthless.

That was way too densely packed for my sleep-deprived brain to parse. Would you be willing to write a post (possibly Discussion post) spelling this out less succinctly? It seems important to get this idea out into the LW-sphere given how much cred the agreement theorem has around here.

Comment author: PhilGoetz 30 May 2011 06:46:07PM 0 points [-]

It's not just densely packed - it makes no sense unless you read the paper first, and read some other things necessary to understand that paper. I'd like to write a post - but not right now.

Comment author: Will_Sawin 30 May 2011 06:56:31PM 0 points [-]

I know enough game theory to prove versions of Aumann's theorem, but I have not read the paper, and your point in (2) makes no sense, period.

The correct game-theoretic statement of 1 knows that 2 knows that E is that E includes P1(P2(w)).

The meet of X and Y is about common knowledge. Saying that E is common knowledge is stronger than saying that 1 knows that 2 knows it. It also implies, for instance, that 2 knows that 1 knows that 2 knows it.