paulfchristiano comments on Say More, Justify Less - Less Wrong

19 Post author: paulfchristiano 14 April 2011 10:41PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (25)

You are viewing a single comment's thread. Show more comments above.

Comment author: prase 15 April 2011 11:02:13AM 0 points [-]

Why can't we reach agreement by updating on the beliefs of others, as per Aumann's theorem? Why do we need to describe how we arrived at our beliefs, instead of simply stating them? In fact in ordinary conversation no one even tries to use Aumann-style negotiation, and from my perspective this seems completely rational. Even when talking to people whose rationality I trust, assuming common knowledge of rationality or honesty is never close to accurate; if we abandon skepticism we can expect to be consistently wrong (even when we aren't deliberately manipulated).

This is an interesting problem. I agree that Aumann-style updating in practice will lead to less correct beliefs systematically, but you don't seem to analyse why it is so. My hunch is that the reason is our inability to judge other people's rationality well, but I'd like to know what do you think about it. The explanation

we aren't sufficiently perfect rationalists to use Aumann agreement

doesn't tell much.

Comment author: paulfchristiano 15 April 2011 12:52:00PM *  0 points [-]

Perfectly updating on the basis of others' opinions is rare; I have never seen anyone who purports to do it correctly (I can't update correctly on the basis of almost anything). Common knowledge of this ability seems impossible to come by. Common knowledge that neither of you is trying to manipulate the other (or speaking for signaling reasons) is also completely non-existent: I have never been in an environment where more than 2 or 3 levels of non-manipulation were known.