Aumann's agreement theorem -- I don't claim to know all its generalizations -- assumes that the two parties have the same priors, and that each knows the other's "information partition" (i.e., what states of the world the other can distinguish). It also assumes that their knowledge of one another's posteriors is "common knowledge" in a technical sense. It also assumes that both parties are perfect Bayesians and that this too is "common knowledge". I see no reason to assume that any of these is true, given MBlume's description of the situation. (In particular, the assumption regarding what each knows about the other seems to me, from Aumann's description, to imply more detailed knowledge of one another's cognitive faculties than any human has of any other's.)
Clearly someone is failing (very broadly understood) at something, since at least one of the two doctors assigns 99% probability to something untrue. But, e.g., the following is perfectly consistent with the scenario as described (although unlikely):
Both doctors are superlatively intelligent, skillful and well informed (about medicine). Both have done the same, entirely sensible, tests; one has been the victim of extreme bad luck and got evidence at the 99.5% level for a wrong diagnosis. Both have also been victims of further mischance, and each has (unknown to the other) got evidence at the 99.5% level that the other is horrendously incompetent even though that is not actually true. (We can agree, I hope, that all this is possible, albeit very unlikely?)
Now each considers the evidence. A, before learning B's verdict:
P(malaria & B incompetent) = 0.995^2 ~= 0.99
P(malaria & B competent) = 0.995 . 0.005 ~= 0.005
P(bird flu & B incompetent) = 0.995 . 0.005 ~= 0.005
P(bird flu & B competent) = 0.005 . 0.005 = 0.000025
Now for a Bayesian update based on B's opinion. Some kinda-plausible figures:
P(B gets given result | malaria & B incompetent) = 0.1
P(B gets given result | malaria & B competent) = 0.001
P(B gets given result | bird flu & B incompetent) = 0.1
P(B gets given result | bird flu & B competent) = 0.99
So the new odds are roughly 0.099 : 0.000005 : 0.0005 : 0.000025, giving a probability of about 99.5% for the "malaria & B incompetent" option.
B goes through an exactly parallel calculation, favouring "bird flu & A incompetent".
Both doctors have been unlucky, but neither has been irrational. Those who think Aumann's theorem requires one of them to have been irrational given the data: please explain what's impossible in the above scenario.
If each scientist has gotten evidence at the 99.5% level that the other is horrendously incompetent, then they should have no problem convincing each other that the other is incompetent with said evidence. (Unless one of them has additional evidence to defend their competency, in which case they will agree on how the additional evidence should change the assessment.) The idea being that with sufficient communication, they have exactly the same information and thus must make the same conclusions.
On the other hand, the requirement of the same priors is interesting. Mightn't this be how they could rationally come to different conclusions?
Let us say you are a doctor, and you are dealing with a malaria epidemic in your village. You are faced with two problems. First, you have no access to the drugs needed for treatment. Second, you are one of two doctors in the village, and the two of you cannot agree on the nature of the disease itself. You, having carefully tested many patients, being a highly skilled, well-educated diagnostician, have proven to yourself that the disease in question is malaria. Of this you are >99% certain. Yet your colleague, the blinkered fool, insists that you are dealing with an outbreak of bird flu, and to this he assigns >99% certainty.
Well, it need hardly be said that someone here is failing at rationality. Rational agents do not have common knowledge of disagreements etc. But... what can we say? We're human, and it happens.
So, let's say that one day,
OmegaDr. House calls you both into his office and tells you that he knows, with certainty, which disease is afflicting the villagers. As confident as you both are in your own diagnoses, you are even more confident in House's abilities. House, however, will not tell you his diagnosis until you've played a game with him. He's going to put you in one room and your colleague in another. He's going to offer you a choice between 5,000 units of malaria medication, and 10,000 units of bird-flu medication. At the same time, he's going to offer your colleague a choice between 5,000 units of bird-flu meds, and 10,000 units of malaria meds.(Let us assume that keeping a malaria patient alive and healthy takes the same number of units of malaria meds as keeping a bird flu patient alive and healthy takes bird flu meds).
You know the disease in question is malaria. The bird-flu drugs are literally worthless to you, and the malaria drugs will save lives. You might worry that your colleague would be upset with you for making this decision, but you also know House is going to tell him that it was actually malaria before he sees you. Far from being angry, he'll embrace you, and thank you for doing the right thing, despite his blindness.
So you take the 5,000 units of malaria medication, your colleague takes the 5,000 units of bird-flu meds (reasoning in precisely the same way), and you have 5,000 units of useful drugs with which to fight the outbreak.
Had you each taken that which you supposed to be worthless, you'd be guaranteed 10,000 units. I don't think you can claim to have acted rationally.
Now obviously you should be able to do even better than that. You should be able to take one another's estimates into account, share evidence, revise your estimates, reach a probability you both agree on, and, if the odds exceed 2:1 in one direction or the other, jointly take 15,000 units of whatever you expect to be effective, and otherwise get 10,000 units of each. I'm not giving out any excuses for failing to take this path.
But still, both choosing the 5,000 units strictly loses. If you can agree on nothing else, you should at least agree that cooperating is better than defecting.
Thus I propose that the epistemic prisoner's dilemma, though it has unique features (the agents differ epistemically, not preferentially) should be treated by rational agents (or agents so boundedly rational that they can still have persistent disagreements) in the same way as the vanilla prisoner's dilemma. What say you?