Followup to: The Strangest Thing An AI Could Tell You
Brain damage patients with anosognosia are incapable of considering, noticing, admitting, or realizing even after being argued with, that their left arm, left leg, or left side of the body, is paralyzed. Again I'll quote Yvain's summary:
After a right-hemisphere stroke, she lost movement in her left arm but continuously denied it. When the doctor asked her to move her arm, and she observed it not moving, she claimed that it wasn't actually her arm, it was her daughter's. Why was her daughter's arm attached to her shoulder? The patient claimed her daughter had been there in the bed with her all week. Why was her wedding ring on her daughter's hand? The patient said her daughter had borrowed it. Where was the patient's arm? The patient "turned her head and searched in a bemused way over her left shoulder".
A brief search didn't turn up a base-rate frequency in the population for left-arm paralysis with anosognosia, but let's say the base rate is 1 in 10,000,000 individuals (so around 670 individuals worldwide).
Supposing this to be the prior, what is your estimated probability that your left arm is currently paralyzed?
Added: This interests me because it seems to be a special case of the same general issue discussed in The Modesty Argument and Robin's reply Sleepy Fools - when pathological minds roughly similar to yours update based on fabricated evidence to conclude they are not pathological, under what circumstances can you update on different-seeming evidence to conclude that you are not pathological?
Very low, because I can (and just did, to make sure) easily contrive situations where the visible present is determined by the recent past presence of two arms and where I would have been forced to strain credulity if I were justifying a negative result. In this case, the experiment was dropping two things at once causing two other objects to roll and meet in the middle (and not on one side, indicating an asymmetric drop).
Of course, it's absolutely inconceivable that, knowing what you expected, you turned toward the point where they met...