Dreaming is the closest I've gotten to testing myself against the challenge of maintaining rationality under brain damage. So far, my trials have exhibited mixed results.
In one memorable dream a few years ago, I dreamed that the Wall Street Journal had published an article about "Eliezer Yudkowsky", but it wasn't me, it was a different "Eliezer Yudkowsky", and in the dream I wondered if I needed to write a letter to clarify this. Then I realized I was dreaming within the dream... and worried to myself, still dreaming: "But what if the Wall Street Journal really does have an article about an 'Eliezer Yudkowsky' who isn't me?"
But then I thought: "Well, the probability that I would dream about a WSJ article like that, given that a WSJ article like that actually exists in this morning's paper, is the same as the probability that I would have such a dream, given that no such article is in this morning's paper. So by Bayes's Theorem, the dream isn't evidence one way or the other. Thus there's no point in trying to guess the answer now - I'll find out in the morning whether there's an article like that." And, satisfied, my mind went back to ordinary sleep.
I find it fascinating that I was able to explicitly apply Bayes's Theorem in my sleep to correctly compute the 1:1 likelihood ratio, but my dreaming mind didn't notice the damaged prior - didn't notice that the prior probability of such a WSJ article was too low to justify raising the hypothesis to my attention.
At this point even I must concede that there is something to the complaint that, in real-world everyday life, Bayesians dispense too little advice about how to compute priors. With a damaged intuition for the weight of evidence, my dreaming mind was able to explicitly compute a likelihood ratio and correct itself. But with a damaged intuition for the prior probability, my mind didn't successfully check itself, or even notice a problem - didn't get as far as asking "But what is the prior probability?"
On July 20 I had an even more dramatic dream - sparking this essay - when I dreamed that I'd googled my own name and discovered that one of my OBLW articles had been translated into German and published, without permission but with attribution, in a special issue of the Journal of Applied Logic to commemorate the death of Richard Thaler (don't worry, he is in fact still alive)...
Then I half woke-up... and wondered if maybe one of my OBLW articles really had been "borrowed" this way. But I reasoned, in my half-awake state, that the dream couldn't be evidence because it didn't form part of a causal chain wherein the outside environment impressed itself onto my brain, and that only actual sensory impressions of Google results could form the base of a legitimate chain of inferences.
So - still half-asleep - I wanted to get out of bed and actually look at Google, to see if a result turned up for the Journal of Applied Logic issue.
And several times I fell back asleep and dreamed I'd looked at Google and seen the result; but each time on half-awaking I thought: "No, I still seem to be in bed; that was a dream, not a sense-impression, so it's not valid evidence - I still need to actually look at Google." And the cycle continued.
By the time I woke up entirely, my brain had fully switched on and I realized that the prior probability was tiny; and no, I did not bother to check the actual Google results. Though I did Google to check whether Richard Thaler was alive, since I was legitimately unsure of that when I started writing this post.
If my dreaming brain had been talking in front of an audience, that audience might have applauded the intelligent-sounding sophisticated reasoning about what constituted evidence - which was even correct, so far as it went. And yet my half-awake brain didn't notice that at the base of the whole issue was a big complicated specific hypothesis whose prior probability fell off a cliff and vanished. EliezerDreaming didn't try to measure the length of the message, tot up the weight of burdensome details, or even explicitly ask, "What is the prior probability?"
I'd mused before that the state of being religious seemed similar to the state of being half-asleep. But my recent dream made me wonder if the analogy really is a deep one. Intelligent theists can often be shepherded into admitting that their argument X is not valid evidence. Intelligent theists often confess explicitly that they have no supporting evidence - just like I explicitly realized that my dreams offered no evidence about the actual Wall Street Journal or the Journal of Applied Logic. But then they stay "half-awake" and go on wondering whether the dream happens to be true. They don't "wake up completely" and realize that, in the absence of evidence, the whole thing has a prior probability too low to deserve specific attention.
My dreaming brain can, in its sleep, reason explicitly about likelihood ratios, Bayes's Theorem, cognitive chains of causality, permissible inferences, strong arguments and non-arguments. And yet still maintain a dreaming inability to reasonably evaluate priors, to notice burdensome details and sheer ridiculousness. If my dreaming brain's behavior is a true product of dissociation - of brainware modules or software modes that can be independently switched on or off - then the analogy to religion may be more than surface similarity.
Conversely it could just be a matter of habits playing out in in my dreaming self; that I habitually pay more attention to arguments than priors, or habitually evaluate arguments deliberately but priors intuitively.
Over this past summer, I decided to do some tests with my dreams. Overall this has gone nowhere (it doesn't help that I've had difficulty keeping myself motivated enough to keep it going for long). I didn't put it in Bayesian terms at the time (despite having been reading LessWrong), but ignoring weak priors is one of the biggest rationality fails I've experienced in dreams (I've documented quite a few incidents). There have been some cases where I have done something resembling updating dream priors, specifically as it relates to elevators, as there was a period from 2004-2006 in which elevators in dreams would frequently behave in an unrealistic manner (traveling more floors than would make sense, usually down, or having the door or floor become structurally unsound). It got to the point that I actively avoided elevators in dreams if I was interested in the situation I was experiencing. I've sort of half-tried for lucid dreaming, and consequently only been sort of half-successful. I've found that anxiety almost always outweighs desire when it comes to influencing the experience. I did try a little test where I would try to will the time on a display to be a particular number, then checked to see if it worked. I tried this several times, but by the time I remembered to write down my results, I only remembered two of the trials (though I do remembering them being pretty representative of the rest of them). I was able to get the time to appear close (no more than five minutes off, and usually much closer) to the target. I'd like to do an experiment involving my vision, but that's proven much more difficult than I'd like. My left eye has been nonfunctional since birth, my right eye started scarring around age 3, stabalized at "I could read subtitles if you pause the video, give me inch-thick glasses and let me sit close to the screen for a couple minutes", then dropped rapidly toward useless after I visited Las Vegas at age 14 (there are several factors that probably came to a head with that incident). Vision in my dreams took much longer to decline, but gradually got to the point where clear visuals are a novelty even while asleep.
Update: Finding this article got me to try again, and while overall it was a vague mess (though I attempted some more number experiments involving other people/NPCs), things got interesting near the end. I did have limited success working with visuals, and within moments of noticing this, observed a dog run up a slide and consciously decided that the probability of this having happened in reality and not having been made up just for the dream was pretty low. It's worth noting that I was fully aware it was a dream by that point.