Yvain, it seems like some of this is potentially answered by how this interacts with other cognitive biases present.

Re: specific delusions, when you have an entire class of equally-explanatory hypotheses, how do you choose between them? The availability heuristic! These hypotheses do have to come from somewhere inside the neural network after all. You could argue that availability is a form of "priors", but these "priors" are formed on the level of neurons themselves and not a specific brain region: some connection strengths are stronger than others.

I would not wish brain damage on anyone, but should one of us have that unfortunate circumstance befall us I would be extremely inclined to go talk to them. I am so damn curious what this feels like from the inside! I am somewhat embarrassed to admit that the thought of having to build completely new neural connections to get around existing damage sounds like an insanely interesting challenge...

I also wonder about reasoning our way out of delusional states. The closest parallel that most people have access to would be the use of various psychoactives. I have heard multiple reports of people who have reasoned their way out of delusional conclusions on cannabinoid agonists and 5-HT2A agonists (and dopamine agonists, with lesser evidence).

The most difficult challenge would appear to be kappa opioid agonism, a dissociative state induced by the federally-legal herb salvia divinorum. Most users report being unaware they ingested a substance at all, no awareness of having a body, and no concept of self-identity, coincident with extreme perceptual distortions. I am no longer clear what Bayesian reasoning would even look like for some points in mindspace.

Edit: I thought of another relevant state: delirium induced by anticholinergics. Unlike 5-HT2A agonists where people do not confuse perceptual distortions for reality, in delirious states people do routinely believe that what they are perceiving is actually occurring. Unfortunately these states are widely regarded as unpleasant, and no rationalist I know personally has experimented with sufficiently large doses of anticholinergics.