If a few years after being a church-going Christian he's now at 5% probability of God, I wouldn't say he's doing anything wrong.
I disagree. Changes of opinion about conclusions should be swift and decisive, which doesn't mean that in the same movement you should wipe out from your mind the understanding and experience gained from the previous, invalidated position. Changing your mind swiftly, while keeping the background that allows to regain mastery in the disbelieved position in case it returns to plausibility, seems to bring the best of both practices. This is the attitude I have towards Robin Hanson's modesty argument: you should change your conclusion towards the universally accepted one, but still be ready to change it back in an informed manner.
The costs of acting on the overly inert and therefore wrong conclusion project on everything you do, while the cost of still thinking of and remembering the background that lead to the very unlikely conclusion are low enough to keep them. At one point you stop actively researching your unlikely idea, at the next point you stop thinking about it, and some clicks further you forget it entirely. Just don't be overconfident about the conclusion, thinking you know how overwhelmingly, 10 to the minus 1000 unlikely it is, when in fact it turns out to be correct.
“Everyone complains of his memory, but nobody of his judgment." This maxim of La Rochefoucauld rings as true today as it did back in the XVIIth century. People tend overestimate their reasoning abilities even when this overconfidence has a direct monetary cost. For instance, multiple studies have shown that investors who are more confident of their ability to beat the market receive lower returns on their investments. This overconfidence penalty applies even to the supposed experts, such as fund managers.
So what an expert rationalist should do to avoid this overconfidence trap? The seeming answer is that we should rely less on our own reasoning and more on the “wisdom of the crowds”. To a certain extent this is already achieved by the society pressure to conform, which acts as an internal policeman in our minds. Yet those of us who deem themselves not very susceptible to such pressures (overconfidence, here we go again) might need to shift their views even further.
I invite you now to experiment on how this will work in practice. Quite a few of the recent posts and comments were speaking with derision about religion and the supernatural phenomena in general. Did the authors of these comments fully consider the fact that the existence of God is firmly believed by the majority? Or that this belief is not restricted to the uneducated but shared by many famous scientists, including Newton and Einstein? Would they be willing to shift their views to accommodate the chance that their own reasoning powers are insufficient to get the right answer?
Let the stone throwing begin.