Jaynes discusses Hempel's Paradox on pages 143 to 144 of Probability Theory: the Logic of Science. I take away a broad lesson: one must always know what alternative hypotheses are available. Failing to be clear about your alternative hypotheses is my first candidate for what you are doing wrong.
My second candidate comes from rule IIIb for plausible reasoning (page 9).
... always take into account all the evidence relevant to the question ...
One conspicuous feature of the world is the presence of rival faiths, each well attested by miracles about which the faithful will admit no doubts. This occurs both between religions (Muslim, Christian, Jew) and within religions (Orthodox, Catholic, Protestant).
It is tempting at this point to rush ahead down a well-worn path to atheism. "Since the rivals are mutually exclusive and the situation symmetrical they must all be wrong." I've put the argument in quotes because I'm suggesting that we hold off and pause for reflection. Since we can see what is coming, we face a choice. We can either leave out the existence of rival religions because, for many people, it settles the issue, and we feel that including them therefore prejudges the issue. Or we can include the existence of rival religions and wonder whether the situation really is symmetrical.
Rule IIIb requires us to ask whether the existence of rival religions (with well attested miracles ...) is relevant. We know that it is highly relevant. We even feel an itch to leave it out to avoid it over powering other considerations.
Rule IIIb then instructs us to take them into account. Ouch! The probability calculation has already gone badly wrong, just because it left out the rival religions and before we ever get to thinking about what the existence of rival religions implies for the issue we are considering.
A friend recently asked how strongly I believe that my deconversion from Christianity was not a mistake. Here's my response, and for those of you who are not Christians, I'm just wondering what numbers you would give:
"There is a part of me that wants to say the chance is far less than 1 percent. But when I consider what 1% must mean about my ability to follow complex arguments and base my judgement on the right premises, it seems absurd to say that.
Trying to honestly estimate the chance that I'm wrong about the Bible being generally reliable is a fascinating exercise... I know the number is low, but I'm not sure how low.
Today I would give myself a 1 in 20 chance of being wrong. If I were to consider the arguments of 20 other groups similar to Christian theologians, I would probably misunderstand them at least 1 time in 20. After talking with 20 groups that have a very different worldview, I might think they are all are mistaken, but once in a while, maybe 5% of the time, it would actually be me.
Wow, 5%!?! If I convert that into "There is a 5% probability that the God of the Bible exists and will send me to hell", I feel scared. But I know how to cheer myself up: I just say, "No way, the chance I'll end up in hell MUST be less than 5%. After all, the God of the Bible is CLEARLY just a big, mean alpha-monkey and... [rehearse all the atheistic arguments here]".
This back-and-forth from certainty to uncertainty makes me feel like I'm doing something seriously wrong.
So what about you? What chance do you place on some variant of Christianity turning up to be true, and what chance do you think a god of some sort exists?"
Numbers please.