I've never been completely happy with the "I could make 1M similar statements and be wrong once" test. It seems, I dunno, kind of a frequentist way of thinking about the probability that I'm wrong. I can't imagine making a million statements and have no way of knowing what it's like to feel confidence about a statement to an accuracy of one part per million.

Other ways to think of tiny probabilities:

(1) If probability theory tells me there's a 1 in a billion chance of X happening, then P(X) is somewhere between 1 in a billion and P(I calculated wrong), the latter being much higher.

If I were running on hardware that was better at arithmetic, P(I calculated wrong) could be got down way below 1 in a billion. After all, even today's computers do billions of arithmetic operations per second. If they had anything like a one-in-a-billion failure rate per operation, we'd find them much less useful.

(2) Think of statements like P(7 is prime) = 1 as useful simplifications. If I am examining whether 7 is prime, I wouldn't start with a prior of 1. But if I'm testing a hypothesis about something else and it depends on (among other things) whether 7 is prime, I wouldn't assign P(7 is prime) some ridiculously specific just-under-1 probability; I'd call it 1 and simplify the causal network accordingly.