You're right about the probabilistic statements, with a potentially tangential elaboration. There are nonsense sentences--not contradictions ("A and not A") but things that fail to parse ("A and")--and it doesn't make sense to assign probabilities to those. One might claim that "God exists" is a nonsense sentence in that way, but I think most New Atheists don't take that approach.
The distinction that people are drawing is basically which framing should have the benefit of the doubt, since not believing a new statement is the default. This is much more important for social rationality / human psychology than it is for Bayesianism, where you just assign a prior and then start calculating likelihood ratios.
This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.
Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.
To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.