because, coming from a devout (can I use that word?) Bayesian, it's akin to a mathematician saying that if 2+2 ceases to be 4, that equation goes out the window.

Duhem-Quine is just as much a problem there; from Ludwig Wittgenstein, Remarks on the Foundations of Mathematics:

"If a contradiction were now actually found in arithmetic – that would only prove that an arithmetic with such a contradiction in it could render very good service; and it would be better for us to modify our concept of the certainty required, than to say it would really not yet have been a proper arithmetic."

Indeed.

To generalize, when we run into skeptical arguments employing the above circularity or fundamental uncertainties, I think of Kripke:

"A skeptical solution of a philosophical problem begins... by conceding that the skeptic's negative assertions are unanswerable. Nevertheless our ordinary practice or belief is justified because—contrary appearances notwithstanding—it need not require the justification the sceptic has shown to be untenable. And much of the value of the sceptical argument consists precisely in the fact that he has shown that an ordinary practice, if it is to be defended at all, cannot be defended in a certain way."