Whatever you could possibly know and value about reality can only exist independently of the physical universe.

That is not what I am saying. I mean that things that we think of as tautologies, or purely logical truths, which are true no matter what universe we are in, exist independently of the physical universe. Facts about the physical universe are not in this class. Indeed, the entanglement of our physical brains with these logical truths is an example of a fact about the physical universe that, of course, depends on the the universe.

If your uncertainty about math doesn't indicate uncertainty of the math, and it's an argument for math being otherworldly...

You have my argument backwards. I first make the point that facts about math are not facts about the physical universe to support that the uncertainty we have about math, which exists in our heads, in our physical universe, does not exist in math itself. The argument does not work the other way, there are plenty of instances of uncertainty in our minds that are not uncertainty in the things elsewhere in the physical universe that they are about.

My comment was an attempt to explain why we need observation to believe things that are objectively true regardless of the world we exists in. Basically, we need evidence that our brains, existing in the physical worlds, are suitable for representing the logical truths.