I see a lot of people arguing that "2", "3", "+", and "=" are defined in terms of the Peano axioms, and as such, aren't actually relevant to the behavior of physical objects. They say that the axioms pin down the numbers, regardless of how physical objects behave or start behaving.

But the Peano axioms use something called a "successor" to generate the natural numbers. And how do we figure out what the successors are? Well, one notation is to append an "S" to the previous number to indicate that number's successor, such that the successor of "SS0" is "SSS0". Then "+" would mean "repeatedly apply the successor operation to the number to the right of this operator for however many times indicated by the number to the left of this operator", and "=" would indicate "these two numbers, when written out using successor-notation, have the character 'S' occur the same number of times".

So how do we calculate "2 + 2"? Well, we take the number of occurrences of the character "S", count them up, then put the total number of occurrences in front of a single "0": "SSSS0", which we interpret as "4". "2 + 2 = 4".

But now let's imagine your brain's visual cortex (or whatever it uses to visualize things) being messed up somehow. Actually, let's go further than that, and suppose everyone's visual cortexes got messed up in the same way, so that now, when we visualize two objects and two more objects, and put them together, we see, not four objects, but three. So if you were to try and visualize a group of two dots coming together with another group of two dots, you would end up seeing, in your mind's eye, a final group of three dots.

Now imagine counting the number of times "S" occurs in "SS0", then putting two of those groups of "S" characters together. How many "S" character are in the final group? Visualize it, now...

Why, three. Of course there's three. If you take "SS" and put it with "SS", of course you get "SSS". What's that? You say you're getting "SSSS"? Where are you getting that extra "S" from? What? No--can't you see it? It's obviously three--how can you put those two groups together and get four?

So, "SS0 + SS0 = SSS0".

My point is this: you can't just point to the Peano axioms and say, "Ha! Your hypothetical situation involving the behavior of mere physical objects is meaningless before the might of my absolute mathematical assumptions!" Remember, when you try to perform "logic" on those axioms, you're still using your brain to do it. And however ivory-tower untouchable you imagine your axioms to be, your brain is a real, physical object performing real physical computations. If we lived in a slightly different universe, your brain would take one look at "SS0" and "SS0", visually add together the number of "S" characters, and see "SSS0". Anyone who saw "SSSS0" would be seen as crazy, or brain-damaged, or something.

Physics trumps math and logic.