I hate to break it to you, but if setting two things beside two other things didn't yield four things, then number theory would never have contrived to say so.

Numbers were invented to count things, that is their purpose. The first numbers were simple scratches used as tally marks circa 35,000 BC. The way the counts add up was derived from the way physical objects add up when grouped together. The only way to change the way numbers work is to change the way physical objects work when grouped together. Physical reality is the basis for numbers, so to change number theory you must first show that it is inconsistent with reality.

Thus numbers have a definite relation to the physical world. Number theory grew out of this, and if putting two objects next to two other objects only yielded three objects when numbers were invented over forty thousand years ago, then number theory must reflect that fact or it would never have been used. Consequently, suggesting 2+2=4 would be completely absurd, and number theorists would laugh in your face at the suggestion. There would, in fact, be a logical proof that 2+2=3 (much like there is a logical proof that 2+2=4 in number theory now).

All of mathematics are, in reality, nothing more than extremely advanced counting. If it is not related to the physical world, then there is no reason for it to exist. It follows rules first derived from the physical world, even if the current principles of mathematics have been extrapolated far beyond the bounds of the strictly physical. I think people lose sight of this far too easily (or worse, never recognize it in the first place).

Mathematics are so firmly grounded in the physical reality that when observations don't line up with what our math tells us, we must change our understanding of reality, not of math. This is because math is inextricably tied to reality, not because it is separate from it.