Eliezer's post How To Convince Me That 2 + 2 = 3 has an interesting consideration - if putting two sheep in a field, and putting two more sheep in a field, resulted in three sheep being in the field, would arithmetic hold that two plus two equals three?
I want to introduce another question. What exactly are you counting?
Imagine one sheep in one field, and another sheep in another. Now put them together. Do you now have two sheep?
"Of course!"
Ah, but is that -all- you have?
"What?"
Two sheep are more than twice as complex as a single sheep. It takes more than twice as many bits to describe two sheep than it takes to describe a single sheep, because, in addition to those two sheep, you now also have to describe their relationship to one another.
Or, to phrase it slightly differently, does 1+1=2?
Well, the answer is, it depends on what you're counting.
If you're counting the number of discrete sheep, 1+1=2. However, why is the number of discrete sheep meaningful?
If you're a hunter counting, not herded sheep, but prey - two sheep is, roughly, twice as much meat as one sheep. 1+1=2. If you're a herder, however, two sheep could be a lot more valuable than one - two sheep can turn into three sheep, if one is female and one is male. The value of two sheep can be more than twice the value of a single sheep. And if you're a hypercomputer running Solomonoff Induction to try to describe sheep positional vectors, two sheep will have a different complexity than twice the complexity of a single sheep.
Which is not to say that one plus one does not equal two. It is, however, to say that one plus one may not be meaningful as a concept outside a very limited domain.
Would an alien intelligence have arrived at arithmetic? Depends on what it counts. Is arithmetic correct?
Well, does a set of two sheep contain only two sheep, or does it also contain their interactions? Depends on your problem domain; 1+1 might just equal 2+i.
In the strictest sense, "adding" sheep is a category error. Sheep are physical objects, you can put two sheep in a pen or imagine putting two sheep in a pen, but you aren't "adding" them, that's for numbers. Arithmetic is merely a map that can be fruitfully used to model (among many other things) certain aspects of sheep collection, separating sheep into groups, etc, under certain circumstances. When mathematical maps work especially well, they risk being confused with the territory, which is what I think is going on here. The "female sheep + male sheep" example should be thought of as an aspect of "putting sheep in pens for long periods of time" which addition does not model, not as an exception to "1+1=2".
In the strictest sense, adding anything except abstract numbers is a category error.
My point has less to do with the map not correlating to the territory, as that the map is not necessarily as intuitive, as a map, as we might expect; arithmetic may be a lot more tightly correlated to a distinctly human way of perceiving the world than naive expectation would lead to be the case.