I am using counting to refer to any process by which a number is assigned as a symbol for a property.
I use + in both its concrete - this field now contains two sheep - and its abstract - this set now contains two sheep - meanings. I hope the context makes it clear when I am using each meaning, and why; because the lack of clarity is, in fact, important. See my response to the first comment, in which I deliberately used its concrete meaning in response to somebody using the abstract meaning. Both of us are in fact correct, and the confusion itself is meaningful.
Because it -is- a linguistic problem - and because linguistic problems can, in fact, be real problems.
Viliam Bur encapsulated what I was trying to establish pretty well: "I think the idea was that perhaps for some alien intelligence the "+" symbol could be useless or even meaningless, and something else would be in the place of "the most simple abstract computational operation". Then the aliens could naively expect that every intelligence in the universe must know this very basic operation."
But more than that - if your basic operations are different, it's possible to come to very different conclusions.
One of my biggest revelations in mathematics was in statistics, when, after the class (including me) worked unsuccessfully for a couple of hours to integrate an equation, the instructor (who I'm sure was laughing at us) walked up to the board, converted into a different coordinate system, and integrated the now very easily integrated equation in about thirty seconds.
If your basic operations are different, you might be able to come to conclusions you otherwise were unable to come to.
And in response to this I'll ask: How many people accept this about the fundamental descriptors they use in their mathematics? How many people operate on the assumption that mathematics are a universal language, or that the universe runs on mathematics (which I generally interpret to mean -their- mathematics)?
And yet most recipes follow a basically arithmetic formula: Add 5 units of meat, add 375 units of heat over 10 units of time.
The point, although it's a long way around to coming, is that arithmetic may be a fundamentally -human- way of evaluating the universe. It goes without saying that it's not the ideal model in many scenarios. And for those considering how to build AI, particularly those interested in solving intractable problems, it may be worth letting it come to its own model.
Eliezer's post How To Convince Me That 2 + 2 = 3 has an interesting consideration - if putting two sheep in a field, and putting two more sheep in a field, resulted in three sheep being in the field, would arithmetic hold that two plus two equals three?
I want to introduce another question. What exactly are you counting?
Imagine one sheep in one field, and another sheep in another. Now put them together. Do you now have two sheep?
"Of course!"
Ah, but is that -all- you have?
"What?"
Two sheep are more than twice as complex as a single sheep. It takes more than twice as many bits to describe two sheep than it takes to describe a single sheep, because, in addition to those two sheep, you now also have to describe their relationship to one another.
Or, to phrase it slightly differently, does 1+1=2?
Well, the answer is, it depends on what you're counting.
If you're counting the number of discrete sheep, 1+1=2. However, why is the number of discrete sheep meaningful?
If you're a hunter counting, not herded sheep, but prey - two sheep is, roughly, twice as much meat as one sheep. 1+1=2. If you're a herder, however, two sheep could be a lot more valuable than one - two sheep can turn into three sheep, if one is female and one is male. The value of two sheep can be more than twice the value of a single sheep. And if you're a hypercomputer running Solomonoff Induction to try to describe sheep positional vectors, two sheep will have a different complexity than twice the complexity of a single sheep.
Which is not to say that one plus one does not equal two. It is, however, to say that one plus one may not be meaningful as a concept outside a very limited domain.
Would an alien intelligence have arrived at arithmetic? Depends on what it counts. Is arithmetic correct?
Well, does a set of two sheep contain only two sheep, or does it also contain their interactions? Depends on your problem domain; 1+1 might just equal 2+i.