Very nice and convincing argument. There were some moments when thinking about it when I though your argument defeated my view. Sadly, we're not quite there yet.
Trying to add 2 miles to 2 apples does not make sense. There is no physical representation of such an operation. So you can't try to abstract that into numbers. Here's an example, to clarify:
Let's say I've got a bag of 2 apples, I add 2, and one falls through the hole in my bag. The number of apples in my bag is 2+2-1=3. The first 2 is an abstraction of the original number of apples in my bag, the second 2 is the number of apples I added, and -1 is the number of apples that fell out through the hole, and 3 is the total. Everything in this equation can be mapped back to reality by adding in details. But in 2 apples + 2 miles, the + cannot be mapped back to reality. And so it is not representative of apples.
I can think of an easy attack on this, by saying that 2 apples + 2 oranges does make sense in our world. But this is a disguised incorrectness. The oranges do not fall into the criterion of "apple", so it does not actually make sense to add them using the '+apples' operator. Of course, we could generalize the '+' operation to mean '+fruit', but then it becomes 2 fruit + 2 fruit, which does make sense, and is true whether you reject my current views or not.
I'm looking forward to your next argument :).
I put two apples in my one bag, then walk two miles and add two more apples to the bag. There is one hole in the bag, and one of the apples falls out.
2 (apples) + 2 (miles) + 1 (bag) - 1 (hole) - 1 (apple) + 2 (apples) = 5
I end up five miles from where I started, because I dropped units from my quantities and did operations on the numbers.
Or, I could apply a constant force of 5 newtons to an object massing 25 kg for a duration of 8 seconds. I change it's velocity by 5N8S(1MKG/(NSEC^2))/25KG 1.6 M/S. By conserving units on all quantities, I convert force-ti...
In his latest post, Logical Pinpointing, Eliezer talks about the nature of math. I have my own views on the subject, but when writing about them, it became too long for a comment, so I'm posting it here in discussion.
I think it's important to clarify that I am not posting this under the guise that I am correct. I'm just posting my current view of things, which may or may not be correct, and which has not been assessed by others yet. So though I have pushed myself to write in a confident (and clear) tone, I am not actually confident in my views ... well, okay, I'm lying. I am confident in my views, but I know I shouldn't be, so I'm trying to pretend to myself that I'm posting this to have my mind changed, when in reality I'm posting it with the stupid expectation that you'll all magically be convinced. Thankfully, despite all that, I don't have a tendency to cling to beliefs which I have seen proven wrong, so I will change my mind when someone kicks my belief's ass. I won't go down without a fight, but once I'm dead, I'm dead.
Before I can share my views, I need to clarify a few concepts, and then I'll go about showing what I believe and why.
Abstractions
Abstract models are models in which some information is ignored. Take, for example, the abstract concept of the ball. I don't care if the ball is made of rubber or steel, nor do I care if it has a radius of 6.7cm or a metre, a ball is a ball. I couldn't care less about the underlying configuration of quarks in the ball, as long as it's a sphere.
Numbers are also abstractions. If I've got some apples, when I abstract this into a number, I don't care that they are apples. All I care about is how many there are. I can conveniently forget about the fact that it's apples, and about the concept of time, and the hole in my bag through which the apples might fall.
Axiomatic Systems (for example, Peano Arithmetic)
Edit: clarified this paragraph a bit.
I don't have much to say about these for now, except that they can all be reduced to physics. Given that mathematicians store axiomatic systems in their minds, and use them to prove things, they cannot help but be reducible to physical things, unless the mind itself is not physical. I think most LessWrongers, being reductionists, believe this, so I won't go into much more detail. I'll just say that this will be important later on.
I Can Prove Things About Numbers Using Apples
Let's say I have 2 apples. Then someone gives me 2 more. I now have 4. I have just shown that 2+2=4, assuming that apples behave like natural numbers (which they do).
But let's continue this hypothetical. As I put my 2 delicious new apples into my bag, one falls out through a hole in my bag. So if I count how many I have, I see 3. I had 2 to begin with, and 2 more were given to me. It seems I have just shown 2+2=3, if I can prove things about numbers using apples.
The problem lies in the information that I abstracted away during the conversion from apples to numbers. Because my conversion from apples to numbers failed to include information about the hole, my abstract model gave incorrect results. Like my predictions about balls might end up being incorrect if I don't take into account every quark that composes it. Upon observing that the apple had fallen through the hole, I would realize that an event which rendered my model erroneous had occurred, so I would abstract this new event into -1, which would fix the error: 2+2-1=3.
To summarize this section, I can "prove" things about numbers using apples, but because apples are not simple numbers (they have many properties which numbers don't), when I fail to take into account certain apple properties which will affect the number of apples I have, I will get incorrect results about the numbers.
Apples vs Peano Arithmetic
We know that Peano Arithmetic describes numbers very well. Numbers emerge in PA; we designed PA to do so. If PA described unicorns instead, it wouldn't be very useful. And if PA emerges from the laws of physics (we can see PA emerge in mathematicians' minds, and even on pieces of paper in the form of writing), then the numbers which emerge from PA emerge from the laws of physics. So there is nothing magical about PA. It's just a system of "rules" (physical processes) from which numbers emerge, like apples (I patched up the hole in my bag ;) ).
Of course, PA is much more convenient for proving things about numbers than apples. But they are inherently just physical processes from which I have decided to ignore most details, to focus only on the numbers. In my bag of 3 apples, if I ignore that it's apples there, I get the number 3. In SSS0, if I forget about the whole physical process giving emergence to PA, I am just left with 3.
So I can go from 3 apples to the number 3 by removing details, and from the number 3 to PA by adding in a couple of details. I can likewise go from PA to numbers, and then to apples.
To Conclude, Predictions are "Proofs"
From all this, I conclude that numbers are simply an abstraction which emerges in many places thanks to our uniform laws of physics, much like the abstract concept "ball". I also conclude that what we classify as a "prediction" is in fact a "proof". It's simply using the rules to find other truths about the object. If I predict the trajectory of a ball, I am using the rules behind balls to get more information about the ball. If I use PA or apples to prove something about numbers, I am using the rules behind PA (or apples) to prove something about the numbers which emerge from PA (or apples). Of course, the proof with PA (or apples) is much more general than the "proof" about the ball's trajectory, because numbers are much more abstract than balls, and so they emerge in more places.
So my response to this part of Eliezer's post:
Logic is stable for the same reasons the laws of physics are stable. Logic emerges from the laws of physics, and the laws of physics themselves are stable (or so it seems). In this way, I dissolve the question and mix it with the question why the laws of physics are stable -- a question which I don't know enough to attempt to answer.
Edit: I'm going to retract this and try to write a clearer post. I still have not seen arguments which have fully convinced me I am wrong, though I still have a bit to digest.