I'm a little confused as to which of two positions this is advocating:

1. Numbers are real, serious things, but the way that we pick them out is by having a categorical set of axioms. They're interesting to talk about because lots of things in the world behave like them (to some degree).

2. Mathematical talk is actually talk about what follows from certain axioms. This is interesting to talk about because lots of things obey the axioms and so exhibit the theorems (to some degree).

Both of these have some problems. The first one requires you to have weird, non-physical numbery-things. Not only this, but they're a special exception to the theory of reference that's been developed so far, in that you can refer to them without having a causal connection.

The second one (which is similar to what I myself would espouse) doesn't have this problem, because it's just talking about what follows logically from other stuff, but you do then have to explain why we seem to be talking about numbers. And also what people were doing talking about arithmetic before they knew about the Peano axioms. But the real bugbear here is that you then can't really explain logic as part of mathematics. The usual analyisis of logic that we do in maths with the domain, interpretation, etc. can't be the whole deal if we're cashing out the mathematics in terms of logical implication! You've got to say something else about logic.

(I think the answer is, loosely, that

1. the "numbers" we talk about are mainly fictional aides to using the system, and
2. the situation of pre-axiom speakers is much like that of English speakers who nonetheless can't explain English grammar.
3. I have no idea what to say about logic! )

I'm curious which of these (or neither) is the correct interpretation of the post, and if it's one of them, what Eliezer's answers are... but perhaps they're coming in another post.