Is it just coincidence that these nonstandard models don't show up anywhere in the empirical sciences, but real numbers and complex numbers do?

The process by which the concepts "natural / real / complex numbers" vs. "nonstandard models of PA" were generated is very different. In the first case, mathematicians were trying to model various aspects of the world around them (e.g. counting and physics). In the second case, mathematicians were trying to pinpoint something else they already understood and ended up not quite getting it because of logical subtleties.

I'm not sure how to explain what I mean by "delicate." It roughly means "unlikely to have been independently invented by alien mathematicians." In order for alien mathematicians to independently invent the notion of a nonstandard model of PA, they would have to have independently decided that writing down the first-order Peano axioms is a good idea, and I just don't find this all that likely. On the other hand, there are various routes alien mathematicians might take towards independently inventing the complex numbers, such as figuring out quantum mechanics.

Before we achieved categoricity, we already knew what mathematical structure we wanted to talk about, and afterwards, it's still useful to add more axioms if we want to prove more theorems.

I guess Eliezer's intended response here is something like "but when you want to explain to an AI what you mean by the natural numbers, you can't just say The Things You Use To Count With, You Know, Those."