I think I'm starting to get it. That there is no property that a natural number could be defined as having, that a infinite chain couldn't also satisfy in theory.

That's really disappointing. I took a course on logic and the most inspiring moment was when the professor wrote down the axioms of peano arithmitic. They are more or less formalizations of all the stuff we learned about numbers in grade school. It was cool that you could just write down what you are talking about formally and use pure logic to prove any theorem with them. It's sad that it's so limited you can't even express numbers.