As a quick heuristic, Edward Nelson has done some very impressive stuff. But a) his investigation into this question is by his own account motivated by pre-existing beliefs and b) he was 84 at the time he wrote this essay. The two of those don't give me a great amount of confidence in the likelyhood that this is correct.
As a more direct issue, saying that induction doesn't hold in general is akin to saying that I can have an infinite chain of dominoes such that each one will cause the next to fall if it falls over, and that the first has fallen, but all the dominoes won't eventually fall. That's an essentially physical representation of induction. If that's not valid then things are very weird.
The mathematics he is summarizing is utterly correct. Actually the small things I singled out I think are due to Godel and not to Nelson. But my summary is much more likely to have errors.
The dominoes analogy is revealing. If you lay out one million actual dominoes as usual and push over the first one I think it's very unlikely that the millionth one will fall.
The great Catholic mathematician Edward Nelson does not believe in completed infinity, and does not believe that arithmetic is likely to be consistent. These beliefs are partly motivated by his faith: he says arithmetic is a human invention, and compares believing (too strongly) in its consistency to idolatry. He also has many sound mathematical insights in this direction -- I'll summarize one of them here.
http://www.mediafire.com/file/z3detbt6int7a56/warn.pdf
Nelson's arguments flow from the idea that, contra Kronecker, numbers are man-made. He therefore does not expect inconsistencies to have consequences that play out in natural or divine processes. For instance, he does not expect you to be able to count the dollars in a stack of 100 dollars and arrive at 99 dollars. But it's been known for a long time that if one can prove any contradiction, then one can also prove that a stack of 100 dollars has no more than 99 dollars in it. The way he resolves this is interesting.
The Peano axioms for the natural numbers are these:
1. Zero is a number
2. The successor of any number is a number
3. Zero is not the successor of any number
4. Two different numbers have two different successors
5. If a given property holds for zero, and if it holds for the succesor of x whenever it holds for x, then it holds for all numbers.
Nelson rejects the fifth axiom, induction. It's the most complicated of the axioms, but it has another thing going against it: it is the only one that seems like a claim that could be either true or false. The first four axioms read like someone explaining the rules of a game, like how the pieces in chess move. Induction is more similar to the fact that the bishop in chess can only move on half the squares -- this is a theorem about chess, not one of the rules. Nelson believes that the fifth axiom needs to be, and cannot be, supported.
A common way to support induction is via the monologue: "It's true for zero. Since it's true for zero it's true for one. Since it's true for one it's true for two. Continuing like this we can show that it's true for one hundred and for one hundred thousand and for every natural number." It's hard to imagine actually going through this proof for very large numbers -- this is Nelson's objection.
What is arithmetic like if we reject induction? First, we may make a distinction between numbers we can actually count to (call them counting numbers) and numbers that we can't. Formally we define counting numbers as follows: 0 is a counting number, and if x is a counting number then so is its successor. We could use the induction axiom to establish that every number is a counting number, but without it we cannot.
A small example of a number so large we might not be able to count that high is the sum of two counting numbers. In fact without induction we cannot establish that x+y is a counting number from the facts that x and y are counting numbers. So we cut out a smaller class of numbers called additionable numbers: x is additionable if x + y is a counting number whenever y is a counting number. We can prove theorems about additionable numbers: for instance every additionable number is a counting number, the successor of an additionable number is additionable, and in fact the sum of two additionable numbers is an additionable number.
If we grant the induction axiom, these theorems lose their interest: every number is a counting number and an additionable number. Paraphrasing Nelson: the significance of these theorems is that addition is unproblematic even if we are skeptical of induction.
We can go further. It cannot be proved that the product of two additionable numbers is additionable. We therefore introduce the smaller class of multiplicable numbers. If whenever y is an additionable number x.y is also additionable, then we say that x is a multiplicable number. It can be proved that the sum and product of any two multiplicable numbers is multiplicable. Nelson closes the article I linked to:
I've omitted his final sentence.