If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should be posted in Discussion, and not Main.
4. Open Threads should start on Monday, and end on Sunday.
Sure, that's exactly what we have to do, on pain of inconsistency. We have to disallow representation schemas powerful enough to internalise the Berry paradox, so that "the smallest number not definable in less than 11 words" is not a valid representation. Cf. the various set theories, where we disallow comprehension schemas strong enough to internalise Russell's paradox, so that "the set of all sets that don't contain themselves" is not a valid comprehension.
Nelson thought that, similarly to how we reject "the smallest number not effectively representable" as an invalid representation, we should also reject e.g. "3^^^3" as an invalid representation; not because of the Berry paradox, but because of a different one, one that he ultimately could not establish.
Nelson introduced a family of standardness predicates, each one relative to a hyper-operation notation (addition, multiplication, exponentiation, the ^^-up-arrow operation, the ^^^-up-arrow notation and so on). Since standardness is not a notion internal to arithmetic, induction is not allowed on these predicates (i. e. '0' is standard, and if 'x' is standard then so is 'x+1', but you cannot use induction to conclude that therefore everything is standard).
He was able to prove that the standardness of n and m implies the standardness of n+m, and that of n×m. However, the corresponding result for exponentiation is provably false and the obstruction is non-associativity. What's more, even if we can prove that 2^^d is standard, this does not mean that the same holds for 2^^(d+1).
At this point, Nelson attempted to prove that an explicit recursion of super-exponential length does not terminate, thereby establishing that arithmetic is inconsistent, and vindicating ultrafinitism as the only remaining option. His attempted proof was faulty, with no obvious fix. Nelson continued looking for a valid proof until his death last September.