I think we can make this more precise. Second-order arithmetic theoretically covers what I believe about the natural numbers. We then need a theory of sets or collections to interpret it. Gaifman, in his discussion of non-standard models, suggests that we go meta instead, and say that whenever we learn a new formula we expand the principle of induction to include this formula. In both cases, the open-ended part comes less from arithmetic than from some larger framework.

So: I don't think my larger 'theory' is necessarily self-consistent. I do think there exists some consistent intermediate theory which includes the consistency of second-order PA, and the uniqueness of its model up to isomorphism.

If we define phi+1 as the union of the statement(s) phi and the assertion of phi's consistency, someone might possibly doubt the consistency of PA+omega, since the concept "omega" need not appear in first-order PA. I find myself hesitating slightly about PA+the Church-Kleene ordinal, although ultimately I think many large cardinals are consistent with(in) set theory. And yes, I think at this point we should start assigning probabilities to mathematical statements.