It's not the lack of a proof that makes it informal, it's that the elements themselves of the theory aren't precisely, formally, mathematically defined. A valid proposition in measure-theoretic probability is a subset of the measure space. nothing else will do. Propositions in Bayseian probability are written in natural language, about events in the real world.

I'm using the word "formal" in the sense that it is used in mathematics. If you're going to say that propositions written in natural language, about events in the real world are "formal" in that sense, then you're just refusing to communicate.