My intuition tells me that if we manage to entirely formalize correct reasoning, the result will have a sort of adversarial quality: you can "prove" statements, but these proofs can be overturned by stronger disproofs.

Game semantics works somewhat like this; a proof is formalized as an "argument" between a Proponent and an Opponent. If an extension of game semantics to probabilistic reasoning exists, it will work much like the 'theory of uncertain arguments' you mention here.