Even if we have infinite evidence (positive or negative) for some set of events, we cannot achieve infinite evidence for any other event. The point of a logical system is that everything in it can be proven syntactically, that is, without assigning meaning to any of the terms. For example, "Only Bs have the property X" and "A has the property X" imply "A is a B" for any A, B and X - the proof makes no use of semantics. It is sound if it is valid and its axioms are true, but it is also only valid if we have defined certain operations as truth preserving. There are an uncountably infinite number of logical systems under which the truth of the axioms will not ensure the truth of the conclusion - the reasoning won't be valid.

Non-probabilistic reasoning does not ever work in reality. We do not know the syntax with certainty, so we cannot be sure of any conclusion, no matter how certain we are about the semantic truth of the premises. The situation is like trying to speak a language you don't know using only a dictionary and a phrasebook - no matter how certain you are that certain sentences are correct, you cannot be certain that any new sentence is gramatically correct because you have no way to work out the grammar with absolute certainty. No matter how many statements we take as axioms, we cannot add any more axioms unless we know the rules of syntax, and there is no way at all to prove that our rules of syntax - the rules of our logical sytem - are the real ones. (We can't even prove that there are real ones - we're pretty darned certain about it, but there is no way to prove that we live in a causal universe.)