I would have thought that ZFC and PA were sets of axioms and would say nothing about how an agent reasons.

The other way around: the agent reasons using ZFC or PA. (And not just sets of axioms, but associated deductive system, so rules of what can be proved how.)

In first-order logic, all valid statements are also provable by a formal syntactic argument.

I simply mean completeness of first-order logic.