The Wikipedia article and this shorter account both say that some form of Gödel's incompleteness theorem applies to second-order logic. I asked about the limits of the first-order approach to the reals because it looks like we'd need to use that if we want to stop the theorem from applying.

That approach still seems odd, but I can sort of see how you could do probability that way. I'll edit the OP to reflect my real question as soon as I feel up to it.