As for amount of needed atoms - they use unlimited number of atoms to prove the existence of Θ.
In which line of the proof do they first require that assumption for their argument? Are they assuming an unlimited number, or just a "sufficiently large" number? If they really require an unlimited number, then it is contrary to the entire spirit of the paper, because they start out in the Introduction as committed finitists.
Sufficiently large to be larger than inverse of our precision requirements.
I've recently been getting into all of this wonderful Information Theory stuff and have come across a paper (thanks to John Salvatier) that was written by Kevin H. Knuth:
Foundations of Inference
The paper sets up some intuitive minimal axioms for quantifying power sets and then (seems to) use them to derive Bayesian probability theory, information gain, and Shannon Entropy. The paper also claims to use less assumptions than both Cox and Kolmogorov when choosing axioms. This seems like a significant foundation/unification. I'd like to hear whether others agree and what parts of the paper you think are the significant contributions.
If a 14 page paper is too long for you, I recommend skipping to the conclusion (starting at the bottom of page 12) where there is a nice picture representation of the axioms and a quick summary of what they imply.