Eliezer,

I've found that Jaynes's infinite set atheism is a little too extreme. It forces you to take the slow route every time when you want to explore stochastic process priors like Gaussian process and Dirichlet process priors. I reserve infinite set atheism for observables -- no infinite sets of observations allowed.