"I am an infinite set atheist - have you ever actually seen an infinite set?"
I interpreted that to mean that Eliezer doubts that a model that requires infinite sets will correspond to reality, not that the mathematics are incorrect. The figurative use of the word "atheist" makes the statement ambiguous, but his use of the phrase "actually seen" indicates that his concern is with modeling reality, not the math per se.
That was my (charitable) interpretation too, until, to my dismay, Eliezer confirmed (at a meetup) that he had "leanings" in the direction of constructivism/intuitionism -- apparently not quite aware of the discredited status of such views in mathematics.
And indeed, when I asked Eliezer where he thinks the standard proof of infinite sets goes wrong, he pointed to the law of the excluded middle.
His idol E.T. Jaynes may be to blame, who in PTLS explicitly allied himself with Kronecker, Brouwer, and Poincaré as opposed to Cantor, Hilbert, and Bourbak...
[edit: sorry, the formatting of links and italics in this is all screwy. I've tried editing both the rich-text and the HTML and either way it looks ok while i'm editing it but the formatted terms either come out with no surrounding spaces or two surrounding spaces]
In the latest Rationality Quotes thread, CronoDAS quoted Paul Graham: