Quote from page:
And in any reasonably large problem I will at some point discard a model and replace it with something new.
It's worth noting that a rigorous Bayesian approach does not license such a model-switch. The strict Bayesian starts with a prior, observes some evidence, and concludes with a new set of probabilities. By using this strategy Gelman is implicitly employing a vague, undefinable meta-model that exists only in his own brain. This isn't terrible, I suppose, if he gets good results, but it does mean that statistics is still as much an art as a science.
Andrew Gelman recently linked a new article entitled "Induction and Deduction in Bayesian Data Analysis." At his blog, he also described some of the comments made by reviewers and his rebuttle/discussion to those comments. It is interesting that he departs significantly from the common induction-based view of Bayesian approaches. As a practitioner myself, I am happiest about the discussion on model checking -- something one can definitely do in the Bayesian framework but which almost no one does. Model checking is to Bayesian data analysis as unit testing is to software engineering.
Added 03/11/12
Gelman has a new blog post today discussing another reaction to his paper and giving some additional details. Notably: