However, there is one important difference between the software world and the statistical modelling world. While it is sometimes possible to produce a "bug-free" piece of software, it is never possible to formulate a statistical model that captures reality exactly; as Box said, "all models are wrong." The challenge in statistical modelling is to find a model which is the best trade off between convenience (conceptual, mathematical, or computational) and verisimilitude. "Model checking" of some form or another is essential to this process; but it doesn't necessarily have to be standardized in a form analogous to a unit test. An alternative end towards the same means is an increased emphasis on model selection for different models of the same data, which can be put into a formalized statistical framework, although this is difficult to do in practice and hence is not very commonly done at present.
While I think your comment is generally true, I feel that it's almost a disservice to emphasize this point. A huge number of problems in the statistical sciences could be overcome by just a tiny bit of uniformity among model checking procedures. If it was seen as "bad form" to submit a journal article without doing some model expansion checks, or without providing test statistic analysis that goes beyond classical p-values, then the quality of publications would jump up. Even uniformity of the classical p-value testing would be helpful. I don't r...
Andrew Gelman recently linked a new article entitled "Induction and Deduction in Bayesian Data Analysis." At his blog, he also described some of the comments made by reviewers and his rebuttle/discussion to those comments. It is interesting that he departs significantly from the common induction-based view of Bayesian approaches. As a practitioner myself, I am happiest about the discussion on model checking -- something one can definitely do in the Bayesian framework but which almost no one does. Model checking is to Bayesian data analysis as unit testing is to software engineering.
Added 03/11/12
Gelman has a new blog post today discussing another reaction to his paper and giving some additional details. Notably: