I googled "Inference to the Best Explanation", and this paper of Gilbert Harman appears to be where the phrase was first coined, although the general idea goes back further. More recently, there's a whole book on the subject, and lukeprog's web site has an introductory article by the author of that book.
There isn't a single piece of mathematics in either of the two papers, which leads me to expect little of them. The book (of which I've only seen a few pages on Amazon) does contain a chapter on Bayesian reasoning, arguing that it and IBE are "broadly compatible". This appears to come down to the usual small-world/large-world issue: Bayes is sound mathematics (say the small-worlders) when you already have a hypothesis that gives you an explicit prior, but it must yield to something else when it comes to finding and judging hypotheses.
That something else always seems to come down to magic. It may be called IBE, or model validation, or human judgement, but however many words are expended, no method of doing it is found. It's the élan vital of statistics.
ETA: I found the book in my university library, but only the first edition of 1991, which is two chapters shorter and doesn't include the Bayes chapter (or any other mathematics). In the introduction (which is readable on Amazon) he remarks that IBE has been "more a slogan than an articulated philosophical theory", and that by describing inference in terms of explanation, it explains the obscure by the obscure. From a brief scan I was not sufficiently convinced that he fixes these problems to check the book out.
Thanks for the comment. The lack of math is a problem, and I think you've said it nicely:
That something else always seems to come down to magic. It may be called IBE, or model validation, or human judgement, but however many words are expended, no method of doing it is found. It's the élan vital of statistics.
Reading this book, Agnostic Inquirer, is quite the headache. It's so obscure and filled with mights, maybes, possibly's, and such that I constantly have this gut feeling that I'm being led into a mental trap but am not always sure which clauses ar...
I'm about 2/3 through an apologetics book that was recommended to me, Menssen and Sullivan's, The Agnostic Inquirer, and was quite surprised to run into a discussion of Bayes theorem and wanted some input from the LW community. The book is quite philosophical and I admit that I am probably not following all of it. I find heady philosophy to be one of these areas where something doesn't seem quite right (as in the conclusion that someone pushes), but I can't always identify what.
In any case, the primary point of the book is to attempt to replace the traditional apologetics method with a new one. The status quo has been to appeal to "natural theology," non-theological areas of discussion which attempt to bring one to the conclusion that some kind of theistic being exists, and from there establish that Christianity is the true formulation of what, exactly, this theistic being is/wants/does, etc by examining revealed theistic truths (aka the Bible). Menssen and Sullivan attempt to suggest that revelation need not be put off so long.
I don't want to get too into it, but think this helps set the stage. Their argument is as follows:
Issues Menssen and Sullivan have with Bayes applicability to this arena:
Then they begin trying to choose the best method for evaluating revelatory content. This is where Bayes comes in. The pages are almost all available via Google books HERE in Section 4.2.1, beginning on page 173. They suggest the following limitations: