gwern comments on Rationality Quotes October 2012 - Less Wrong

8 Post author: MBlume 02 October 2012 06:50PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (298)

You are viewing a single comment's thread.

Comment author: gwern 06 October 2012 12:49:25AM *  15 points [-]

Despite the difficulty of exact Bayesian inference in complex mathematical models, the essence of Bayesian reasoning is frequently used in everyday life. One example has been immortalized in the words of Sherlock Holmes to his friend Dr. Watson: “How often have I said to you that when you have eliminated the impossible, whatever remains, however improbable, must be the truth?” (Arthur Conan Doyle, The Sign of Four, 1890, Ch. 6). This reasoning is actually a consequence of Bayesian belief updating, as expressed in Equation 4.4. Let me re-state it this way: “How often have I said to you that when p(D|θ_i ) = 0 for all i!=j, then, no matter how small the prior p(θ_j ) > 0 is, the posterior p(θ_j |D) must equal one.” Somehow it sounds better the way Holmes said it.

--Kruschke 2010, Doing Bayesian Data Analysis, pg56-57

Comment author: thomblake 09 October 2012 08:22:02PM *  19 points [-]

When you have eliminated the impossible, whatever remains is often more improbable than your having made a mistake in one of your impossibility proofs.

-Steven Kaas (via)

Comment author: PhilGoetz 01 November 2012 09:07:50PM 5 points [-]

It always irritates me slightly that Holmes says "whatever remains, however improbable, must be the truth", when multiple incompatible hypotheses will remain.

My Holmes says, "When you have eliminated the possible, you must expand your conception of what is possible."

Comment author: pragmatist 06 October 2012 07:35:07AM *  2 points [-]

You have an inequality symbol missing at the end of the quote (between i and j). That made it slightly difficult for me to parse it on my first read-through ("Why does it say 'for all i, j' when the only index in the expression is 'i'?").

Comment author: WingedViper 09 October 2012 01:52:37PM 1 point [-]

I don't know if you know, but just in case you (or someone else) don't: There is no inequality symbol on the computer keyboard, so he used a typical programmer's inequality symbol which is "!=". So yes, it is not easily readable (i! is a bad combination...) but totally correct.

Comment author: pragmatist 09 October 2012 02:20:38PM *  3 points [-]

The symbol wasn't there when I wrote my comment. It was edited in afterwards.

Comment author: [deleted] 10 October 2012 12:53:56PM *  2 points [-]

(i! is a bad combination...)

The way to handle that is whitespace: i != 0. (I once was teased by my tendency to put whitespace in computer code around all operators which would be spaced in typeset mathematical formulas.)

EDIT: I also use italics for variables, boldface for vectors, etc. when handwriting. Whenever I get a new pen I immediately check whether it's practical to do boldface with it.

Comment author: Dan_Moore 09 October 2012 02:29:23PM 2 points [-]

A space between variable & operator would help.

Comment author: khafra 09 October 2012 07:13:02PM *  1 point [-]

Of course, an infitesimal prior dominating the posterior pdf might also be a hint that your model needs adjustment.