= 27d567bc2d48d9f40e9ccc20ea370b15)
Alejandro1 comments on Open Thread for February 18-24 2014 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (454)
Some Bayesian thoughts on the classic mystery genre, prompted by watching on Netflix episodes of the Poirot series with David Suchet (which is really excellent by the way).
A common pattern in classic mystery stories is that there is a an obvious suspect, who had clear motive, means and opportunity for the crime (perhaps there is also some physical evidence against him/her). However, there is one piece of evidence that is unexplainable if the obvious person did it: a little clue unaccounted for, or perhaps a seemingly inconsequential lie or inconsistency in a witness' testimony. The Great Detective insists that no detail should be ignored, that the true explanation should account for all the clues. He eventually finds the true solution, which perfectly explains all the evidence, and usually involves a complicated plot by someone else committing the crime in such a way to get an airtight alibi, or to frame the first suspect, or both.
In Bayesian terms, the obvious solution has high prior probability P(H), and high P(E|H) for all components of E except for one or two apparently minor ones. The true solution, by contrast, has very high probability P(E|H) for all components of E. It is also claimed by the detective to have high prior P(H) (the guilty party tends to be someone with an excellent motive, they just had been dismissed as a suspect because of a seemingly perfect alibi). However, there is here a required suspension of disbelief, in that in real life there is a very low prior probability of someone plotting a crime (and successfully carrying it out) with a convoluted, complicated plot in order to get an alibi. In real life, the detective's solution would be dismissed because of a low P(H), and the detective's insistence on finding a solution that maximizes P(E|H) at the cost of P(H) would be flawed from the point of view of Bayesian rationality.
The question then becomes how this trope should properly be averted in rationalist fiction. (Besides the HPMOR approach.)
What is the HPMOR approach? Having so many characters be super-intelligent plotters, so that the prior probability that the explanation for something involves a complicated plot is much higher than in real life?
“The concept of 'evidence' had something of a different meaning, when you were dealing with someone who had declared themselves to play the game at 'one level higher than you'.”
--Chapter 86
ETA I was referring to my pet suspect for the hidden hand (who's clever enough to play Dumbledore and Quirrell against each other and leaves scant evidence of her existence). But I think Eliezer was referring to Defense Professor having a high prior and being Voldemort:
“The Potions Master said dryly, "The Defense Professor is always a suspect, Mr. Potter. You will notice a trend, given time.”
--Chapter 79
My guess is one possible aversion is 'have the protagonist be mind-altered so the obvious clues don't add up', which is one of the leading theories for 'why hasn't Harry figured out yet that Quirrel is Voldemort even though people were figuring that out by like ch20?'
Well, part of it is that Quirrel is Voldemort in canon, which is significant evidence that Harry doesn't have.
There's tons of evidence in-universe; even if he shouldn't suspect by chapter 20, then by ch100 the failure has become impossible. (And I recall that knowing about canon was actually a problem for a lot of people: "surely it can't be that obvious? Eliezer would never take such an obvious tack!")
I personally feel unsure what the Defense Professor wants, if he's the Dark Lord Tom. My tentative guess is, nsgre gur Qrzragbe snvyrq gb shysvyy gur cebcurpl, ur qrpvqrq gung nal fbyhgvba rkprcg qverpgyl 'pbeehcgvat' Uneel gb funer uvf tbnyf jbhyq whfg oevat n arj fcryy vagb rkvfgrapr. Guvf zvtug rkcynva jul ur qbrfa'g whfg qebc n ebpx ba Uneel be fgnaq nfvqr sbe gur pragnhe, bapr ur fgnegf gb ernyyl srne arj fcryyf.
Guvf ulcbgurfvf vapernfrf gur punapr gung Gbz unf tvira Uneel'f arj cneragf n gvzr-qrynlrq vyyarff (gung bayl ur pna erzbir) be bgurejvfr chg gurz va Obk N nf n onpxhc. Fb V'q orggre or evtug nobhg gur Qrnguyl Unyybjf nf jryy.
Reality doesn't work that way. In reality most solutions don't explain every clues that you find. A lot of clues are just random noise.
Yes, that was part of my point too. Maximizing P(E|H) at the expense of P(H) is much less likely (in real life) to give you a true solution than maximizing P(H|E) with Bayes, even if some components of P(E|H) are not particularly high (which is normal in real life).
They almost always end with the actual murderer being accused and then immediately getting angry and confessing thereby giving them the only actual hard evidence that could ever be used in a conviction. It's convenient that way. (See, for an extreme example, one of the episodes of the recent season 3 of Sherlock.)
Of course these stories also make crime look more difficult than it is, so they serve a useful purpose that probably outweighs the misrepresentation of real detective work.
I'm not sure. The criminal is always caught, but only because of a genius Great Detective; the regular police are usually portrayed as incompetent, at least in the true classics of the genre (Conan Doyle, early Christie). Modern shows like CSI might be a case where your statement applies better.