Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

FAWS comments on The Amanda Knox Test: How an Hour on the Internet Beats a Year in the Courtroom - Less Wrong

42 Post author: komponisto 13 December 2009 04:16AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (632)

You are viewing a single comment's thread. Show more comments above.

Comment author: FAWS 22 February 2010 03:11:44AM *  2 points [-]

One of the lessons of this exercise, that may be worth stating explicitly, is that there's no "outside referee" you can look to to make sure your beliefs are correct. In real life, you have to make judgments under uncertainty, using whatever evidence you have.

I meant that questions that should have an easily determinable answer, like "Did someone clean the blood outside her room up before the police was called" were unreasonably difficult to settle. Every site was mixing arguments and conclusions with facts. Sure, it's possible to find the answers if you look long enough, but it's much more work than it should be, and more work than I was willing to invest for a qestion that didn't interest me all that much in the first place.

Here's the real lesson: Bayesian calculations are not some mysterious black-magic technique that you "apply" to a problem. They are supposed to represent the calculations your brain has already made. Probability theory is the mathematics of inference. If you have an opinion on this case, then, ipso facto, your brain has already performed a Bayesian update.

The brain doesn't operate with very small or very big numbers, though. And I doubt it operates with conditional probabilities of the sort used above, as far as it operates Bayesian at all I would guess it's more similar to using venn diagrams.

The mistake you made was not making up numbers; it was making up numbers that, as you point out in the end, didn't reflect your actual beliefs.

The point is that I didn't spot that until after I did the calculation, and while I don't usually do much in the way of statistics I intuitively got the simple Bayesian problems like the usual example with false positives in a medical test right before hearing about Bayes theorem for the first time, so I don't think it's because I'm particularly bad at this. If you need to tweak afterwards anyway doing the Bayesian update explicitly isn't very useful as self-control.