I just went to Wikipedia and found a more articulate version of what I'm trying to say:
Gardner-Medwin argues that the criterion on which a verdict in a criminal trial should be based is not the probability of guilt, but rather the probability of the evidence, given that the defendant is innocent (akin to a frequentist p-value). He argues that if the posterior probability of guilt is to be computed by Bayes' theorem, the prior probability of guilt must be known. This will depend on the incidence of the crime, which is an unusual piece of evidence to consider in a criminal trial. Consider the following three propositions:
A: The known facts and testimony could have arisen if the defendant is guilty,
B: The known facts and testimony could have arisen if the defendant is innocent,
C: The defendant is guilty.
Gardner-Medwin argues that the jury should believe both A and not-B in order to convict. A and not-B implies the truth of C, but the reverse is not true. It is possible that B and C are both true, but in this case he argues that a jury should acquit, even though they know that they will be letting some guilty people go free. See also Lindley's paradox.
I am not really a stats person and I'm not prepared to defend Garder-Medwin's model as being correct -- but right or wrong, it's a better description than Bayesian inference of most people's intuitive concept of the task of a juror.
In other words, when I imagine myself as a juror I'm automatically more concerned about a false positive (convicting an innocent person), and I will intuitively try to answer the question "has the prosecution proved its case" rather than "is this person guilty."
If asked to answer the second question and quantify my odds of guilt, I'm likely to understate them, precisely because I can't separate that estimate from the real-world effect of a guilty verdict.
Or in your terms, the "question of what probability corresponds to 'beyond reasonable doubt' [or whatever the equivalent standard in Italy]" can't be completely excluded from the question when we imagine ourselves as jurors, only made implicit.
This reminds me slightly of Eliezer's "true Prisoner's Dilemma" article, which I really liked. Just as you can't posit that someone is my confederate (in his case) and then ask me to consider them in a purely selfish, impartial way -- you can't tell me I'm a juror and then ask me to make a purely impartial assessment. I'm describing a much weaker effect than he was, and maybe it's more socially conditioned than inherent to human nature, but I think the general concept is the same.
So ...better to say "forget the fact that there's even a trial going on, just imagine that tomorrow the absolute truth will be revealed and you have to bet on it now."
He argues that if the posterior probability of guilt is to be computed by Bayes' theorem, the prior probability of guilt must be known. This will depend on the incidence of the crime, which is an unusual piece of evidence to consider in a criminal trial.
This is an interesting point and one where I think the legal system is wrong from a strict rationality sense but I can see the argument given that juries are human and so not very rational.
It is common for juries to either not be given information which is very relevant to the prior probabilities of gui...
As many of you probably know, in an Italian court early last weekend, two young students, Amanda Knox and Raffaele Sollecito, were convicted of killing another young student, Meredith Kercher, in a horrific way in November of 2007. (A third person, Rudy Guede, was convicted earlier.)
If you aren't familiar with the case, don't go reading about it just yet. Hang on for just a moment.
If you are familiar, that's fine too. This post is addressed to readers of all levels of acquaintance with the story.
What everyone should know right away is that the verdict has been extremely controversial. Strong feelings have emerged, even involving national tensions (Knox is American, Sollecito Italian, and Kercher British, and the crime and trial took place in Italy). The circumstances of the crime involve sex. In short, the potential for serious rationality failures in coming to an opinion on a case like this is enormous.
Now, as it happens, I myself have an opinion. A rather strong one, in fact. Strong enough that I caught myself thinking that this case -- given all the controversy surrounding it -- might serve as a decent litmus test in judging the rationality skills of other people. Like religion, or evolution -- except less clichéd (and cached) and more down-and-dirty.
Of course, thoughts like that can be dangerous, as I quickly recognized. The danger of in-group affective spirals looms large. So before writing up that Less Wrong post adding my-opinion-on-the-guilt-or-innocence-of-Amanda-Knox-and-Raffaele-Sollecito to the List of Things Every Rational Person Must Believe, I decided it might be useful to find out what conclusion(s) other aspiring rationalists would (or have) come to (without knowing my opinion).
So that's what this post is: a survey/experiment, with fairly specific yet flexible instructions (which differ slightly depending on how much you know about the case already).
For those whose familiarity with the case is low:
I'm going to give you two websites advocating a position, one strongly in favor of the verdict, the other strongly opposed. Your job will be to browse around these sites to learn info about the case, as much as you need to in order to arrive at a judgment. The order, manner, and quantity of browsing will be left up to you -- though I would of course like to know how much you read in your response.
1. Site arguing defendants are guilty.
2. Site arguing defendants are innocent.
I've chosen these particular sites because they seemed to contain the best combination of fierceness of advocacy and quantity of information on their respective sides that I could find.
If you find better summaries, or think that these choices reflect a bias or betray my own opinion, by all means let me know. I'm specifically avoiding referring you to media reports, however, for a couple of reasons. First, I've noticed that reports often contain factual inaccuracies (necessarily, because they contradict each other). Secondly, journalists don't usually have much of a stake, and I'd like to see how folks respond to passionate advocacy by people who care about the outcome, as in an actual trial, rather than attempts at neutral summarizing. Of course, it's fine if you want to read media reports linked to by the above sites.
(One potential problem is that the first site is organized like a blog or forum, and thus it is hard to find a quick summary of the case there. [EDIT: Be sure to look at the category links on the right side of the page to find the arguments.] If you think it necessary, refer to the ever-changing Wikipedia article, which at the moment of writing seems a bit more favorable to the prosecution. [EDIT: I'm no longer sure that's true.] [EDIT: Now I think it's true again, the article having apparently changed some more. So there's really no telling. Be warned.])
After you do this reading, I'd like to know:
1. Your probability estimate that Amanda Knox is guilty.
2. Your probability estimate that Raffaele Sollecito is guilty.
3. Your probability estimate that Rudy Guede is guilty.
4. How much you think your opinion will turn out to coincide with mine.
Feel free to elaborate on your reasoning to whatever degree you like.
One request: don't look at others' comments until you've done the experiment yourself!
For those whose familiarity with the case is moderate or high:
I'd like to know, as of right now:
1. Your probability estimate that Amanda Knox is guilty.
2. Your probability estimate that Raffaele Sollecito is guilty.
3. Your probability estimate that Rudy Guede is guilty.
4. How much you think your opinion will turn out to coincide with mine.
5. From what sources you've gotten the info you've used to arrive at these estimates.
Then, if possible, do the experiment described above for those with little familiarity, and report any shifts in your estimates.
Again, everyone should avoid looking at others' responses before giving their own feedback. Also, don't forget to identify your prior level of familiarity!
If the level of participation warrants it, I'll post my own thoughts (and reaction to the feedback here) in a later post. (Edit: That post can be found here.)