Continuing my interest in tracking real-world predictions, I notice that the recent acquittal of Knox & Sollecito offers an interesting opportunity - specifically, many LessWrongers gave probabilities for guilt back in 2009 in komponisto’s 2 articles:
- “You Be the Jury: Survey on a Current Event”
- “The Amanda Knox Test: How an Hour on the Internet Beats a Year in the Courtroom”
Both were interesting exercises, and it’s time to do a followup. Specifically, there are at least 3 new pieces of evidence to consider:
- the failure of any damning or especially relevant evidence to surface in the ~2 years since (see also: the hope function)
- the independent experts’ report on the DNA evidence
- the freeing of Knox & Sollecito, and continued imprisonment of Rudy Guede (with reduced sentence)
Point 2 particularly struck me (the press attributes much of the acquittal to the expert report, an acquittal I had not expected to succeed), but other people may find the other 2 points or unmentioned news more weighty.
2 Probabilities
I was curious how the consensus has changed, and so, in some spare time, I summoned all the Conscientiousness I could and compiled the following list of 54 entries based on those 2 articles’ comments (sometimes inferring specific probabilities and possibly missing probabilities given in hidden subthreads), where people listed probabilities for Knox’s guilt, Sollecito’s guilt, and Guede’s guilt:
Knox | Sollecito | Guede | LWer |
---|---|---|---|
.20 | .20 | .70 | badger |
.05 | .10 | .90 | mattnewport |
.20 | .25 | .90 | AngryParsley |
.05 | .05 | .95 | tut |
.05 | .05 | .95 | bentarm |
.85 | .60 | .20 | bgrah449 |
.01 | .01 | .99 | kodos96 |
.01 | .01 | .99 | Daniel_Burfoot |
.40 | .40 | .90 | nerzhin |
.45 | .45 | .60 | Matt_Simpson |
.33 | .33 | .90 | Cyan |
.50 | .50 | .95 | jimmy |
.05 | .05 | .99 | Psychohistorian |
.40 | .40 | .90 | Threads |
.50 | .50 | .80 | Morendil |
.15 | — | — | Eliezer_Yudkowsky |
.20 | .35 | .98 | LauraABJ |
.10 | .10 | .90 | curious |
.20 | .20 | .96 | jpet |
.06 | .06 | .70 | saliency |
.80 | .60 | .95 | Mario |
.20 | .20 | .95 | Yvain |
.70 | — | — | Shalmanese |
.05 | .05 | .95 | gelisam |
.05 | .05 | .90 | Mononofu |
.90 | .90 | .90 | lordweiner27 (changed mind) |
.50 | .50 | .99 | GreenRoot |
.99 | .99 | .99 | dilaudid |
.13 | .15 | .97 | Jack |
.05 | .05 | .90 | wedrifid |
.01 | .01 | .90 | Nanani |
.35 | .35 | .95 | imaxwell |
.01 | .01 | .99 | jenmarie |
.25 | .25 | .75 | Jawaka |
.41 | .38 | .99 | magfrump |
.40 | .20 | .60 | gwern |
.08 | .10 | .95 | loqi |
.25 | .25 | .50 | JamesAndrix |
.90 | .85 | .99 | Unknowns |
.35 | .35 | .90 | Sebastian_Hagen |
.90 | .90 | .99 | brazil84 |
.30 | .30 | .40 | ChrisHibbert |
.02 | .02 | .98 | wnoise |
.50 | .40 | .90 | John_Maxwell_IV |
.10 | .10 | — | k3nt |
.01 | .01 | .99 | Sinai |
.00 | .00 | 1.0 | KayPea |
.00 | .00 | .60 | MerleRideout |
.15 | .10 | .80 | TheRev |
.01 | .01 | .99 | komponisto |
.30 | — | — | pete22 |
.01 | — | — | SforSingularity |
.00 | .00 | .90 | AnnaGilmour |
.05 | .05 | .95 | Seth_Goldin |
.60 | .60 | .95 | bigjeff5 |
It’s interesting how many people assign a high-probability to Knox being guilty; I had remembered LW as being a hive of Amanda fans, but either I’m succumbing to hindsight bias or people updated significantly after those articles. (For example, Eliezer says .15 is too high, but doesn’t seem otherwise especially convinced; and later one reads in Methods of Rationality that "[Hagrid] is the most blatantly innocent bystander to be convicted by the magical British legal system since Grindelwald's Confunding of Neville Chamberlain was pinned on Amanda Knox.")
EDIT: Jack graphed the probability against karma:
2.1 Outliers
If we look just at >41% (chosen to keep contacts manageable), we find 12 entries out of 54:
Knox | Sollecito | Guede | LWer |
---|---|---|---|
.45 | .45 | .60 | Matt_Simpson |
.50 | .40 | .90 | John_Maxwell_IV |
.50 | .50 | .80 | Morendil |
.50 | .50 | .95 | jimmy |
.50 | .50 | .99 | GreenRoot |
.60 | .60 | .95 | bigjeff5 |
.70 | — | — | Shalmanese |
.80 | .60 | .95 | Mario |
.85 | .60 | .20 | bgrah449 |
.90 | .85 | .99 | Unknowns |
.90 | .90 | .90 | lordweiner27 |
.90 | .90 | .99 | brazil84 |
.99 | .99 | .99 | dilaudid |
I have messaged each of them, asking them to comment here, describing if and how they have since updated, and any other thoughts they might have. (I have also messaged the first 12 commenters or so, chronologically, with <41% confidence in Knox’s guilt, with the same message.) The commenters:
AngryParsley / Cyan / Daniel_Burfoot / Eliezer_Yudkowsky / GreenRoot / John_Maxwell_IV / LauraABJ / Mario / Matt_Simpson / Morendil / Psychohistorian / Shalmanese / Threads / Unknowns / badger / bentarm / bgrah449 / bigjeff5 / brazil84 / dilaudid / jimmy / kodos96 / lordweiner27 / mattnewport / nerzhin / tut
I look forward to seeing their retrospectives, or indeed, anyone's retrospectives on the matter.
- Allknowing and most merciful Bayes;
- We have erred, and strayed from thy ways like biased sheep.
- We have followed too much the devices and desires of our own hearts.
- We have offended against thy axiomatic laws.
- We have left undone those updates which we ought to have done;
- And we have done those updates which we ought not to have done;
- And there is no calibration in us.
- But thou, O Bayes, have mercy upon us, miserable wannabes.
- Spare thou them, O Bayes, who confess their faults.
Can you give a numerical probability? You claimed that your assessment had shifted somewhat in light of the information that Knox and Sollecito had been found not guilty on appeal.
I'm going to do a run down of my probability estimates for Knox, Sollecito, and Guede, given the evidence available to me excluding the argument I've already given for the unlikelihood of Knox and Sollecito's presence at the scene given the state of the forensic evidence. Could you explain what, if any of it, you disagree with, and how your own analysis would differ?
Prior probabilities. Common wisdom is that most people are killed by someone they know. I'm going to go with 80% likelihood of murder by an acquaintance, although I can't actually find a single figure all sources agree on. So we'll weight the prior probability of guilt such that people Kercher knows account for 80%, and everyone else accounts for 20%. Unfortunately, I can't find a good source estimating how many people most people know, and it depends how close a standard of acquaintance you're measuring. I'm going to give a ballpark estimate of about 1000, and thus apportion an average of .00125 prior likelihood of guilt between them.
For Knox, let's adjust up for closeness of acquaintance (we'll call it a factor of 10:1) for a likelihood of .0125. She had no prior arrest record; I'll adjust down by 70 percent to .00875, and she was a woman who was neither a close family member nor an intimate partner, so I'll adjust it down by another 90%, to .000875. I don't feel that the prosecution established any good reason to suspect a motive, but I'll be charitable to their case and not revise further down for that.
I'll call Sollecito's probability .0007, adjusting upwards for his increased likelihood as a male, but downwards based on his much lesser closeness to the victim; I have to take probability mass away from slight acquaintances if I'm adding it to close ones.
Guede was male, only slightly acquainted with Kercher, had been involved in breaking and entering prior to the case, and also did not have a clear motive. People with prior criminal records account for a majority of violent crimes despite being a minority of the population. The population without a criminal record is giving up a majority of their probability mass to the minority with, so this is a pretty significant boost. I don't know the proportion of Italy's population that has a record, but if we assume similar rates to the U.S. this will increase Guede's likelihood by about 11 times. So let's call his prior probability about .0077.
There's a considerable amount of fudge factor in these numbers, and you are of course free to take issue with them, but I think they give us a reasonable ballpark order to start with.
First, we'll deal with Guede. His handprint in blood places him conclusively at the scene of the crime (I doubt that anyone has ever left a handprint in blood at a scene at which they were not present) and he does not deny having been at the scene, so we'll call the odds of his presence infinitesimally below 1. He changed his story repeatedly over the course of his interrogation, beginning with "another anonymous black man did it while I was in the bathroom after having sex with her." The prior likelihood of this is very low, and Guede left DNA traces he was unlikely to have left if he was not involved in killing her. I'd adjust Guede up from the prior by an order of about 10^7, which would leave a likelihood of guilt that approximates to 1-(10^-5).
Let's move on to Amanda Knox. Giobbi first flagged her as a suspect based on the belief that her mannerisms at the crime scene were odd. I expect that a person is significantly more likely to behave oddly at a crime scene (relative to the norm for an uninvolved person) if they were involved than if they were uninvolved. I'd put an odds ratio of maybe 15 to 1 on it, except that Knox was already known to have odd mannerisms (such as doing yoga stretches in places considered inappropriate,) so I'd revise that down to maybe 5 to 1. So that gives us a probability of involvement for Knox of about .0044. All the evidence brought forward by the police after this point is filtered evidence, because they focused their investigations on Knox and Sollecito, and put more effort into finding reasons to suspect them than anyone else who might have been involved.
In the United States in 1976-1996, 16% of murders were committed by more than one person; since I don't have any better figures I'll assume similar rates for present day Italy. Ignoring the failure of forensics to turn up evidence of anyone else, I find the arguments posed by the prosecution for believing that the murder had multiple perpetrators extremely uncompelling. Arguments such as the number of stab wounds and the fact that Kercher had wounds on both sides of her neck are extremely weak evidence; particularly taken alongside claims such as one which they offered for the break-in being staged, that the pattern of the broken glass was inconsistent with a break from outside, which was tested and found to be false, these arguments sound to me like scrambling for anything that sounds like evidence. Ignoring the failure of anyone else to leave biological traces, I'd be willing to extend perhaps a 3 to 1 odds ratio on their evidence for multiple attackers, so together with the prior unlikelihood of a murder having multiple perpetrators, that brings Knox's likelihood of guilt to about .0022.
Knox and Sollecito failed to implicate Guede, and Guede failed to implicate Knox and Sollecito; for their mutual failure to implicate each other, I'd reduce Knox and Sollecito's likelihood about 20 times, bringing Knox to about .00011.
I do not find any evidence presented by the prosecution compelling with regards to establishing a motive or likelihood of presence at the crime for Knox or Sollecito, their case seems to me to be a collection of behaviors and actions attributed to Knox with the implication "this seems more unusual for a person who was not involved than a person who was." Since this evidence is filtered, I only find it reasonable to update on it to the extent that I believe the body of evidence is stronger than what I should expect for anyone with my current assignment of guilt subject to a similar police investigation. I think if you really looked and sincerely believed you had reason to find anything, you could find a substantial body of reasons to be suspicious of anyone. One of the most common ways that people err is in believing that a false proposition must have no valid evidence, and every argument for increasing your confidence in it must be refutable. I would have to assume that people I know are generally significantly weirder and more suspicious than most of the population to assign an odds ratio of even 10 to 1 on the prosecution's evidence, but my social circle probably is fairly odd, so I'll give them that, but I think I'm being generous.
All in all, that gives approximately .001 odds of guilt for Knox. Sollecito was tagged as a suspect mainly through association with her, so I would say that he is at most as likely as she is to have been involved.
Of course, there is plenty of uncertainty in these figures, and this does not portray the entirety of the relevant evidence even ignoring the improbability of other people involved in the murder not leaving biological traces (one particular piece of evidence I even left out deliberately, you can try and guess what,) but I think it reflects a reasonable probability estimate given that omission for an amount of evidence that a person can reasonably be expected to process without a substantial time commitment.
Perhaps about 30%.
I don't have the energy to do a point by point response, but I will point out what I think is the central flaw in your reasoning.
The flaw, it seems to me, is that you seem to dismiss the circumstantial evidence against Knox (and Sollecito) as "filtered evidence." There are two problems with this.
First, the evidence is not filtered in the sense that the only evidence we (or the jury) saw i... (read more)