Continuing my interest in tracking real-world predictions, I notice that the recent acquittal of Knox & Sollecito offers an interesting opportunity - specifically, many LessWrongers gave probabilities for guilt back in 2009 in komponisto’s 2 articles:
- “You Be the Jury: Survey on a Current Event”
- “The Amanda Knox Test: How an Hour on the Internet Beats a Year in the Courtroom”
Both were interesting exercises, and it’s time to do a followup. Specifically, there are at least 3 new pieces of evidence to consider:
- the failure of any damning or especially relevant evidence to surface in the ~2 years since (see also: the hope function)
- the independent experts’ report on the DNA evidence
- the freeing of Knox & Sollecito, and continued imprisonment of Rudy Guede (with reduced sentence)
Point 2 particularly struck me (the press attributes much of the acquittal to the expert report, an acquittal I had not expected to succeed), but other people may find the other 2 points or unmentioned news more weighty.
2 Probabilities
I was curious how the consensus has changed, and so, in some spare time, I summoned all the Conscientiousness I could and compiled the following list of 54 entries based on those 2 articles’ comments (sometimes inferring specific probabilities and possibly missing probabilities given in hidden subthreads), where people listed probabilities for Knox’s guilt, Sollecito’s guilt, and Guede’s guilt:
Knox | Sollecito | Guede | LWer |
---|---|---|---|
.20 | .20 | .70 | badger |
.05 | .10 | .90 | mattnewport |
.20 | .25 | .90 | AngryParsley |
.05 | .05 | .95 | tut |
.05 | .05 | .95 | bentarm |
.85 | .60 | .20 | bgrah449 |
.01 | .01 | .99 | kodos96 |
.01 | .01 | .99 | Daniel_Burfoot |
.40 | .40 | .90 | nerzhin |
.45 | .45 | .60 | Matt_Simpson |
.33 | .33 | .90 | Cyan |
.50 | .50 | .95 | jimmy |
.05 | .05 | .99 | Psychohistorian |
.40 | .40 | .90 | Threads |
.50 | .50 | .80 | Morendil |
.15 | — | — | Eliezer_Yudkowsky |
.20 | .35 | .98 | LauraABJ |
.10 | .10 | .90 | curious |
.20 | .20 | .96 | jpet |
.06 | .06 | .70 | saliency |
.80 | .60 | .95 | Mario |
.20 | .20 | .95 | Yvain |
.70 | — | — | Shalmanese |
.05 | .05 | .95 | gelisam |
.05 | .05 | .90 | Mononofu |
.90 | .90 | .90 | lordweiner27 (changed mind) |
.50 | .50 | .99 | GreenRoot |
.99 | .99 | .99 | dilaudid |
.13 | .15 | .97 | Jack |
.05 | .05 | .90 | wedrifid |
.01 | .01 | .90 | Nanani |
.35 | .35 | .95 | imaxwell |
.01 | .01 | .99 | jenmarie |
.25 | .25 | .75 | Jawaka |
.41 | .38 | .99 | magfrump |
.40 | .20 | .60 | gwern |
.08 | .10 | .95 | loqi |
.25 | .25 | .50 | JamesAndrix |
.90 | .85 | .99 | Unknowns |
.35 | .35 | .90 | Sebastian_Hagen |
.90 | .90 | .99 | brazil84 |
.30 | .30 | .40 | ChrisHibbert |
.02 | .02 | .98 | wnoise |
.50 | .40 | .90 | John_Maxwell_IV |
.10 | .10 | — | k3nt |
.01 | .01 | .99 | Sinai |
.00 | .00 | 1.0 | KayPea |
.00 | .00 | .60 | MerleRideout |
.15 | .10 | .80 | TheRev |
.01 | .01 | .99 | komponisto |
.30 | — | — | pete22 |
.01 | — | — | SforSingularity |
.00 | .00 | .90 | AnnaGilmour |
.05 | .05 | .95 | Seth_Goldin |
.60 | .60 | .95 | bigjeff5 |
It’s interesting how many people assign a high-probability to Knox being guilty; I had remembered LW as being a hive of Amanda fans, but either I’m succumbing to hindsight bias or people updated significantly after those articles. (For example, Eliezer says .15 is too high, but doesn’t seem otherwise especially convinced; and later one reads in Methods of Rationality that "[Hagrid] is the most blatantly innocent bystander to be convicted by the magical British legal system since Grindelwald's Confunding of Neville Chamberlain was pinned on Amanda Knox.")
EDIT: Jack graphed the probability against karma:
2.1 Outliers
If we look just at >41% (chosen to keep contacts manageable), we find 12 entries out of 54:
Knox | Sollecito | Guede | LWer |
---|---|---|---|
.45 | .45 | .60 | Matt_Simpson |
.50 | .40 | .90 | John_Maxwell_IV |
.50 | .50 | .80 | Morendil |
.50 | .50 | .95 | jimmy |
.50 | .50 | .99 | GreenRoot |
.60 | .60 | .95 | bigjeff5 |
.70 | — | — | Shalmanese |
.80 | .60 | .95 | Mario |
.85 | .60 | .20 | bgrah449 |
.90 | .85 | .99 | Unknowns |
.90 | .90 | .90 | lordweiner27 |
.90 | .90 | .99 | brazil84 |
.99 | .99 | .99 | dilaudid |
I have messaged each of them, asking them to comment here, describing if and how they have since updated, and any other thoughts they might have. (I have also messaged the first 12 commenters or so, chronologically, with <41% confidence in Knox’s guilt, with the same message.) The commenters:
AngryParsley / Cyan / Daniel_Burfoot / Eliezer_Yudkowsky / GreenRoot / John_Maxwell_IV / LauraABJ / Mario / Matt_Simpson / Morendil / Psychohistorian / Shalmanese / Threads / Unknowns / badger / bentarm / bgrah449 / bigjeff5 / brazil84 / dilaudid / jimmy / kodos96 / lordweiner27 / mattnewport / nerzhin / tut
I look forward to seeing their retrospectives, or indeed, anyone's retrospectives on the matter.
- Allknowing and most merciful Bayes;
- We have erred, and strayed from thy ways like biased sheep.
- We have followed too much the devices and desires of our own hearts.
- We have offended against thy axiomatic laws.
- We have left undone those updates which we ought to have done;
- And we have done those updates which we ought not to have done;
- And there is no calibration in us.
- But thou, O Bayes, have mercy upon us, miserable wannabes.
- Spare thou them, O Bayes, who confess their faults.
Well, the judgment of people here is evidence just like anything else. Lets say I initially predicted Knox's guilt with p=0.01, Since I think my beliefs track the truth and the beliefs of other Less Wrong posters track the truth I should expect other posters to agree with my assessment if my belief is accurate. The majority of posters disagreeing with me is far more likely if I'm wrong than if I'm right. So upon learning that the vast majority of posters disagree with me I should be more uncertain about my prediction.
How uncertain I should be is a difficult question-- in many cases in that thread it was resolved by discussing evidence. Many people with initially high probabilities shifted their estimates downward after evidence they missed was pointed out to them. If you think you have evidence other Less Wrong posters don't have then it makes sense to not take their opinions seriously. Alternatively, if you think Less Wrong posters are irrational or poorly calibrated and don't expect their beliefs as a group to track the truth well then it makes sense to more or less ignore their opinion. I suppose one could also ignore the opinions of the Less Wrong posters on the ground that the opinions of random people reading about the case are swamped by the opinions of people who have studied the case for months-- and thus make very little difference. But now Knox and Sollecito have been released-- if your trust in the experts was what lead you to ignore Less Wrong you should update on the new court decision.
So why didn't you update on the opinions of Less Wrong posters?
I wonder. The opinions of members of a given community are not independent events. There's influence by high status members, and by perceived community consensus (note how in a previous post, brazil84 got downvoted just for admitting, when asked, that this consensus didn't move his own opinion much - I don't know, but to me that's ominous). So isn't there's a risk of counting the same evidence (the arguments and facts that convinced the "first movers" in forming this community consensus) multiple times?
What you say, that if others of my group di... (read more)