LESSWRONG
is fundraising!
LW
$

Comment Permalink

My high school psychology teacher gave my class an interesting demonstration on the usefulness of eyewitnesses.

For one class, we found a notice on the door saying that the class had been relocated to another room. We went to that room, and shortly after our teacher arrived, followed by another teacher. He complained that she had not gotten proper clearance to move her class to that room, and he needed it for an exercise for his own class. They spent a couple minutes arguing, and harsh words were exchanged, after which he left the room.

Almost immediately after he left, our teacher asked us to create profiles of his physical description. Estimates of his height ranged from 5'6 to 6'3, his hair was variously described as being brown, black, or red, and his weight was somewhere from 140 pounds to 230.

The information that the students retrieved from their short term memory, which had not yet been encoded as long term memory, was already profoundly unreliable.

See in context

48 Amanda Knox: post mortem

by gwern

20th Oct 2011

2 min read

484

48

Continuing my interest in tracking real-world predictions, I notice that the recent acquittal of Knox & Sollecito offers an interesting opportunity - specifically, many LessWrongers gave probabilities for guilt back in 2009 in komponisto’s 2 articles:

Both were interesting exercises, and it’s time to do a followup. Specifically, there are at least 3 new pieces of evidence to consider:

the failure of any damning or especially relevant evidence to surface in the ~2 years since (see also: the hope function)
the independent experts’ report on the DNA evidence
the freeing of Knox & Sollecito, and continued imprisonment of Rudy Guede (with reduced sentence)

Point 2 particularly struck me (the press attributes much of the acquittal to the expert report, an acquittal I had not expected to succeed), but other people may find the other 2 points or unmentioned news more weighty.

2 Probabilities

I was curious how the consensus has changed, and so, in some spare time, I summoned all the Conscientiousness I could and compiled the following list of 54 entries based on those 2 articles’ comments (sometimes inferring specific probabilities and possibly missing probabilities given in hidden subthreads), where people listed probabilities for Knox’s guilt, Sollecito’s guilt, and Guede’s guilt:

Knox	Sollecito	Guede	LWer
.20	.20	.70	badger
.05	.10	.90	mattnewport
.20	.25	.90	AngryParsley
.05	.05	.95	tut
.05	.05	.95	bentarm
.85	.60	.20	bgrah449
.01	.01	.99	kodos96
.01	.01	.99	Daniel_Burfoot
.40	.40	.90	nerzhin
.45	.45	.60	Matt_Simpson
.33	.33	.90	Cyan
.50	.50	.95	jimmy
.05	.05	.99	Psychohistorian
.40	.40	.90	Threads
.50	.50	.80	Morendil
.15	—	—	Eliezer_Yudkowsky
.20	.35	.98	LauraABJ
.10	.10	.90	curious
.20	.20	.96	jpet
.06	.06	.70	saliency
.80	.60	.95	Mario
.20	.20	.95	Yvain
.70	—	—	Shalmanese
.05	.05	.95	gelisam
.05	.05	.90	Mononofu
.90	.90	.90	lordweiner27 (changed mind)
.50	.50	.99	GreenRoot
.99	.99	.99	dilaudid
.13	.15	.97	Jack
.05	.05	.90	wedrifid
.01	.01	.90	Nanani
.35	.35	.95	imaxwell
.01	.01	.99	jenmarie
.25	.25	.75	Jawaka
.41	.38	.99	magfrump
.40	.20	.60	gwern
.08	.10	.95	loqi
.25	.25	.50	JamesAndrix
.90	.85	.99	Unknowns
.35	.35	.90	Sebastian_Hagen
.90	.90	.99	brazil84
.30	.30	.40	ChrisHibbert
.02	.02	.98	wnoise
.50	.40	.90	John_Maxwell_IV
.10	.10	—	k3nt
.01	.01	.99	Sinai
.00	.00	1.0	KayPea
.00	.00	.60	MerleRideout
.15	.10	.80	TheRev
.01	.01	.99	komponisto
.30	—	—	pete22
.01	—	—	SforSingularity
.00	.00	.90	AnnaGilmour
.05	.05	.95	Seth_Goldin
.60	.60	.95	bigjeff5

It’s interesting how many people assign a high-probability to Knox being guilty; I had remembered LW as being a hive of Amanda fans, but either I’m succumbing to hindsight bias or people updated significantly after those articles. (For example, Eliezer says .15 is too high, but doesn’t seem otherwise especially convinced; and later one reads in Methods of Rationality that "[Hagrid] is the most blatantly innocent bystander to be convicted by the magical British legal system since Grindelwald's Confunding of Neville Chamberlain was pinned on Amanda Knox.")

EDIT: Jack graphed the probability against karma:

2.1 Outliers

If we look just at >41% (chosen to keep contacts manageable), we find 12 entries out of 54:

Knox	Sollecito	Guede	LWer
.45	.45	.60	Matt_Simpson
.50	.40	.90	John_Maxwell_IV
.50	.50	.80	Morendil
.50	.50	.95	jimmy
.50	.50	.99	GreenRoot
.60	.60	.95	bigjeff5
.70	—	—	Shalmanese
.80	.60	.95	Mario
.85	.60	.20	bgrah449
.90	.85	.99	Unknowns
.90	.90	.90	lordweiner27
.90	.90	.99	brazil84
.99	.99	.99	dilaudid

I have messaged each of them, asking them to comment here, describing if and how they have since updated, and any other thoughts they might have. (I have also messaged the first 12 commenters or so, chronologically, with <41% confidence in Knox’s guilt, with the same message.) The commenters:

AngryParsley / Cyan / Daniel_Burfoot / Eliezer_Yudkowsky / GreenRoot / John_Maxwell_IV / LauraABJ / Mario / Matt_Simpson / Morendil / Psychohistorian / Shalmanese / Threads / Unknowns / badger / bentarm / bgrah449 / bigjeff5 / brazil84 / dilaudid / jimmy / kodos96 / lordweiner27 / mattnewport / nerzhin / tut

I look forward to seeing their retrospectives, or indeed, anyone's retrospectives on the matter.

Allknowing and most merciful Bayes;
We have erred, and strayed from thy ways like biased sheep.
We have followed too much the devices and desires of our own hearts.
We have offended against thy axiomatic laws.
We have left undone those updates which we ought to have done;
And we have done those updates which we ought not to have done;
And there is no calibration in us.
But thou, O Bayes, have mercy upon us, miserable wannabes.
Spare thou them, O Bayes, who confess their faults.

Postmortems & Retrospectives

Personal Blog

48

Mentioned in

17[LINK] Amanda Knox exonerated

11A test of your rationality skills

Amanda Knox: post mortem

New Comment

484 comments, sorted by

top scoring

Click to highlight new comments since: Today at 2:02 PM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

[-]bentarm13y390

Over the summer, Eliezer suggested (approximately, I am repeating this from memory) the following method for making an important decision:

write down a list of all of the relevant facts on either side of the argument.
assign numerical weights to each of the facts, according to how much they point you in one direction or another.
burn the piece of paper on which you wrote down the facts, and go with your gut.

This was essentially the method I used in coming to my (probably slightly low) estimate of the probability that Knox and Sollecito were innocent. It just felt like they were innocent, and I saw essentially no reason to suspect they were guilty. I will note that the 'pro-guilt' site that komponisto linked to was just horribly devoid of anything that I might consider evidence (if anything, that site did more to convince me of Knox's innocence than the pro-innocence site), and I did spend probably about 10 minute trying to find some evidence that they had missed, but completely failed.

On a different not, as I said at the time, 0.95 and 0.05 were just proxies for "pretty damn sure" and "pretty damned unlikely" - I have very little idea what 5% probability fee... (read more)

[-]Eliezer Yudkowsky13y510

This is a better summary of what I said than what I actually said, so I hereby declare your distorted version to be my true teaching.

[-]lessdazed13y190

I have very little idea what 5% probability feels like

???

1d20!!!

8FiftyTwo13y

I don't have an intuitive feeling for d20s, but it occurs to me that a useful resource might be a list of day to day events of certain probabilities so we can calibrate our intuitions to them. Googling hasn't found me anything useful, could anyone give an example of an normal event that has a 5% chance of occuring?

[-]saturn13y130

You look at a clock and the seconds are :00, :01, or :02.

4JoshuaZ13y

It is a little over the chance that if you are dealt two cards from a standard deck of cards that one of them will be the ace of spaces. It is a little under the chance that if you are dealt three cards from a standard deck that one them will be the ace of spades. It is roughly the chance that if you pick three random members of the US House of Representatives that at least two of the three will not be reelected. If It is about half as likely as the chance that a given US soldier in Iraq over the last decade will have been killed or too badly injured to return to duty (generally estimated to be around 9%). ETA: This number is wildly off. Disregard. It is slightly less likely than your expectation for Schrodinger's cat to be alive if you run the experiment 5 times. It is a bit under the chance that if you put your money on two numbers on a roulette wheel that one of them will turn up. It is slightly over the chance that if you meet two random South Koreans that their last names will both be "Kim". ETA: Here's a depressing one: It is around the chance that if you pick two children with childhood leukemia that they will both survive five years.

3lessdazed13y

Who exactly? --Le Feu (Under Fire), translation.

6JoshuaZ13y

Oh. Hmm. I don't remember where I saw this but that number is my background fact set. But when I look at the actual numbers this is clearly false. There have been around 30,000 people wounded or killed. (Source) and around a million who have served. That means that the probability of being wounded or killed at all is around .03, which is much smaller, and that's even before the fact that I said wounded severely enough that one can't keep fighting. Also in retrospect my number was obviously too high. Severe failure of rationality on my part. Ugh.

0komponisto13y

I thought that number was highly suspicious, but I attributed it to the combined category (killed or too injured to return -- which of course are very different things from the perspective of the individual concerned!).

2dlthomas13y

It's somewhere between the chance of flipping 4 successive heads and the chance of flipping 5 successive heads with a fair coin.

2thomblake13y

I was going to respond for I thought I knew many such things, but the few that did not involve rolling d20s involved rolling d%.

0wedrifid13y

My guess would be more that 1 in 20 wrong for a 95% confidence.

[-]komponisto13y250

Despite the fact that my opinion on the case has hardly changed at all, these posts -- and thinking about the case in general -- were a tremendous learning experience for me. Some of the lessons include:

Less Wrong is good at getting the right answer. Believe it or not, the strong survey consensus in favor of innocence -- prior to my second post -- came as something of a pleasant surprise to me. You don't find this in many other places, despite the fact that the case is a no-brainer. I had assumed there would be more wishy-washiness and probabilities close to 50% than there turned out to be. (There was some of this, but less than I expected.)
People in general are bad at getting the right answer. As shown by the original verdict, not to mention all the numerous pro-guilt commentators on internet forums and elsewhere. What's surprising about that? Not much, perhaps, but I would say that one thing that is important about it is that it shows that huge, glaring errors of judgement are not restricted to Far Mode. Even on a mundane question such as this, people are susceptible to strange cognitive biases that can severely distort their assessment of evidence.
Confidence should depend

... (read more)

6wedrifid13y

To the extent that I use that nomenclature I would have called this judgement to be a far mode one. It is the throwing about of far mode political abstractions to achieve perceived near mode goals. Those near mode goals have very little to do with the guilt or innocence of the victim (Amanda) and a lot to do with how your political utterance ("She's a witch! Burn her!") will be perceived by your peers.

6komponisto13y

Yes, we seem to have quite different understandings of what these terms ("near", "far", "political") mean. An example of (erroneous) "near" reasoning in my usage would be: "Amanda is guilty because there had to be multiple attackers because there were so many wounds on the victim". Whereas an example of (erroneous) "far" reasoning would be: "Amanda is guilty because f**ck those arrogant imperialist Americans trying to tell us how to run our country".

0Jack13y

True while we're uncertain of your rationality. But at this point I find you reliable enough to think that your confidence is what mine would be if I followed the case as closely as you. And that means I'm just going to adopt your probability estimate.

2komponisto13y

Well, of course, at the time in question no one knew what my probability estimate was; I merely meant that they need not have reached 0.999 confidence from a few minutes of browsing. (Thanks for the compliment, in any case!)

[-]Eliezer Yudkowsky13y140

See my added comment. I did not assign a probability of 15%. I said that if you assigned a probability higher than 15%, it meant you had a really major problem with crediting the opinions of other people and the authority of idiots. My probability that Knox and Sollecito were guilty was "that's privileging the hypothesis", i.e., "I see no real evidence in its favor so same as prior probability", i.e., "really damned' low".

2gwern13y

When people gave ranges, I just used the anchoring number. You gave a range starting at 15%, so that's what I listed.

[-]Eliezer Yudkowsky13y170

That's a neat compact algorithm but this doesn't change the fact that it produces the wrong answer.

Again, 15% isn't the maximum of a range. It's a number that's not just "wrong" but "sufficiently wrong to imply you need to adjust your emotional makeup".

If you need a number for me, put in "<0.01". I wouldn't have bet $20,000 at 99-to-1 odds over it at the time of writing that first paragraph, but I'm not quite sure anymore that this really means my probability is >0.01, it's not like I'd have taken the bet the other way.

[-]Morendil13y130

The Knox thread was one of the first steps in my getting interested in predictions in general. It was a slow process and is still ongoing, but it has had me spend time on various calibration exercises, on PredictionBook, on the Crowdcast instance dedicated to the Good Judgment project, on Inkling Markets because I saw a few arbitrage opportunities there that sounded like fun. I'm not as into predictions as gwern appears to be, but they're growing on me.

All that and I'm still not very sure what to think of the Knox case. Yes, if our predictions were being scored I'd be getting a non-trivial penalty from my 50% chance of her guilt - that is, if we take the outcome of the appeals process as an arbitration of the prediction, and judge, for the purposes of scoring, that she was "in fact" innocent. (I'm not saying I have much doubt now about her innocence: I'm saying that we won't ever know for sure, and part of the point of these prediction exercises is to allow us to better deal with that permanent uncertainty.)

On the other hand, some of the people listed above would be taking a much more serious hit. One thing I've learned from my various exercises is that you can't expect t... (read more)

7gwern13y

One of the unfortunate things about living when we do is that it seems unlikely there will be any future oracles developed which reveal definitive answers to ancient crimes. I refer, of course, to DNA evidence, which gave us an astonishing oracle to ask questions about old crimes, revealing a shockingly high lower bound on the justice system's error rates. If we had been around and recorded predictions about various death row inmates, then the Innocence Project's &etc. results would've been an assessment of our calibration worth writing home about!

5NancyLebovitz13y

I wouldn't give that an extremely high probability. One of the trends in science is getting surprising amounts of information from tiny amounts of input-- DNA is one example, and finding extrasolar planets is another. I don't have specifics in mind, but I wouldn't be surprised if another method or two which are at least as powerful are developed.

3gwern13y

My reasoning goes along the lines of fingerprints and then DNA are highly precise near unique identifiers, which are also pretty sturdy and accurate. I don't know of any more biological traces which could significantly improve, much less be orders of magnitude superior to what went before like fingerprints & then DNA were. There are plenty of future improvements in crime-solving, sure - lifelogging and pervasive surveillance comes to mind as the most obvious improvement. But none of the ones I can think of will be oracles in the sense I mean here of giving us the correct answers for cases we already 'solved', none of them will be retrospective. (Lifelogging will be employed as soon as available, witness the Canadian thing with stitching together hundreds of photos online to identify & arrest scores of rioters; I would be surprised if huge archives of recording built up and then only decades later are suddenly made public and cold case units began cracking cases with them, for example.)

2NancyLebovitz13y

Brain scans which can retrieve memories accurately? Admittedly, this would be limited to crimes with living witnesses. The thing is, I think science leads to weird, surprising discoveries. I'm not going to predict what's impossible a century from now.

[-]Desrtopa13y140

Brain scans which can retrieve memories accurately?

I believe that the current understanding is that memory encoding, not just retrieval, is pretty unreliable, so even if you can read exactly what's in people's brains, it may not be much help.

1gwern13y

And then there's the problem of locating the right person to interrogate - a task equivalent to and as hard as locating the hypothesis. :) But I probably shouldn't be too dismissive: brain-scanning is one of the better contenders for the next oracle.

4Desrtopa13y

Well, having the right technologies can certainly make locating the hypothesis a lot easier; think how much harder it would have been to locate Guede as a suspect without DNA testing. If we had a reliable way of determining self perceived truth value, nearly all interrogation could be narrowed down to "Do you know who did it?" "Who did it?" and "Did you do it?" On second thought, given a device that were capable of doing that with a negligible failure rate, it might be simpler to just replace policing with occasional checkups. "So, committed any crimes this month?"

2NancyLebovitz13y

Self-perceived truth value sounds like a subtle problem. I'd settle for memory testing for eye-witnesses, though memory testing under stress is probably too much to ask.

[-]Desrtopa13y210

My high school psychology teacher gave my class an interesting demonstration on the usefulness of eyewitnesses.

The information that the students retrieved from their short term memory, which had not yet been encoded as long term memory, was already profoundly unreliable.

1dbaupp13y

This would probably only work if the person knew activity X was a crime. There are probably ways of becoming convinced that X wasn't a crime (or vice versa).

1Desrtopa13y

This occurred to me, but I suspect that it would still be easier to catch more people this way than with active policing, particularly for the crimes you most care about catching people at, which most people will already know are wrong and probably if not definitely illegal. And you could always ask them, "Are you cultivating deliberate unawareness?"

0NancyLebovitz13y

The whole point of deliberate unawareness is that it's unaware-- the deliberate part gets hidden in mental fog.

0dbaupp13y

I agree with that this method seems far easier to catch the majority of people committing those crimes, since the cultivation process would most likely be non-trivial, so the "casual" criminal wouldn't have access to it. There is still the problem of someone else doing the cultivation, so one mastermind could create a militia of people immune to the checkups. (I'm not quite sure if this is actually possible though, although, superficially at least, hypnosis seems to do this sort of thing) (Thanks for the "cultivating deliberate unawareness" term)

0gwern13y

Such a system would be far from most current Western legal systems and hence is of not much interest to me in the context of discovering bounds on error rates in current Western legal systems.

0satt13y

I thought the same thing. It also occurred to me that (although it's unlikely) we might discover a neural signature distinguishing spurious memories from accurate ones. Or some less powerful but still useful signature, such as one that distinguishes memories that have been accessed after creation (and so potentially overwritten) from memories that have never been accessed (which are presumably more reliable).

1falenas10813y

There is weak evidence that the memories you make during the day are reviewed during REM sleep, which would mean every memory is gone over at least once.

2Jack13y

Even just reliable brain-scan based lie detectors...

[-]PhilGoetz13y100

I want to see the regression of LW karma (or the log of LW karma) on probability-Amanda-is-guilty!

You ought to leave Eliezer out of the equation, or assign him a Karma value equal to Yvain's, or else he'll dominate the regression.

[-]Jack13y180

Feel free to try and make something fit

8MinibearRex13y

Doesn't look like there's any sort of function you can use, but there are almost no points in the top right corner of your graph. Almost nobody with high karma on LW assigned a substantial probability that Knox was guilty.

2PhilGoetz13y

Neat! It appears the shared version is writable. How can I make a copy of a Google doc, so I can mess around with it myself? I entered this into OpenOffice, and it also gave me a slope of -.02, an intercept of .4, and an R-squared of .02. Weird that the R-squared is so low, since 8 people with log(karma) < .7 gave p > .5, and no people with log(karma) > .7 did. Also, Eliezer gave p=.15, and that doesn't appear on the graph.

[-]komponisto13y100

Also, Eliezer gave p=.15, and that doesn't appear on the graph.

No he didn't.

4Jack13y

Left Eliezer off since we don't have a firm probability for him and his karma is a huge outlier.

2Jack13y

There is a copy option under the file tab. You're welcome to add to mine though. For users with Ln(karma) > 7 no answer was > 0.5. But within that range people were about as attracted to 0.5 as they were to 0.01. For further investigation I'd want like to see residuals and the log10 of the prediction.

2dbaupp13y

Do you want it enough to do it yourself?

[-]Jack13y100

Amazing job putting this together.

I had remembered LW as being a hive of Amanda fans, but either I’m succumbing to hindsight bias or people updated significantly after those articles.

My .13 probability in the first thread definitely went down following further discussion and, in particular, komponisto's second post. In the last year I've been comfortable using "She's definitely innocent" talking about the case with non-LWers.

[-]bigjeff513y90

I updated my 60% guilt for Knox/Sollecito almost immediately after reading the follow-up article. As I noted on that page, my 60% judgement was a clear case of anchoring. I started with the pro-guilt evidence and only managed an 80% guilt after reading only their evidence. That I was then only able to re-adjust down to 60% was absurd. Since I was (and still am) fairly weak as a rationalist, I probably should have withheld any kind of assessment until after I had read the pro-innocence website, rather than try to make an early assessment and update it w... (read more)

[-]APMason13y90

Strange thing about this is, if I've calculated it right, the average probability estimate of Guede's guilt is only ~87%. It seems to me that if this were your real probability estimate of his guilt, and you were on the jury at the guy's trial, you would be obligated to vote innocent. If you operate on the basis that a 13% chance of innocence is not a reasonable doubt, about thirteen out of every hundred people who go to jail will be innocent. That is (let me check) more than one in ten, which strikes me as rather a lot. I think my own estimate of Guede's guilt is above 99%, so I would vote guilty, but I'm surprised the average here is so low.

[-]steven046113y210

If you operate on the basis that a 13% chance of innocence is not a reasonable doubt, about thirteen out of every hundred people who go to jail will be innocent.

That's if everyone who went to jail had a 13% chance of innocence. Presumably much of the time it would be lower.

1APMason13y

Yes indeed. My mistake.

3byrnema13y

Not 13 out of every 100 people in jail, but still 13 out of every 100 people sentenced by the jury as guilty in the case of a probability estimate of only ~87%. ....The argument still works to show that the probability of guilt at 87% is too low to vote guilty.

4Peter_de_Blanc13y

This argument does not show that.

0byrnema13y

Which argument? I meant the argument loosely defined as the one where you count which fraction of innocent people are jailed to determine if the probability of guilt at 87% is appropriate. Steven0641 correctly pointed out that the target space for the fraction isn't all people in jail, but then you modify the target space to all people judged guilty with probability 87% and the argument 'works'.

0steven046113y

The argument works if adding a 13% innocent population to jail is clearly wrong even though sending an individual with 13% probability of innocence to jail is not clearly wrong. Peter's point, I think, is that we don't have that "if".

2komponisto13y

I thought that "13% innocent population in jail is wrong" was a premise, and "individual with 13% probability of innocence in jail in wrong" was the conclusion. Which seems perfectly reasonable to me: if you have an 87% certainty threshold for conviction, it means you're willing to tolerate up to 13% of convicts being innocent, an unacceptably high number by my lights.

2Luke_A_Somers13y

It gets worse - the most severe crimes face the strongest pro-conviction biases.

-1komponisto13y

...which is of course exactly the opposite of how it should work.

5FAWS13y

I agree if you mean that the damage from an irrational bias is higher when the stakes are higher, but disagree if you mean that rational marginal certainty levels needed for conviction would be higher for severe crimes. The risks from letting a thief go free (more thefts) seem lower than the risks from letting a murderer go free (more murders) even compared to the damage done to a potential convicted innocent (assuming no death penalty, and also assuming higher conviction rates would actually result in fewer of the real culprits going free, which often does not seem to be the case).

0steven046113y

So it sounds like you're saying we do have the "if". But are you sure the number is not just unacceptably high because in any realistic example of a 13% innocent population of convicts, many of them would have to have been seen as having substantially greater than 13% chances of innocence? If not for some biasing effect like that, it's hard for me to see why the moral question would suddenly be clear once it was stated in population frequencies rather than in individual probabilities.

0komponisto13y

Actually, no, because the equivalence of the two formulations is obvious to me. But it might not be for everyone; it's well known that many people find thinking in terms of frequencies more intuitive than thinking in terms of bare probabilities. For such people, a statement about probabilities may simply not have any moral force unless and until it is translated into a statement about frequencies.

0Nornagest13y

Well, people's intuitions about justice aren't all that consistent, so I don't think this particular moral question is going to suddenly become clear to all observers no matter how it's stated. That being said, though, I don't think we have any particular reason to think that Guede was convicted on unusually shaky evidence, so it seems reasonable -- given certain assumptions -- to take our estimates of his case as representative of murder cases in general. A 13% innocence threshold for each particular case won't give you a 13% innocent prison population (assuming good estimates, which is probably generous in this context), but if we adopt that criterion and Guede's in the middle of the probability distribution for murder defendants, it seems likely that the resulting population-level incidence would still land on the bad side of 8 or 10%. Which doesn't look much better.

0komponisto13y

By the way, I should probably clarify that I don't think the LW average of 87% probability of guilt for Guede at all means that he should have been acquitted. I attribute the low number to a lack of confidence due to not having delved sufficiently deeply into the case, as per the third point in my earlier comment. One should only believe that a miscarriage of justice has occurred in his case if one believes that the jury should not have had more than (say) 87% confidence. But in order to believe that one would presumably have to be highly confident in one's belief about what information the jury had.

0Nornagest13y

I agree, for reasons outlined here. Like you, I'm speaking hypothetically.

0byrnema13y

I would say that sending an individual with 13% probability of innocence to jail is clearly wrong, because 1 out of 10 of them would be innocent. So the premise instead is: adding a 13% innocent population of any subset or category of individuals to jail is clearly wrong leading to the conclusion: sending an individual with only 87% probability of guilt to jail is wrong

2lessdazed13y

One wonders how many of those are people the jury correctly thinks have done other crimes, or subjectively think deserve more punishment for past crimes. That would be a different malfunction from the expressed intent of the system and would imply the system otherwise does much better than the 87/13 ratio.

1steven046113y

Yes, that's what I meant by what I said. But the problem is that, at least to me, the premise is no more obvious than the conclusion.

1byrnema13y

I see. It's a little more obvious to spell out "more than 1 out of 10 innocent" instead of "only 87% probablity of guilt" but if you see them as immediately equivalent then indeed the argument will do nothing for you.

7billswift13y

In a situation like a trial, where I would be limited to just those "facts" presented by the lawyers, it would be extraordinarily unlikely for me to give better than 90% probability of anything.

7Nornagest13y

Well, I haven't looked at those estimates for a few months, but I'd imagine that a lot of the margin in the outside view of Guede's case comes from uncertainties introduced by the media handling of the case or by an imperfect view of the evidence. Neither of those factors would, presumably, apply to a jury. That being said, I wouldn't be all that surprised if thirteen out of a hundred prisoners in Europe and the US were innocent of some of the charges that put them in jail. It's higher than my own estimate would be, but within the same order of magnitude.

0APMason13y

I agree that it wouldn't be hugely surprising. I meant it strikes me as higher than acceptable.

2AlexSchell13y

I would think that hypothetical juror judgments of guilt or innocence may be a lot more prone to bias than a more "dispassionate" look at the evidence generating a probabilty estimate. Even if one should count one's own hypothetical guilt/innocence judgment as a small bit of evidence in the right direction, explicitly trying to calibrate this judgment with one's prior probability estimate is going to make one over-correct one's estimate.

[-]komponisto13y70

One person whose reflections would be particularly welcome, of course, is Rolf Nelson.

Our debate (which is currently stalled due to my fault -- it's my turn to reply) has dealt with one of the few important pieces of evidence to emerge after the original verdict: the incompatibility of the digestive evidence with the prosecution's hypothesized time of death. (This was covered during the original trial, but was never a focus of discussion among outside commentators until the folks at the JREF forum brought it to attention last year.)

[-]Psychohistorian13y50

Since I was randomly chosen to comment on this, I'll throw in my two cents. I haven't thought about too much and my first instinct was to trust whatever value judgements I had made at the time, which I thought were something like 5-5-95, but were actually 1-1-99. Since me-at-the-time was much more familiar than me-right-now, I'd still probably defer to his judgement; if anything, her exoneration and other evidence should move those numbers slightly closer to the extremes.

[-]Matt_Simpson13y50

I assigned a relatively high probability of guilt for Amanda Knox because of a combination of ignorance and over-correction. I read material on the website of the innocent side first, and felt fairly convinced largely because of their assessment of the DNA evidence. The other website had a conflicting assessment of the DNA evidence, which I didn't know how to adjudicate without trying to learn more about how DNA evidence works. I didn't do this, and so remained uncertain but still thought that Knox was innocent with fairly high probability.

Additionally, I ... (read more)

[-]jimmy13y50

My high estimate came from spending insufficient mental energy to come to a stable estimate as well as, as Eliezer said "if you assigned a probability higher than 15%, it meant you had a really major problem with crediting the opinions of other people and the authority of idiots."

[-]kodos9613y40

So I'm a few months late to the game here, but I was one of the people selected to give my retrospective on this, so here goes:

My original estimates were .01, .01 and .99. I realize now that my calibration was off in the same way Komponisto has conceded his was: those numbers are way too strong to be rational estimates for something you read about on the internet for an hour or two. If I had it to do over again with benefit of hindsight, I'd probably say something more like .1 .1 .9.

The thing is though, in the time since then I've done quite a bit more re... (read more)

[-]JQuinton13y40

Hmm... I'm a bit confused about what was supposed to be predicted here. Were we supposed to predict whether Knox would be convicted, or predict whether Knox actually committed a murder? If I had been involved in the original conversations, I would have assigned a very low probability to Knox's actual guilt, but a higher probability to her being found guilty. One is a question that specifically pertains to Knox herself, and the other is a commentary on the state of the Italian justice system.

4gwern13y

Short of a new oracle (like DNA was previously), we will never have any judgement which is reliable enough to convince both sides that the judgement is correct. So, this was not a case of predicting the legal outcome - you will notice I mentioned I got the appeal wrong - but rather a question of how and whether people have changed their probability since then. Did they increase their belief in her guilt? Decrease it? Leave it unchanged due to a complicated convergence of pro and anti evidence? This wasn't an exercise 'you guys made these predictions, they have been vindicated or falsified, please check your new calibration and ponder how to do better in the future' but 'so, what do you guys think now?' The discussion has been a little more aggressive than I hoped for, but I still see changing opinions, which is healthy: it'd be strange if the passage of time didn't change one's belief at all!

0thomblake13y

For anyone who actually worked out figures before and after, this seems like it would be the least likely scenario.

0gwern13y

Yes, it is extremely improbable that the evidences would approximately or exactly counterbalance but in fairness I had to mention it, else it wasn't a complete breakdown (greater, less, equal).

[-]brazil8413y40

I can't remember exactly what I said in the last thread, but I think my opinion is basically the same now. I am reasonably confident that Knox and Sollecito were involved in the murder and very confident that Guede was.

3TrE13y

What evidence did you collect since that last post, and how shifted it your views (rather, how did it fail to shift them)?

0brazil8413y

The main evidence was that the appeals court in Italy reversed the convictions of Knox and Sollecito. This undermined my confidence a bit. On the other hand, arguing that Knox and Sollecito were involved in the murder has made me a bit more confident in my beliefs. I hope that this is because arguing has given me the opportunity to think more carefully about the case, but it may also be the false confidence which comes from emotional investment in a position. Either way, I doubt it's made much of a difference since I was pretty confident from the beginning.

3Jack13y

Did you at any point update on your fellow Less Wrong posters' estimates?

2brazil8413y

I'm not sure what you are asking, but the opinions of other posters here has not had much of an impact on my own.

[-]wedrifid13y160

And here we have a case study on what not to do and why.

0brazil8413y

If you want to make an argument for why I should put more weight on other posters' opinions about Knox and Sollecito, I'm happy to consider it.

[-]wedrifid13y190

I'm afraid this is a lesson for others to learn by observation and not one which you can learn yourself (without changing your mind). The reasoning goes along the lines:

brazil84 didn't learn from the opinions and reasoning of other fairly rational and intelligent people.
brazil84 expended sufficient energy on the topic in question to be able to arrive at a sane conclusion.
brazil84 did not arrive at a sane conclusion.
Don't do what brazil84 did because it makes you wrong and also makes you look silly.

Note that this is both an somewhat opposing but also complimentary lesson to the one Eliezer notes.

-2brazil8413y

I vaguely recall that you got pretty annoyed at me a year or so ago when I pointed out a contradiction in your reasoning. I suspect that your anger at me over that incident is informing your commentary. But anyway, if there really are any lurkers reading this, feel free to look back at the actual arguments I made concerning Knox and draw whatever you conclusion you like. Also pay specific attention to my exchange with wedrifid.

[-]Desrtopa13y160

I vaguely recall that you got pretty annoyed at me a year or so ago when I pointed out a contradiction in your reasoning. I suspect that your anger at me over that incident is informing your commentary.

I've had no interaction with you on this site at all, but I have read your posts on the previous Amanda Knox threads, and while I believe I have a far greater aversion than wedrifid to making statements so likely to antagonize others, I have to say I find your judgment in this case in conjunction with your position as a lawyer downright frightening.

-4brazil8413y

Well what exactly frightens you? I'm not a judge.

9Desrtopa13y

Judges are mostly selected from among lawyers, so that would be a lot more comforting if I were confident that the selection process were a genuinely good filter for people of exceptional judgment. But I would have a lot more trust in our justice system if I thought that lawyers tended to be people who would not readily become convinced of and argue strongly for positions in the absence of good reasons for believing them true.

[-]Prismattic13y110

Judges are mostly selected from among lawyers...

At least in the United States, judges are mostly selected from among prosecutors. Defense attorneys, including public defenders, aren't very well represented on the bench. General judgment aside, this a serious systemic bias of the system.

2lessdazed13y

What mechanism makes it a problem? I can see why it would perhaps be most fair for an individual to have one of a former prosecutor and former public defender presiding over the individual's original trial and appeal. The more judges make binding rules for the system, rather than in their courtroom, the more of a problem this would be (though I am under the impression this is not the case). But why are individual differences in judicial bias OK?

5Prismattic13y

I see it as a problem in the sense that prosecutors and defenders are conditioned to view the judicial process in somewhat different ways, and I don't think that judges can simply wave away years of conditioning to see things from a prosecutorial point of view. Individual bias isn't great either, but at least there the bias isn't necessarily going to be in the same direction at each step of the process. The fact that the trial judge and all the appeals court judges are all likely to be former prosecutors, on the other hand, cuts the same direction at every step.

4lessdazed13y

I think this is a legitimate but small concern. If half of all judges at each level were former prosecutors and half former defense attorneys, half of those who had a trial and one level of appeal would face the same bias at each step (assuming even promotion, and unless this was specifically corrected for). A more fair system than evening out representation would be embracing a type of bias and having formal rules counteracting that, e.g. have all judges be former prosecutors and have a set of judicial ruels favoring defendants. I think it likely you are being blinded to the gross unfairness of having an individual tried under a former defense attorney when the rules are written with no regard for bias or assuming the average judicial bias is in favor of the prosecutor, or under a former prosecutor even if half of all judges are former defense attorneys, by the legitimate point that fairness would be increased if each defendant had at most one of each type between trial and appeal. This is especially true considering how much less important appeal is than an original trial.

2wedrifid13y

Only mostly? I had assumed it was an actual legal requirement. That's interesting. Where you come from how many judges have ever not been lawyers and how on earth do they know what they are doing? EDIT: From the looks of it some (40) states in the US allow non-lawyers to be low level judges, usually for small towns doing straightforward cases. From what I can tell in Australia (and most comparable countries) a law qualification of some sort is required.

7Desrtopa13y

I don't know if there are any, but given the sheer number of judges, I would suspect that there have been judges who have never served as lawyers; there is no requirement that a federal judge have ever served as an attorney, and requirements for state judgeship vary by jurisdiction. The Bureau of Labor Statistics states that a bachelor's degree and work experience are the minimum requirements for judgeship or a magistrate position, but most workers have law degrees, so I'd take it as implied that some do not, and are thus exceedingly unlikely to have been lawyers.

2Prismattic13y

It's rare now, but used to be common, for prospective lawyers to pursue an apprenticeship rather than a law degree. You can still use either as a qualification to take the bar exam.

4Vladimir_M13y

That depends on the jurisdiction, and also on how exactly you define "lawyer." (Do you mean someone with a law degree, or a member of the local bar association or some equivalent guild? The former is usually, but not quite always, a requirement for the latter.) If anything, in many state and local jurisdictions within the U.S., judges are elected by popular vote, and I would guess that in some of those there are no such requirements for candidates, at least in theory. Interestingly, for a U.S. Supreme Court appointment, a law degree from a top 14 law school (and at least one academic degree from Harvard or Yale) has been a de facto requirement for decades, but as recently as the 1930s and 1940s, there have been occasional SCOTUS justices appointed without a law degree at all.

4wedrifid13y

Apparently the last United States Supreme Court Justice without a law qualification was Robert H. Jackson - although he passed the bar exam without official training and was a prominent practicing lawyer. I haven't found the last time someone was appointed to that role without passing the bar but research so far does seem to suggest it is an entirely political position, without qualifications required.

0Jack13y

They've all been lawyers, it's just not an official requirement.

0pedanterrific13y

Presumably for the same reason there's technically no official requirement that they be human, either.

6wedrifid13y

That is, those with the power to appoint such judges would look like tools if they voted in non-lawyers or non-humans so they will not do either. (It occurs to me there is a mutual exclusion joke in there somewhere.)

0Jack13y

It actually isn't a requirement for Supreme Court justices -- I'm not sure about other cases.

0brazil8413y

I see your point, but I suspect the problem is more in your own judgment than in mine. Consider that I have had the experience of being wrong on these sorts of issues -- and having to face it -- many many times.

7Desrtopa13y

In this case, knowing that you persist in your assignment of a high likelihood of guilt for Knox and Sollecito given the data that's now available to you, I feel confident in saying that your ability to say oops is too poor. After the advent of DNA testing, retrospective analysis of many crimes exonerated people who had previously been convicted. Had this case occurred before DNA testing was available, Knox and Sollecito would most likely have been convicted, but a DNA test should have been sufficient to exonerate them.

0brazil8413y

FWIW that assessment happens to be incorrect. I say "oops" regularly. And even in my professional life where there is a lot of money at stake, I occasionally have to withdraw a case and apologize. Or ask for leave to amend a paper. Exactly what DNA evidence do you believe exonerates them? I am not aware of any, but I am happy to consider it in good faith. And yes, revise my probability estimates accordingly. My understanding is that some doubt has been cast on the DNA evidence against the two. Which isn't the same thing as being exonerated. But I really would like to hear about such evidence.

[-]komponisto13y140

Exactly what DNA evidence do you believe exonerates them?

The DNA evidence showing that Guede was the killer.

This is exactly the sort of thing that gets people exonerated under the Innocence Project. The only difference is, in most cases, the authorities don't usually go back to the scene desperately looking for new evidence to incriminate the original suspect.

-7brazil8413y

0Desrtopa13y

Did you read the independent experts' report that Komponisto linked to?

[-]komponisto13y100

(Er, not to sound vain, but my collaborator and I did a bit more than link to the report! ;-) )

-3brazil8413y

No I did not. As noted above, the main new evidence I have is the fact that the convictions were thrown out. Would you mind quoting the part of the report which talks about the exonerating DNA evidence?

8Desrtopa13y

Each component of the report in which they retested the samples (knife and bra clasp) for biological evidence of Knox and Sollecito returned negative results, as detailed in the conclusion. Having found specific reasons to doubt the results of the Italian crime labs' testing, the retest found no positive evidence associating Knox or Sollecito with the crime, and this is a case where absence of evidence is significant evidence of absence.

-4brazil8413y

Let's make sure I understand your argument: You seem to be saying that the (apparent) lack of DNA evidence on the knife and bra clasp is convincing evidence that Knox and Sollecito were NOT involved in the murder. Do I understand you correctly?

9Desrtopa13y

There was an abundance of physical evidence of Rudy Guede on the scene. If Knox and Sollecito had any physical involvement with the crime, they should have left biological evidence on a similar order, but in fact tests for evidence of their involvement did not return positive results except when corrupted by serious mishandling and poor testing procedures. The knife and bra clasp were major pieces of the prosecution's case because they alleged that they held genetic evidence implicating Knox and Sollecito, but in fact they did not, and even if they had carried evidence of Knox and Sollecito, it would have been a suspiciously small amount of evidence compared to what one would reasonably expect had they actually been involved. Think of the allegation of Knox and Sollecito's involvement in the murder as a claim that there is an elephant in a room. Another party examines the room and finds... X: Well, there does seem to be an elephantish sort of smell in the room, but it's otherwise devoid of large mammals. Y: But you agree that it's a distinctly elephantish sort of smell? X: Yes, it does smell much more like an elephant than anything else I can think of. Y: In that case, the presence of an elephant makes it far more likely for you to observe this smell than the non-presence of an elephant, therefore it meets the bayesian definition of evidence, so you must revise your belief in the presence of an elephant upwards! X: But hiding an elephant in a room is very difficult. If there were actually an elephant in this room, I would expect a lot more evidence than that. I can't think of any plausible way that you could get an elephant in here and hide every sign of its presence but its smell, only very implausible ones. Your claim was sufficient evidence to promote the hypothesis of an elephant in this room to my attention in the first place, so this is less evidence than I need to maintain my prior uncertainty. To maintain my prior uncertainty, I would need to observe e

-2brazil8413y

Well I'll consider this argument, but let's make sure we are on the same page about biological evidence. According to one web site I found, Guede's DNA was found (1) on a swab of Kercher's privates; (2) mixed with Kercher's blood on Kercher's handbag and the left cuff of her sweatshirt; and (3) on toilet paper in one of the bathrooms in the house. Do you agree with that? Not yet as I am skeptical of your argument that involvement by Knox or Sollecito would have most likely resulted in the same kind of biological evidence as there was implicating Guede. But I'm willing to consider your argument and I admit that I have not heard it before today. Let's start by making sure we agree about the DNA evidence against Guede.

2Desrtopa13y

That reflects my understanding of the state of the DNA evidence against Guede. Keep in mind that that does not imply that this is the sum of all DNA evidence Guede left on the scene; when you have already found this much biological evidence, there is no reason to continue searching in order to obtain a comprehensive inventory of every trace left by the suspect.

0brazil8413y

Well, I'm not an expert on DNA evidence but just based on common sense, it seems to me that if playing the primary role in a murder and sexual assault reasonably leaves 2 DNA traces in a person's blood stains a 1 trace in the person's privates, it's plausible that playing a secondary role might very well leave no DNA traces in the person's blood and none in their privates. Of course the DNA traces in other parts of the house are far less interesting since Knox and Sollecito can be expected to have left DNA biological evidence there regardless of whether they played some role in the murder. So I remain skeptical of your argument but I am happy to consider an authoritative source which says that even someone who plays a secondary role in a murder with a knife is very likely to leave biological traces behind. Do you have such an authority? Or are you just going by your own general knowledge and common sense? I disagree. The authorities had every reason to continue searching for DNA evidence since they suspected others besides Guede.

5Desrtopa13y

The prosecution claimed that more than one person must have been involved in the murder because, they alleged, Kercher was stabbed more times than was likely by a single assailant, and the wounds were consistent with the use of more than one knife. However, the latter part has been challenged, and I am myself aware of murders known to be committed by a single person in which the victim was stabbed more times than Kercher was. The allegation was a particularly ignorant one in the first place. However, assuming that it was true that more than one person was involved in stabbing Kercher, then even if he was the only one who sexually assaulted her, then from my own knowledge of forensics I can say that it is extremely unlikely that only one person would have left biological traces on her unless everyone else was wearing extensive protection. Keep in mind that the contrast is not 3 traces versus zero traces, but three distinct samples which have considerably more than the threshold needed to be detectable by the forensic processes versus no traces above threshold level; if anyone else was involved in physically stabbing Ms. Kercher to death, as the prosecution claimed, they would have to have done so while leaving behind no traces within orders of magnitude of what Guede left. I have studied enough of forensics and biological labwork to make more than a common sense guess in this matter, but if my say so isn't enough for you, I could email a forensics specialist, although you could do the same. I'm not aware of any website that explains this, although that doesn't mean that that there are none. The traces of Guede's DNA were acquired by a simple procedure; find a spot likely to yield biological evidence, take a sample, test it. A swab from Kercher's genitals, for example, contained DNA from Guede. They did not test to see "let's see how many swabs we can take which register positive for Guede's DNA," because it's unnecessary. You seem to be treating the distinct sampl

7komponisto13y

None of that, of course, was the real reason. The real reason was that they had initially arrested Knox, Sollecito, and Lumumba on the basis of crackpot psychological theories, and had egg on their face after having to release Lumumba when he turned out to have an unimpeachable alibi; so they needed Knox and Sollecito to still be guilty in order not to look like the complete morons that they, in fact, were. The bottom line was already written, and it was motivated searching from that point on.

0brazil8413y

For what it may be worth, I don't think this is a completely accurate summary of the prosecution's argument. For example, it seems that the prosecution argued that the lack of defensive wounds suggests multiple attackers. Also, the fact that she was (allegedly) stabbed on both sides of her neck. How much above the threshhold were each of the 3 samples which matched Guede? Well what's your job; educational background, etc? I'm not sure I understand your point. Do you agree with me that this sampling would have been done in batches? Do you agree that Knox and Sollecito were suspects almost from the very beginning? Do you agree that therefore the authorities would have had reason to continue collecting DNA evidence even after they found crime scene samples which clearly matched Guede?

5Desrtopa13y

Had she been physically restrained by anyone else, we should strongly expect that person to have left biological traces. Besides, it's easy to stab someone in both sides of the neck without switching hands. Like the claim that she was stabbed too many times for one person to account for, this sounds a great deal like reaching to rationalize a story. I don't know, although from the fact that all sources, including the Massei-Cristiani report, agree that there was an "abundance" of genetic evidence, I can infer that the traces were well above the limit of quantification, whereas the alleged genetic material from Sollecito, when tested by the police, was amplified by a procedure which increases sensitivity by more than an order of magnitude above the base level of detection. This places a ceiling for the amount of genetic material attributed to Sollecito of about one thirtieth of what was contained in any of the samples from Guede, but that's assuming that Guede's samples were only slightly over the limit of quantification. To be described as an "abundance", there was probably many times more. And the independent experts' report determined that the DNA from Sollecito, along with three other people on the sample, was probably due to corruption of the evidence well after the crime was committed. Amanda Knox's DNA was found in easily measurable levels on a knife that, independent report determined, had no traces of blood on it and could not have been used in the murder at all; it was improperly flagged as a piece of evidence in the first place. I have a Bachelor's in Environmental Science, currently pursuing credentials to become a high school chemistry teacher. In the meantime, I tutor. Yes, yes, no. The sampling was done in batches, but not in a comprehensive "let's test everything in the room at once" sweep. Knox and Sollecito were suspects from near the beginning, but it was on the basis of a scenario that was quickly proven to be wrong. Once it was proven that th

-4brazil8413y

So you agree then, that you mischaracterized the prosecution's argument? I'm not sure I understand your point. You seemed to be arguing that the amount of DNA traces left by Guede were "orders of magnitude" above the detection threshhold, i.e. at least a hundred times higher. And that if Knox or Sollecito had been in physical contact with Kercher, they would have left at least a hundredth of the DNA left by Guede -- enough to have been unambiguously detected. Did I misunderstand your argument? Ok, and assuming that's true, they would have had a strong motive to do more swabs right? And if they in fact did so, then under your hypothesis, they should have found additional Guede DNA material, right? And what classes did you take which gave you your expertise in forensics?

4Desrtopa13y

Not in any way that makes their case seem weaker than it actually was. The traces left by Guede were all above the limit of quantification. The only piece of evidence on the scene that contained traces above the limit of detection for anyone other than Kercher or Guede was the bra clasp, which contained DNA from Kercher as the major contributor, and DNA above the limit of detection but below the limit of quantification for several other people, according to the independent experts' report, DNA which they claim was most likely due to mishandling and contamination. That leaves no traces above the limit of detection from any uncontaminated source for any potential perpetrator other than Guede, so if we are to posit that anyone else was involved, we must suppose that they did it while shedding orders of magnitude less biological material. By the time they went to retrieve further evidence, Meredith Kercher had already been buried. Her body, along with the objects on which they had found traces of Guede's DNA, had already been removed. They did not re-swab her body to see if they could get more of Guede's DNA off of it, nor did they attempt to see if they could get more traces from any of the other articles on which they had already found his DNA. Had they scoured the scene for more traces of Guede, I would be surprised if they did not find any, but they weren't looking for more traces of Guede, they were looking for traces of Knox and Sollecito. Further traces of Guede found on the scene 47 days later might not even have been kept due to their lesser value as evidence since there was already plenty of opportunity for corruption and the case against him was rock solid; if they found, say, small traces of his DNA on the floor on the reexamination, they'd have no particular reason to make note of it. Two semesters of biology labs covered forensics. My labs in atmospheric and environmental chemistry also dealt with collection and analysis of trace chemicals from a scene.

-4brazil8413y

Umm, does that mean yes or no? Umm, does that mean yes or no? And what are the numerical limits for quantification and detection? Umm, does that mean yes or no? But when you "scour the scene," there's no way to narrow your search to DNA of a particular individual. Because the matching is determined back at the lab. Agree? If you claim that there is likely other DNA evidence against Guede which was discarded, then I will need a cite. I have an extremely hard time believing that the lab would have tested a sample; realized it was Guede DNA; and not made a note of it in the record which would have come into the trial materials. I don't think that's enough for your opinion to simply be accepted on the basis of authority. Is there no reference online for your claim that secondary participation by Knox or Sollecito -- such as grabbing or holding -- would have left plenty of DNA material which the Italian authorities would have been very likely to find? At a minimum, if what you are is saying is true, then surely one of the defense experts would have testified at trial that Knox could not have been a participant because none of her DNA traces were on or about Kercher's person. Agreed?

[-]Desrtopa13y100

Umm, does that mean yes or no?

For practical purposes, no.

Umm, does that mean yes or no?

It means that I do not strongly expect that there would be anything left on the scene which they would file as further evidence against Guede.

But when you "scour the scene," there's no way to narrow your search to DNA of a particular individual. Because the matching is determined back at the lab. Agree?

Yes and no. If they already have already established previously that person X has come into contact with objects X Y and Z, further testing of objects X Y and Z is likely to reveal further traces.

For the latter last two points, I do not have a strong expectation that further evidence against Guede would have been brought to court given that later retrieval would have been less reliable, and the case against him was already watertight, nor do I have a strong expectation that the defense would have claimed that Amanda Knox could not have been involved based on the weakness of the evidence of her involvement.

If we're going to go on though, let's review.

Meredith Kercher is killed. Knox and Sollecito are waiting outside when the police arrive to investigate the residence, and Edgarg... (read more)

4Jack13y

I'd like to hear people's probabilities for Brazil revising his estimate down below 50% in the next year.

3JoshuaZ13y

This is interesting. But I can see someone seeing it as rude to speculate on the chance that someone else is going to change their mind. In some contexts this could come across as condescending and can easily make worse feelings that a discussion involves others who are opponents rather than people mutually trying to get a better understanding of reality. With that disclaimer out of the way: I'm estimate an 8% chance that Brazil will reduce his estimate below 50% in the next year and publicly say so on Less Wrong.

0Jack13y

Mine is 2%.

2komponisto13y

Hmm, from the look of things, it might be time to update the model...

0Jack13y

Whoa! Indeed :-( Edit: In fact not my prediction stands.

-6brazil8413y

0brazil8413y

Err, that should be "very likely to leave biological traces behind in sufficient amounts that investigators are likely to find and match them."

[-]wedrifid13y100

I vaguely recall that you got pretty annoyed at me a year or so ago when I pointed out a contradiction in your reasoning.

I don't recall any conversations with you. (Mind you I expect I would have if I believed you then. Actually being wrong is embarrassing.)

No, from the premise "brazil84 is blatantly and obviously wrong despite paying attention to the topic" "don't do what brazil84 did" is a reasonably good inference to make. But as I noted you don't share that premise so naturally you should not be expected to believe it. This is why you were not the intended audience.

5lessdazed13y

Usually, it depends.

5wedrifid13y

Oh, true. For example your correction here gives somewhat less than 0 embarrassment points. Let me clarify that to "In cases where someone is attempting to criticize me regarding a substantial assertion I have made in a manner that makes the situation highly status relevant actually being wrong is embarrassing".

-5brazil8413y

9magfrump13y

Wedrifid is just like that. All the time.

3brazil8413y

Ok, then perhaps my suspicions are unwarranted.

0pedanterrific13y

Are you referring to the 'exchange' that starts around here and continues from there? If so... I'm not so sure bringing this to people's attention is in your best interests.

-2brazil8413y

No, I was referring to the exchange in the earlier Knox thread. For me, this is not a competition to see how many people I can win over or how many karma points I can accumulate.

1pedanterrific13y

If it's this you're referring to, I have to wonder how you got from there to "I suspect that your anger at me over that incident is informing your commentary". That seems like blowing it way out of proportion.

0brazil8413y

Ok, I think I found it: http://lesswrong.com/lw/1j7/the_amanda_knox_test_how_an_hour_on_the_internet/1c6t?context=9#comments And here's an exchange between me and him which got pretty personal: http://lesswrong.com/lw/1ph/youre_entitled_to_arguments_but_not_that/1njx?context=1#1njx He's apparently deleted some of his comments, but I wouldn't have asked him to stop making things personal if he hadn't done so. Here's one thing which he said (and later deleted, it seems): Clearly he and I have some history here, to put it politely. Anyway, was reading over the thread and now I remember I banned the guy.

0pedanterrific13y

Yes, that's the third link I suggested. And that was the first. Dude, that was in the post you linked directly to.

0brazil8413y

Ok, so you can see that he and I have a history here. It seems pretty likely to me that his most recent comments were motivated by personal animus. Feel free to draw your own conclusions.

6pedanterrific13y

Okay, looking back, wedrifid lays out his point as: And you respond with... an ad hominem in the form of an appeal to motive. Hm.

-6brazil8413y

0brazil8413y

No, that wasn't it either.

4pedanterrific13y

...Maybe this one? (That's the one that ends in you saying "Lol, that's nonsense.") Side note: I'm not sure whether to be amazed, impressed or disturbed at how many arguments you've had with wedrifid. This whole line of conversation is veering into the farcical.

-3brazil8413y

Well the bottom line is that he and I have a history here, a lot more than I realized a few minutes ago. And now he seems to be claiming he doesn't remember our exchanges; it's apparently just a coincidence that he needs to publicly announce -- in essence -- I've reached conclusions which are insane.

7pedanterrific13y

To be fair, those exchanges probably wouldn't be particularly noteworthy from wedrifid's point of view.

4brazil8413y

I concede that if he has personal exchanges like that on a regular basis, it's plausible that he really doesn't remember me.

4thomblake13y

Given these two statements, I'm surprised at your surprise.

-2brazil8413y

What exactly do you think I am surprised about?

4thomblake13y

Your memory of the exchanges seems to have been hazy at best, and yet you seemed incredulous that wedrifid doesn't remember them.

-5brazil8413y

4Jack13y

Well, the judgment of people here is evidence just like anything else. Lets say I initially predicted Knox's guilt with p=0.01, Since I think my beliefs track the truth and the beliefs of other Less Wrong posters track the truth I should expect other posters to agree with my assessment if my belief is accurate. The majority of posters disagreeing with me is far more likely if I'm wrong than if I'm right. So upon learning that the vast majority of posters disagree with me I should be more uncertain about my prediction. How uncertain I should be is a difficult question-- in many cases in that thread it was resolved by discussing evidence. Many people with initially high probabilities shifted their estimates downward after evidence they missed was pointed out to them. If you think you have evidence other Less Wrong posters don't have then it makes sense to not take their opinions seriously. Alternatively, if you think Less Wrong posters are irrational or poorly calibrated and don't expect their beliefs as a group to track the truth well then it makes sense to more or less ignore their opinion. I suppose one could also ignore the opinions of the Less Wrong posters on the ground that the opinions of random people reading about the case are swamped by the opinions of people who have studied the case for months-- and thus make very little difference. But now Knox and Sollecito have been released-- if your trust in the experts was what lead you to ignore Less Wrong you should update on the new court decision. So why didn't you update on the opinions of Less Wrong posters?

3MarkusRamikin13y

I wonder. The opinions of members of a given community are not independent events. There's influence by high status members, and by perceived community consensus (note how in a previous post, brazil84 got downvoted just for admitting, when asked, that this consensus didn't move his own opinion much - I don't know, but to me that's ominous). So isn't there's a risk of counting the same evidence (the arguments and facts that convinced the "first movers" in forming this community consensus) multiple times? What you say, that if others of my group disagree with me and I'm in a strong minority, then I'm probably wrong - how far does that go? The majority of humanity is probably wrong about a lot of things that we on Less Wrong are probably right about, by virtue of our greater rationality, and we don't seem to be updating in their direction, are we? Well, if brazil84 is a lawyer, then similarly, by virtue of his expertise, it seems reasonable to me that he should not easily let his opinion be influenced by that of laymen.

9komponisto13y

That might make sense if the question under discussion were a legal question (e.g. how a statute is likely to be interpreted by a court). But that isn't the case here. In fact, even if the domain that brazil84 is claiming expertise in -- determining whether people are telling the truth or not -- were one in which lawyers were more likely to have expertise (and frankly I know of no reason to believe this), the fact is that it has precious little relevance to this case. This case is not about which human statements to believe. Instead, it's about applying Occam's Razor to physical evidence.

6MarkusRamikin13y

Point taken.

2brazil8413y

It's a combination of having little respect for the opinions of anonymous internet posters as well as faith in my own ability to look at incomplete evidence concerning real world disputes and draw reasonable conclusions. As an attorney I do this every day. In fact, my livelihood depends on doing it. All day long people call me up and spin tales and I have to guess at what happened in their case based on limited evidence. I've been wrong many times over the years, both in believing people who turned out to have been BSing me as well as being skeptical of people who turned out to have been telling the truth.

[-]wnoise13y150

anonymous internet posters

Pseudonymous. There are many similarities, but having a long-standing name does have significant differences, even if the name isn't tied to one's "real-life" name.

[-]Prismattic13y140

There seems to be a certain disjoint between the second half of this paragraph and the first.

0RomanDavis13y

Confidence isn't really about evidence?

[-]loqi13y110

So, to summarize why you didn't update:

You didn't know the names of the people commenting.
You have faith that you're more reliable than those people.
You would lose your job if you weren't so great at seeing through bullshit.
You have often failed to see through bullshit.

Boy was Upton Sinclair ever right.

2brazil8413y

I'm not sure that's the way to put it, but let me ask you this: How much stock do you put in the unsupported assertion of an anonymous person on the internet? Please quote me where I made that assertion. Well I need to be decent at a minimum. But basically yeah. I assess cases day in and day out. That's a huge advantage. I know that I'm much better than I was 15 years ago, even though I was just as smart then as I am now. Sure, getting this kind of feedback is a good way to improve one's judgment. Do you seriously disagree? :shrug: I agree, but employment is sadly not the only motivator for self-deception. Let me ask you this: Do you agree that the tone of your post is a bit nasty?

9Vive-ut-Vivas13y

How much stock do you put in the supported assertion of an anonymous person on the internet? I think that's a more relevant question here. To what degree does a poster's anonymity detract from his argument?

0brazil8413y

Quite a lot. But I don't think that's the right question. See, the basic argument being made is that even though I have considered Mr. Anonymous' arguments and decided they were without merit, I should still be significantly less certain of my position simply because a number of these anonymous people (making basically the same weak arguments) disagree with me. Did I misunderstand the argument being made?

2komponisto13y

Yes. The point is that in "deciding [the arguments] were without merit", you didn't take sufficient account of the quality (not merely the quantity, by the way) of the people making them. If a high-quality person says "X is true", you might be able to dismiss it if you have sufficient knowledge. But if they say "X is true because of A,B, and C", you can't dismiss X without also dismissing A, B, and C. And here the problem is with your judgement about A, B, and C, not (just) your judgement about X.

0brazil8413y

I'm pretty confident that I did. If you see a problem with the arguments I made back in the original thread, please feel free to respond (preferably there) and I'm happy to consider your point in good faith.

5Desrtopa13y

To the extent that you don't think that you're more reliable than those people, you're engaging in a treatment of evidence that is simply wrong. The fact of someone's belief is evidence weighted according to the reliability of their mechanisms for establishing belief. That's the principle behind Aumann's Agreement Theorem.

1brazil8413y

I'm not sure I understand your point. My belief that I have superior judgment in this area is based on actual knowledge about myself and my experiences. "Faith" implies that there is no such basis. I don't recall claiming or implying that I was basing my assessment on "faith," but I could be wrong. Which is why I am giving loqi a chance to back up his statement.

4Desrtopa13y

But not knowledge of the other commenters and their experiences, whom you seem to have lumped into the reference class of "anonymous internet commenters," which you assign a low assessment of competence. If you want to find a lot of people with significant expertise in rendering judgment under uncertainty, I think this is a pretty good place to look.

-7brazil8413y

0Jack13y

If "have faith" is changed to "believe" everyone here should agree.

0thomblake13y

Nobody seems to have answered this question directly, though it seems easy... See the direct parent of the post you were replying to (which I think should have been obvious since it was presented as a summary): Also, don't you at least see the tension between: It seems the logical conclusion is that you've lost your job.

-2brazil8413y

Ok, so you agree that in the exact post where I used the word "faith," I summarized the factual basis for confidence in my own judgment? That would be the case if my livelihood depended on exercising perfect judgment at all times. Which fortunately it does not. Let me ask you basically the same question I asked the other poster: Do you agree that getting feedback about one's judgment (including being wrong from time to time) is helpful in improving one's judgment?

2thomblake13y

No, I don't particularly care to parse all that enough to agree to anything. I was just answering your question since it seemed like nobody else had bothered to. People seem to have an odd problem answering questions with obvious-seeming answers, even though they are often helpful to people. For example, the other day on aiqus someone was asking how to type the | symbol, and the answer was straightforwardly a series of directions starting from locating the "Enter" key on a US keyboard. It turned out to be very helpful to the OP, as there was a piece of lint blocking the | symbol.. I was pleasantly surprised that the OP did not merely become the subject of ridicule, as I've often seen with 'obvious' seeming questions in other contexts. No thanks.

-3brazil8413y

Suit yourself, but you will be missing the problem with loqi's statement. Again, it's your choice. But I think that answering the question will help you to see why it's not necessarily a contradiction to (1) have one's livelihood depend on making good judgments; and (2) regularly make judgments which turn out to be wrong.

0thomblake13y

I saw that. That's why I used the word 'tension' rather than the word 'contradiction'. (Though looking for a reference for how the word 'tension' is used in the discipline of Philosophy, I can't seem to find anything online - it's used extensively on SEP, and there was a book written in 1936 on the word's proper use, but the sense used in Philosophy doesn't even make it into OED).

-2brazil8413y

Well you also said "It seems the logical conclusion is that you've lost your job."

4thomblake13y

Indeed, that's why I used the word "seems". A good rule of thumb: If it looks like someone is making an obviously stupid mistake, you're probably misunderstanding them. It's a benefit of the principle of charity.

-1brazil8413y

I don't understand your point. Are you saying that you knew all along that there wasn't contradiction; that you were simply observing that there might appear to be a contradiction to some people?

0thomblake13y

Yes No, I was initially pointing to the tension between the two statements, and underscoring that by noting the seeming implication. You did not acknowledge the tension when those statements were juxtaposed by loqi, so I was trying to make it clear that they are in apparent conflict. Given "S will lose his job if he could not X" and "S often makes mistakes when trying to X", it does not deductively follow that "S lost his job", but it's the result to bet on. Learning in that context that S did not lose his job, one should perform a Bayesian update to decrease the probability of the premises.

-4brazil8413y

Ok, I see your point now. But using the same principle of charity, it's easy enough to read my statements so that they are not in contradiction (or tension) with eachother.

-1loqi13y

Yes. It's a combination of having little respect for the feelings of typically-wrong pseudonymous internet posters as well as faith in my own ability to look at incomplete justifications for sloppy reasoning and draw snarky conclusions.

1brazil8413y

Ok, and again my questions: Please quote me where I made that assertion. Sure, getting this kind of feedback is a good way to improve one's judgment. Do you seriously disagree?

-4loqi13y

Please quote me where I accused you of having faith that you're more reliable than those people.

0brazil8413y

Right here: http://lesswrong.com/lw/84j/amanda_knox_post_mortem/52b8 By the way, I have my own rules of debate. One rule is that I will not engage with people who "strawman" me, i.e. misrepresent my position. I also won't engage with people who refuse to answer reasonable questions to let me understand their position. So I will try one last time: Please quote me where I made that assertion. Sure, getting this kind of feedback is a good way to improve one's judgment. Do you seriously disagree? ---------------------------------------- Your choice.

0loqi13y

Thanks! Thanks!

-5brazil8413y

8Desrtopa13y

Keep in mind that you are yourself an anonymous internet poster dealing with other anonymous internet posters with confidence in their ability to look at incomplete evidence concerning real world disputes and draw reasonable conclusions. I would say this is a situation where consideration of the outside view is warranted.

2brazil8413y

Well to me, I'm not anonymous. But anyway, I also try to go by peoples' actual arguments. I think this is a reasonable amount of consideration.

5Desrtopa13y

Which is a very tenuous basis on which to put yourself in a separate reference class. You should adjust your confidence according to the strength of others' arguments relative to what you would expect given your prior confidence value, and you should also adjust your confidence according to the fact of others' belief weighted according to your confidence in their mechanisms for establishing truth. If I believe proposition A, and someone gives me argument X for disbelieving it, and I find argument X weak, I should adjust my confidence little if at all. But if a large population of people whose judgment I have no reason to believe is less sound than my own for cases in this class tells me that proposition A is wrong on the basis of argument X, and I'm just not getting it, I should significantly decrease my confidence, on the likelihood that I really am just not getting it.

1brazil8413y

Well let me ask you this: roughly speaking how much weight do you give to the unsupported assertion of an anonymous person on the internet versus your own conclusions of which you are reasonably confident in an area where you are reasonably confident of your skill and experience?

3Desrtopa13y

Depends on where the anonymous internet people are selected from. From Youtube comments? Very little. From here? Quite a lot more. If I knew that it were something that the people here had put a lot of thought into, and that nearly everybody here thought that I was completely wrong, I would need tremendous prior certainty not to be reduced below .5.

0brazil8413y

Even if the dispute were in an area where you believed you had unusual expertise?

7Desrtopa13y

If the other members were aware of my assessment of my expertise and reasons for assigning it, and were not moved from high confidence that I was wrong, then yes. I would need very strong confidence in my having unique qualifications to not mostly discount on the basis of their discounting.

4Jack13y

Quoted so it won't get missed. This is a really important point.

0brazil8413y

I think the difference is that you have a lot of respect for posters here as a group. I do not.

[-][anonymous]13y30

This comment is unrelated to the main article.

I take issue with your Bayes-prayer. I don't mind so much that it seems to be just a normal prayer with some replaced words, rather than being something good in its own right, though I think this would offend other LWers. However, it does violate one message on LW that I've found very important to internalize:

Never confess to me that you are just as flawed as I am unless you can tell me what you plan to do about it.

So, apparently, we're really really bad at bayescraft. What are we going to do about it?

5Normal_Anomaly13y

It doesn't offend me, and I don't anticipate that it would offend many other LWers. So I'm going to test this with a poll. If you were NOT offended by changing a Christian prayer to be about Bayes, upvote this comment. EDIT: This isn't actually what eridu was talking about.

5wedrifid13y

Not offended but I do think it is rather lame. It did make me downvote and mentally dissociate from the post. More "ewww" than "How dare you!".

0gwern13y

Was it really that lame? Man, now I'm wondering whether the Fate/Stay Night bit of http://lesswrong.com/lw/7z9/1001_predictionbook_nights/ turned off a bunch of readers who simply haven't mentioned it.

0[anonymous]13y

I really liked that, because I think it captured a good part of the essence of tsuyoku naritai. I don't think the Bayes-prayer did.

4[anonymous]13y

I think this would offend the LW zeitgeist because to me, this seems like awful political art. I tried to imply that because I don't like linking to the sequences every time I make some point based off of them, but I think you missed that. Taking that one line out of context didn't help. Edit: To be as explicit as I can, I don't care that this is a christian prayer. I just think it's not well done, because it doesn't reflect tsuyoku naritai.

2komponisto13y

To me it seems humorous. Which is in stark contrast to awful political art such as the poem EY describes in the linked post.

2[anonymous]13y

Why is it humorous besides pointing out that the author is on our side?

4komponisto13y

It's a combination of irreverence (mocking religion by treating non-divine entities as gods), incongruity (you don't expect a bunch of science-minded techno-nerds to sit around praying), and self-mockery (poking fun at our level of enthusiasm for Bayesian concepts).

-1[anonymous]13y

None of that resolves to anything more than "I'm on your team! Go team!"

0Normal_Anomaly13y

That makes sense. I thought you were saying that other people objected to using language originating with Christianity. Your complaint is perfectly reasonable and I slightly agree with it.

0[anonymous]13y

I apologize for not being more clear.

1Desrtopa13y

I wasn't offended, but I did find it a bit ridiculous, and not really in a funny way, although in a different context I might have found it funny.

1Normal_Anomaly13y

If you WERE offended by by changing a Christian prayer to be about Bayes, upvote this comment.

-6Normal_Anomaly13y

0lessdazed13y

These might be relevant. I mean, these might be rationally Bayesian.

1gwern13y

If there's anyone here who has made more suggestions than me and tried harder than me on calibrating one's predictions, I would appreciate an introduction so I can pick their brain.

2[anonymous]13y

I'm saying it deserves mention within the prayer, to remind the reciter that it's not enough to confess one's flaws without also forming a plan to obliterate them.

1lessdazed13y

If Bayes is all knowing, that sort of defeats the point, doesn't it?

[-]John_Maxwell13y00

I'd appreciate being removed as I don't feel like I spent much time reading or thinking before making my estimate and probably never should have commented. (Sorry for ruining your perfectly good and valid experiment...)

Edit: Actually, it doesn't look like I spent all that much less time on a log scale than other folks. It does seem like we didn't really have enough time to get a solid grip on things though.

[+]lukeprog13y-50

Moderation Log

484Comments