You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

komponisto comments on Experiment: Knox case debate with Rolf Nelson - Less Wrong Discussion

18 Post author: komponisto 08 July 2011 08:22AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (68)

You are viewing a single comment's thread.

Comment author: komponisto 08 July 2011 08:23:06AM *  3 points [-]

So, I'll start things by picking up our conversation from the other thread. Rolf says he believes that the knife and bra clasp are among the strongest pieces of evidence against Knox and Sollecito; I responded by pointing to the recent independent review that strongly critiqued that evidence.

If [the independent experts'] main point is that the evidence doesn't meet the standard of scientific rigor, then I might not disagree with them on anything factual. Very little evidence, either way, does meet the standard of scientific rigor. Fingerprints never reach the standard of scientific rigor. DNA testing, as practiced, probably rarely if ever meets the standard of scientific rigor. Eyewitness testimony obviously can never come close to meeting the standard of scientific rigor. Heck, most science doesn't meet the standard of scientific rigor. We still need to evaluate evidence on its full merits.

Absolutely true -- but of course the reason scientific rigor exists is because errors happen when it isn't applied, and so we need to take this into account and scrupulously avoid overconfidence when evaluating such evidence. And, even more to the point here, Conti and Vecchiotti (the independent experts) don't merely say that it doesn't meet the standard of scientific rigor; they actually say specifically that it is "not reliable". This is important, because they were asked by the court to assess "the degree of reliability"; hence they could presumably have answered anywhere along a continuum -- and the answer they came back with was "pretty much zero".

So, to confirm that our initial analysis here diverges, around how much are you currently shifting based on the test results for the knife and for the bra clasp? For you, did either one shift P(guilt) by a factor of 100? 10? Not at all?

Roughly speaking, I would say that together, all the evidence against Knox and Sollecito, of which the bra clasp and knife are for me the overwhelming majority (everything else being near-negligible), shifted P(guilt) upward by about an order of magnitude -- a factor of 10, from a prior somewhere between 0.0001 and 0.001 to a posterior somewhere between 0.001 and 0.01. The Conti-Vecchiotti report cuts that down a bit, though not necessarily hugely, since their findings were pretty much what I already expected them to be.

You can guess my hypothesis for why the DNA tests came out the way they did. Do you have a specific alternative hypothesis or hypotheses you want me to consider? Is your main claim here that you believe the knife was accidentally contaminated in the laboratory, and the bra strap was accidentally contaminated in Kercher's room? Or is there a different alternative hypothesis I should consider first?

Yes, I think those would be my best explanations for those results, though there is significant uncertainty even about the attribution of the DNA in the first place. It's also possible that the knife was accidentally contaminated by investigators at Sollecito's apartment, that the clasp was contaminated in the lab, or that there was deliberate malfeasance applied to either of the items at some point; but these are less likely secondary possibilities, in my view.

I assume we're not going to quash any evidence, since we're a court of Bayes and not a court of law? That is, I'm proposing that whenever we want to exclude or diminish evidence, we should have a Bayesian reason for why the evidence doesn't really alter P(guilt). The proposal is partly because it's the correct Bayesian thing to do, and partly because trying to divine and mimic Italian criminal procedure and admissability rules would add (IMHO unnecessary) additional complexity.

Agree entirely. I would only add that a court of Bayes should always be more accurate than a court of law and never less.

Comment author: rolf_nelson 09 July 2011 08:23:04PM *  1 point [-]

Thanks k, today I'll give my thoughts on the knife. I'm sure there are some mistakes in my analysis below, but let's see if we can start to pinpoint areas of disagreement. Let "ddk.lc" be the specific hypothesis that the double-dna knife was accidentally contaminated in the laboratory, and "a.g" be the hypothesis that Amanda is guilty.

I want to estimate the base rate of lab cross-contamination in the late 2000's. Two observations:

  1. Looks like Washington State only admitted to one case of laboratory cross-contamination in homicide cases from around 2001-2003, based on SPI. Maybe there were 2 to 5 that weren't noticed or otherwise were unreported. Washington State has about 200 homicides/year, I'd guess about 1/3 go to state labs?

  2. Cases of mistaken "cold matches" that are investigated by the police but turn out to be cross-contamination seem to be extremely rare.

Contamination rates have probably declined slightly since 2001-2004, (widespread DNA forensics is relatively new), so I'm guessing a base-rate of about one accidental cross-contamination per 50 homicides.

(Can that base rate be applied to this case? I could lower it a little based on the first independent report [EDIT: Sorry, I mean the trial judges' sentencing report] affirming the results, on it being a high-profile case, on the defense being allowed to participate in the testing but declining to fully participate, on no "smoking gun" piece of sloppiness found, and on the defense not stressing any history of past contamination. I could raise it a little based on the second independent report criticizing the lab for not following "international protocols" (though I wouldn't particularly expect them to), and on the low-count DNA. Not having much data here, I'll stick with the base rate as my "wild-ass guess".)

Here's some more WAGs. If there's a single lab contamination, the odds that it contaminates a likely murder weapon at the cottage is about .002. The odds that the DNA spread is Meredith's, rather than one of the many people associated with this case or with other cases the lab is processing in parallel, is about .05.

So I assess P(lc) as .02, and P(ddk.lc | lc) as .0001, which gives P(ddk.lc) as .00002, assuming complete innocence.

In contrast, I estimate P(ddk | a.g) is about .05. So if we exclude the large "systemic uncertainty" of my analysis, the DNA evidence on the double-dna knife alone would make me shift by a factor of 2500 to 1 in favor of a.g rather than ddk.lc.

Let me touch on the circumstantial evidence around the knife, with the caveat that there's even more systemic uncertainty than in my DNA analysis.

  1. The knife was on top of the other knives, and matched one of the murder weapons: meh, shift by a factor of 2 to 1 in favor of a.g over ddk.lc.

  2. The same knife was bleached, and the other knives weren't: Shift by a factor of 5 to 1.

  3. Raffaele, at one point, claimed that Meredith visited his cottage and pricked herself on the knife: Shift by a factor of 100.

  4. Amanda's reaction to the knife: Shift by 10.

  5. Where Amanda's DNA was found on the handle suggests someone stabbing rather than cooking with it: meh, shift by 2.

  6. Amanda wrote in her diary speculating whether Raffaele may have framed her by pressing the knife in her hand while she slept: Shift by 10, this is not something you'd write if you knew the knife isn't a murder weapon.

So for the non-DNA evidence around the knife I give a slightly larger shift (200000 to 1), but paradoxically I assign it less importance because there's more systemic uncertainty and more guess-work on my part, compared with the DNA evidence.

Comment author: komponisto 10 July 2011 08:50:00AM *  3 points [-]

Okay, so, obviously a lot of disagreement here.

Regarding the base rate of contamination, the first thing to remember is that in the case of the knife we're dealing with a Low Copy Number (LCN) sample, for which the risk of contamination is greatly increased unless extremely stringent protocols are followed -- protocols which were not followed here. So we're probably talking about an order of magnitude increase in the base rate: more like 1/5 instead of 1/50.

(Incidentally, there was never any "first independent report" prior to this one: the defense's request for an independent review during the first trial was rejected. What you are most likely referring to is the fact that Stefanoni's own bosses at the Polizia Scientifica signed off on her work, well before the trial. Based on what's in the Conti-Vecchiotti report I consider it likely that they didn't scrutinize it very carefully.)

Secondly, thanks to Conti and Vecchiotti, we have considerable Inside View information about the reliability of this sample in particular. And there's every reason to believe this result is completely bogus. Indeed, Conti and Vecchiotti come pretty close to accusing Stefanoni of outright scientific fraud. Here are some of their observations:

  • The sample in question (Trace B) tested negative for blood, as did every other sample taken from the blade. (Samples from the handle were not tested for blood.) No attempt was made to scientifically determine the actual nature of the alleged biological material.

  • When "quantification" (test to determine whether there was enough DNA to be analyzed) was performed, Traces B and C both yielded a result of "too low". Stefanoni reported Trace B as a positive result, and Trace C as a negative result, without any justification. There is no documentation in the lab data to support her statement in court that the Trace B sample was in the range of several hundred picograms. Stefanoni also claimed to have executed steps in the quantification procedure that are not documented.

  • The "amplification" (chemical copying of the sample in order to produce a large enough amount for analysis) was performed only once, despite the fact (admitted by Stefanoni) that it should be repeated in order to be considered reliable.

  • Stefanoni did not perform negative controls, which could have indicated the presence of contamination.

  • The sample was analyzed in the same laboratory at the same time as numerous samples containing Meredith Kercher's DNA.

In short, everything smacks of a deliberate effort to obtain a certain result -- which was obtained easily enough given the lax procedural standards and absence of safeguards. I thus have very little hesitation in dismissing the knife DNA.

Now, regarding the "circumstantial evidence surrounding the knife" that you listed, I see two major general issues. One is that you are misinformed about a number of details, apparently as a result of uncritically accepting claims from pro-guilt sources (cf. the "first independent report" misconception above). You should be extremely skeptical of pro-guilt advocacy sites such as PMF or True Justice -- they are caught up in a death spiral of hate and constitute an absolute breeding ground of anti-epistemology the likes of which are usually only seen in political or religious conflicts. No, that doesn't mean everything they say is false, and yes, the pro-innocence community is vulnerable to the same disease, but in general I would advise against assuming any claim of theirs is true unless it is explicitly conceded by the other side. For example, this is wrong:

The knife was on top of the other knives, and matched one of the murder weapons

In actuality, the hypothesis of more than one murder weapon was invented specifically to accommodate the fact that this particular knife -- which investigators had already decided was the murder weapon based on "police intuition" followed by the bogus DNA result discussed above -- turned out not to match certain wounds on the victim, nor an imprint found in a bedsheet; a clear instance of motivated cognition and -- given that a smaller knife would have been compatible with all of the wounds as well as the imprint -- a patent violation of Occam's Razor. Another claim without any foundation is:

The same knife was bleached, and the other knives weren't

The closest thing to evidence for this is that a police officer claimed to have smelled bleach upon opening the drawer that the knife was in. (That doesn't distinguish the knife in question from the others, needless to say.)

The other major issue is that in my opinion you wildly overestimate the evidentiary strength of the observations that are true. This applies to all of items 3 through 6. For example, I see no reason why Raffaele's claim about Meredith visiting his house should even have a likelihood ratio as high as 2, let alone 100. The innocent explanation is that he had been told about the knife DNA result, and believed it. Given that he didn't have the Conti-Vecchiotti report available, why do you consider the likelihood of this to be only 1/100 that of a corresponding guilty scenario? Similarly, why is Amanda's reaction to the knife drawer even evidence of guilt at all, let alone 10 decibels' worth?

Item 5 seems to me to be another example of overconfidence:

Where Amanda's DNA was found on the handle suggests someone stabbing rather than cooking with it: meh, shift by 2.

Even if this is so (which I don't have any particular reason to accept), presumably all one has to do to to get one's DNA in this position is shift one's grip on the knife in some way -- as opposed to actually stabbing someone! Isn't the denominator of this ratio quite substantial?

Amanda wrote in her diary speculating whether Raffaele may have framed her by pressing the knife in her hand while she slept: Shift by 10, this is not something you'd write if you knew the knife isn't a murder weapon.

But in this situation "knowing the knife isn't a murder weapon" doesn't come close to being necessary for innocence. She knew that Raffaele was a suspect along with her, knew that Meredith had been killed with a knife, and (depending on the chronology) may have known that Raffaele's knife was considered the murder weapon by the police. Under those assumptions, it's an entirely natural speculation, it seems to me.

Also, here is Amanda's quote in context:

So unless Raffaele decided to get up after I fell asleep, grabbed said knife, went over to my house, used it to kill Meredith, came home, cleaned the blood off, rubbed my fingerprints all over it, put it away, then tucked himself back into bed, and then pretended really well the next couple of days, well, I just highly doubt all of that.

Comment author: rolf_nelson 12 July 2011 04:38:14AM *  0 points [-]

a smaller knife would have been compatible with all of the wounds as well as the imprint

(Massei, 170) seems to disagree (although I could be misreading it), but I welcome any counter-arguments. If you want to claim extreme coroner bias or extreme trial-court bias (or both), then eventually you'll want to separate out the hypothesis of bias and make your case for it.

The closest thing to evidence for (the knife being bleached) is that a police officer claimed to have smelled bleach upon opening the drawer that the knife was in. (That doesn't distinguish the knife in question from the others, needless to say.)

I disagree: "Let me state beforehand that it was extremely clean" (Massei, 99)

The innocent explanation is that (Raffaele) had been told about the knife DNA result, and believed it.

I give the odds that he would tell such a lie if guilty as .1, and if innocent as .001; what odds do you give? Why would Raffaele believe the DNA result? He's more likely to believe it's not the murder weapon if he's innocent, and also less likely to lie if innocent.

Similarly, why is Amanda's reaction to the knife drawer even evidence of guilt at all, let alone 10 decibels' worth?

I think if she's innocent, reacting to the knife like that is about .05 likely, and if she's guilty, about .005 likely. What odds do you give? You wouldn't be more likely to react strongly to an otherwise-irrelevant event if you knew it was going to practically end your life?

Where Amanda's DNA was found on the handle suggests someone stabbing rather than cooking with it: meh, shift by 2.

Even if this is so (which I don't have any particular reason to accept), presumably all one has to do to to get one's DNA in this position is shift one's grip on the knife in some way -- as opposed to actually stabbing someone! Isn't the denominator of this ratio quite substantial?

My reasoning: suppose a chef grabs a knife in cooking position ten times and nose-picking position once. Suppose further DNA is only found in one spot. It's probably ten times more likely that the DNA is found in cooking position, and that his unhygienic habit will remain undetected (aside from the giant nasal scars).

But in this situation "knowing the knife isn't a murder weapon" doesn't come close to being necessary for innocence.

I disagree: not necessary, but probable, by a factor of >10 IMHO.

"So unless Raffaele decided to get up after I fell asleep, grabbed said knife, went over to my house, used it to kill Meredith, came home, cleaned the blood off, rubbed my fingerprints all over it, put it away, then tucked himself back into bed, and then pretended really well the next couple of days, well, I just highly doubt all of that."

I'm more concerned by: “This could have happened: Raffaele [killed Meredith] and then, having come back home, pressed my fingerprints — I was asleep — onto the knife", from London Times

Comment author: komponisto 12 July 2011 08:29:47PM *  1 point [-]

a smaller knife would have been compatible with all of the wounds as well as the imprint

(Massei, 170) seems to disagree (although I could be misreading it), but I welcome any counter-arguments.

The argument there is pretty weak: Massei and Cristiani simply find it hard to believe that a single knife could have caused such different-looking wounds. Not much more than an argument from personal incredulity; they don't support it with any detailed arguments or expert opinions indicating incompatibility of a smaller knife. In fact, they proceed to argue at length that Item 36 itself is compatible with all the wounds, by disputing the defense arguments that it is incompatible with some of them. (For counterarguments on this, see pp. 36-51 of Knox's appeal document, and pp. 20-23 of Sollecito's. Since you cited the original page number of Massei, I'm guessing you can read Italian. If not, I can provide translations. Briefly, the important things to note, besides the object-level defense arguments themselves, are that prosecution expert Bacci "disavowed" his earlier judgement of compatibility, and that civil-party expert Liviero wasn't even aware of the defense experts' counterarguments and hadn't considered them.)

The closest thing to evidence for (the knife being bleached) is that a police officer claimed to have smelled bleach upon opening the drawer that the knife was in. (That doesn't distinguish the knife in question from the others, needless to say.)

I disagree: "Let me state beforehand that it was extremely clean" (Massei, 99)

Such a statement strikes me as extremely weak evidence, likely tainted by hindsight among other things. Bleaching is the kind of thing you establish with chemical tests, not someone's judgement of "looking clean".

The innocent explanation is that (Raffaele) had been told about the knife DNA result, and believed it.

I give the odds that he would tell such a lie if guilty as .1, and if innocent as .001; what odds do you give? Why would Raffaele believe the DNA result? He's more likely to believe it's not the murder weapon if he's innocent, and also less likely to lie if innocent.

Firstly, since we don't have the transcript of the interrogation, we don't know that it was a "lie". It could, for example, have been an exchange like this:

-- We found Meredith's DNA on a kitchen knife in your house!

-- Well, I guess she must have come to my house and pricked herself, then!

But in any case my probability that he would make up some story similar to this (or otherwise say something that would likely be reported as him making up such a story), given (1) innocence, (2) that he had been told about the DNA result, and (3) had demonstrated confusion about his memories elsewhere (e.g. suggesting at one point that Knox had gone out on the evening of the crime), is in the region of 50%. People tend to trust authorities, and defying the data is an advanced rationalist skill that cannot be counted on, especially in the face of shouting policemen. From what I understand, it is relatively easy to get people to make things up in stressful interrogation situations.

I think if she's innocent, reacting to the knife like that is about .05 likely, and if she's guilty, about .005 likely. What odds do you give? You wouldn't be more likely to react strongly to an otherwise-irrelevant event if you knew it was going to practically end your life?

(I assume you meant to reverse those numbers.) Let's make sure we agree on what event we're referring to. My assumption was that you were talking about Amanda's distressed reaction when an officer opened up a drawer containing knives in her cottage (i.e. not Raffaele's apartment -- nothing to do with Item 36). I don't understand why this event would have any life-ending import. I'm not aware of any similar story regarding Item 36, the closest thing being an expression of concern about it to her parents at one point. Very plausible innocent explanations for both include, in the first case, distress at Meredith's demise primed by the sight of knives, and in the second, the knowledge that she had handled the knife while cooking.

(I predict that if you took 100 young females and put them into a situation where they knew a friend or roommate of theirs had been killed with a knife by an intruder, and then showed them a drawer full of knives, the number of those who would react with visible emotion would be well into the double digits.)

My reasoning: suppose a chef grabs a knife in cooking position ten times and nose-picking position once. Suppose further DNA is only found in one spot. It's probably ten times more likely that the DNA is found in cooking position, and that his unhygienic habit will remain undetected (aside from the giant nasal scars).

This seems to be an argument against the proposition that Amanda used the knife for cooking more than for stabbing. But that seems unlikely to be sound, since even if it was used for stabbing at one point, it had presumably been used for cooking a fair amount before. It was a kitchen knife after all, and Amanda had spent a lot of time at Raffaele's apartment.

In any case, this kind of inference seems like it is subject to tremendous uncertainty. How much difference is there between cooking positions and stabbing positions anyway? Surely there are any number of ways to do both. Why 10:1 or 2:1, instead of 3:2 or 10:9? As a matter of fact, I'm curious in general: what sort of evidence of guilt would have a likelihood ratio below 2 for you?

But in this situation "knowing the knife isn't a murder weapon" doesn't come close to being necessary for innocence.

I disagree: not necessary, but probable, by a factor of >10 IMHO.

Can you explain this more? Why should an innocent person have knowledge of what the murder weapon was or wasn't? If she were innocent, why should she, under the circumstances she was in at the moment, with the information she had, doubt Raffaele's guilt any more strongly than she indicated in the diary excerpt (already fairly strong, IMO)?

I'm more concerned by: “This could have happened: Raffaele [killed Meredith] and then, having come back home, pressed my fingerprints — I was asleep — onto the knife", from London Times

That is almost certainly just a mistranslation of the passage I quoted above (in its original form). See here and here (#8). (Admittedly, I haven't found a "neutral" source explicitly recognizing this mistake, but it is extremely plausible a priori, not surprising at all that the media wouldn't bother to correct it, and moreover no one can produce photographic evidence of the second version, unlike the first. Even the guilters use the version I quoted.)

Comment author: rolf_nelson 14 July 2011 07:11:43AM 0 points [-]

While it remains interesting, we don't seem to be getting any traction toward significantly changing each other's mind through this one-on-one debate, should we just cut our losses and end it? I'm open to other suggestions on how to proceed. You haven't yet presented arguments toward innocence, but if we just follow the same pattern we have been so far we're probably not going to get anywhere with those either. Probably the only thing we can agree on is that at least one of us is biased on this case. ;-)

If you want to proceed, I'll ask you whether you agree with my "one contamination in fifty homicides" estimate, and what your probability is that the independent report is basically correct, and whether you would agree with me that this puts an upper bound on the contamination probability. I'll also continue to try to nail down, line-by-line, where we disagree on the facts of the case, and say that I feel you're giving too little weight to the statements of lab technicians, police officers, and trial judges, especially when those people were there and we weren't.

If you don't want to proceed, then I thank you for humoring me as far as you did with this experiment, and I will express my condolences to all innocent people in jail, whether they include Amanda or not, and hope that justice prevails in this case.

Comment author: komponisto 17 July 2011 07:00:01AM 2 points [-]

While it remains interesting, we don't seem to be getting any traction toward significantly changing each other's mind through this one-on-one debate, should we just cut our losses and end it? I'm open to other suggestions on how to proceed. You haven't yet presented arguments toward innocence, but if we just follow the same pattern we have been so far we're probably not going to get anywhere with those either.

I wouldn't have expected us to have made much progress on mind-changing by this point, since we've only had a couple of iterations of back-and-forth; and moreover we've been starting to succumb to the danger of going "shallow and wide" (i.e. listing disagreements on many different aspects of the case) rather than "narrow and deep" (i.e. trying to iron out a particular important point of disagreement). For instance, I think the contrasting opinions on the wounds, or inferences about how the knife was held, are areas of lesser importance, which we probably don't even need to get into at this point, since there are much more important issues to focus on (such as how likely contamination is, or the incompatibility of the stomach evidence with the prosecution's hypothesized time of death).

So making progress would probably require us to pick a small number of narrowly-defined issues to hash out, one at a time.

Here's an important question to assess whether we've said anything important yet: has anything I've said surprised you?

You haven't yet presented arguments toward innocence, but if we just follow the same pattern we have been so far we're probably not going to get anywhere with those either.

That may be right, so if we do continue further it will be best to try to avoid that pattern. In any case, I think it's probably worthwhile for me to at least state for the record what I believe to be the most important arguments for innocence, so I'll go ahead and do that. Here are what I would consider to be the three strongest arguments for innocence, from a starting prior of "Knox and Sollecito have been convicted":

(1) The hypothesized crime would be so singularly unusual as to make a miscarriage of justice a priori as likely as guilt, even without knowledge of defense arguments. In other words, the base rate of crimes in this reference class is less than or equal to the base rate of wrongful convictions.

(2) Guede is unquestionably guilty. Hence, Meredith Kercher's death itself does not require explanation. (There is furthermore little connection between Guede and either Knox or Sollecito.)

(3) The stomach evidence is incompatible with a time of death much beyond 9:00 - 9:30 pm, given the testimony from Meredith's friends about when her last meal took place. Yet there was activity on Sollecito's computer at 9:10 pm according to Massei-Cristiani, and 9:26 pm according to Sollecito's appeal.

(Note that although (1) is in principle screened off by detailed knowledge of prosecution and defense arguments, it is effectively preserved in the form of arguments about lack of motive, lack of criminal history, demographics, etc.)

Now here's what I predict about where this would lead us. Pursuing (1) would lead us to a discussion very similar to the one we've been having about the base rate of contamination. Before we started the discussion I would have been most curious about your response to (2), but given your numbers on the knife evidence it seems pretty clear that you think the evidence against Knox and Sollecito is strong enough to overcome my prior against Knox and Sollecito's guilt given Guede's (whatever yours is). That leaves (3). I'm not exactly sure what your reaction will be, because I don't know how aware you are of this line of argument already. I suspect I could probably get you to agree that it would be extremely unusual for no food to have passed into the duodenum 5 hours after a meal (as required by the prosecution theory), even conditioned on the already unusual fact of none having passed after 150 minutes. However, I can't predict how far you will lower your probability of guilt as a result.

If you want to proceed, I'll ask you whether you agree with my "one contamination in fifty homicides" estimate, and what your probability is that the independent report is basically correct, and whether you would agree with me that this puts an upper bound on the contamination probability.

I don't think I would have a problem positing that the expert report constitutes 50:1 evidence in favor of contamination, possibly much more.

I'll also...say that I feel you're giving too little weight to the statements of lab technicians, police officers, and trial judges, especially when those people were there and we weren't.

This is interesting, because it seems to me that you're giving too little weight to Conti and Vecchiotti. I would be curious to know specifically whose opinion I should be weighting more than I am, and what information you think they are likely to have that I don't. For example, I don't see why I should give particular weight to Massei and Cristiani's opinion, given that they made their reasoning plain in their 427-page report, and I can read it and decide what I think. And why should I trust Stefanoni, in light of what Conti and Vecchiotti say? Should I take seriously Edgardo Giobbi's assertion that his psychological profiling techniques are effective, when he has admitted to attaching significance to the fact that Knox and Sollecito were seen eating pizza? And Mignini, of course, has the Monster of Florence issue.

The only person who was actually "there" is Rudy Guede; is there any particular reason I should believe him?

Anyway, basically, my feeling on whether to proceed is the following: I don't consider it particularly costly to proceed, especially if done in a relaxed, low-key way. There's no particular urgency, beyond that automatically entailed by the quest for epistemic accuracy. It might be more effective to "drag it out" over a more extended period, and exchange fewer messages per unit time, than to try to accomplish mind-changing in a short intense session (which has more of a danger of feeling like a comptetition anyway).

If you don't want to proceed, then I thank you for humoring me as far as you did with this experiment, and I will express my condolences to all innocent people in jail, whether they include Amanda or not, and hope that justice prevails in this case.

I appreciate your saying this, however far we end up carrying the discussion, and I hope you still feel able to say the same at the end (if that turns out not to be now).

Comment author: rolf_nelson 20 July 2011 06:26:35AM 1 point [-]

So making progress would probably require us to pick a small number of narrowly-defined issues to hash out, one at a time.

Sounds good, like you suggested let's cover the time of death, and also continue to go deep on the question of lab contamination.

Here's an important question to assess whether we've said anything important yet: has anything I've said surprised you?

It hasn't been predictable, but it hasn't caused me to shift significantly in favor of innocence or guilt so far. I did learn I was wrong about which knife Amanda reacted strongly to, but that's within the bounds of how many errors I expected to be making here.

I suspect I could probably get you to agree that it would be extremely unusual for no food to have passed into the duodenum 5 hours after a meal (as required by the prosecution theory), even conditioned on the already unusual fact of none having passed after 150 minutes. However, I can't predict how far you will lower your probability of guilt as a result.

I haven't looked into this much. According to Massei, Umani Ronchi, a court-appointed expert, testified that a farinaceous meal takes 6-7 hours for gastric emptying, and additionally that it's possible some of the food passed into the duodenum but then, after death, slid into the small intestine. Massei also claims that even Vinci agreed with the range of 18:50 - 4:50 for time of death. Did the defence experts take into account the composition of the meal, or testify that sliding of the food after death is unlikely?

I don't think I would have a problem positing that the expert report constitutes 50:1 evidence in favor of contamination, possibly much more.

The sample in question (Trace B) tested negative for blood, as did every other sample taken from the blade. (Samples from the handle were not tested for blood.) No attempt was made to scientifically determine the actual nature of the alleged biological material.

OK, what are the odds that a small dna trace left by "stabbing + cleaning" would test positive for blood, and what are the odds a small dna contamination to the knife would test positive for blood? (By the way, do you have a specific contamination hypothesis in mind?) In both cases, keep in mind only one "small zone" of the striation was tested for blood, and the rest of the striation was consumed in DNA analysis.

When "quantification" (test to determine whether there was enough DNA to be analyzed) was performed, Traces B and C both yielded a result of "too low". Stefanoni reported Trace B as a positive result, and Trace C as a negative result, without any justification. There is no documentation in the lab data to support her statement in court that the Trace B sample was in the range of several hundred picograms. Stefanoni also claimed to have executed steps in the quantification procedure that are not documented.

Sounds like she didn't document everything; how much are you shifting based on this? Part of the problem is I don't know how much the average technician documents, so I don't know how usual or unusual this is. If nobody documents everything, but we still see a .02/homicide contamination rate, then Stefanoni's not documenting doesn't change anything.

The "amplification" (chemical copying of the sample in order to produce a large enough amount for analysis) was performed only once, despite the fact (admitted by Stefanoni) that it should be repeated in order to be considered reliable.

Can I get a source for Stefanoni's admission? Is this from the report?

Stefanoni did not perform negative controls, which could have indicated the presence of contamination.

Is this also from the report? Have you translated this part yet?

The sample was analyzed in the same laboratory at the same time as numerous samples containing Meredith Kercher's DNA.

I gave a .05 chance that, if there was a cross-contamination, it would have been of Meredith's DNA. Are you giving a different probability?

Comment author: komponisto 20 July 2011 02:34:24PM *  1 point [-]

I haven't looked into this much. According to Massei, Umani Ronchi, a court-appointed expert, testified that a farinaceous meal takes 6-7 hours for gastric emptying, and additionally that it's possible some of the food passed into the duodenum but then, after death, slid into the small intestine. Massei also claims that even Vinci agreed with the range of 18:50 - 4:50 for time of death. Did the defence experts take into account the composition of the meal, or testify that sliding of the food after death is unlikely?

We're talking here not about the time it takes for the stomach to emtpy completely, but rather the time it takes for ingesta to begin passing into the duodenum ("T_lag"). (At death, there was 500mL of ingesta in the stomach -- consistent with the meal size -- and nothing in the duodenum.) According to this paper, the median value of T_lag is 81.5 minutes, and the 75th percentile is 102 minutes. (Furthermore, the median time for half the contents to empty ["T_1/2"] is 127 minutes, and the 75th percentile is 168.3 minutes.) The prosecution scenario of death during the 11:00 pm hour would require a T_lag of more than 240 minutes, possibly more than 300. My understanding is that this is basically unheard of, whatever the composition of the meal.

Ronchi claimed that the coroner, Lalli, had failed to seal the duodenum via ligature, as is apparently the standard procedure; this was the basis for his claim that food could have slipped into the small intestine. However, video of the autopsy revealed that Ronchi was wrong, and that Lalli had indeed properly sealed the duodenum. (Sollecito appeal, p. 165)

OK, what are the odds that a small dna trace left by "stabbing + cleaning" would test positive for blood, and what are the odds a small dna contamination to the knife would test positive for blood? (By the way, do you have a specific contamination hypothesis in mind?) In both cases, keep in mind only one "small zone" of the striation was tested for blood, and the rest of the striation was consumed in DNA analysis.

There were four samples on the blade, B,C,E, and G that were tested for blood; the results were all negative. These traces were all presumed to be blood by Stefanoni; it seems reasonable to suppose that if there had been blood on the knife, these would have been the most likely spots in which to have found it.

A positive blood test result would require more than DNA from (white) blood cells -- it would require hemoglobin. So it seems to me that the only way to get a positive blood test result from contamination would be to spill a blood sample on the knife. I estimate the probability of this having happened as being in the range of 0.001.

On the other hand, in the event that the knife had been used for stabbing, and that the victim's DNA remained on the knife, I would estimate a probability northward of 0.9 that at least one of the "presumed blood" traces would have tested positive for blood. (Cf. ChrisHalkides' comment below: "Two experts have publicly stated that the chances of cleaning a bloody knife to the point at which blood is no longer detected but DNA is detected, are small." This agrees with my intuition, and "no more than 0.1" seems a reasonable interpretation of "small".)

Sounds like she didn't document everything;

It's worse than that; what Conti and Vecchiotti describe is suggestive of deliberate misrepresentation if not outright fabrication of results. The "several hundred picograms" quantification result for Trace B appears to have been completely made up. She claimed in court that she had obtained this result using Real Time PCR, but this was not the case: records obtained by Conti and Vecchiotti show that another method ("Qubit Fluorometer") was used, and that the result obtained was "too low" -- the exact same as for Trace C. There was no justification for treating Trace B as a positive result and Trace C as a negative, and in particular no reason for subjecting Trace B to amplification.

The "amplification" (chemical copying of the sample in order to produce a large enough amount for analysis) was performed only once, despite the fact (admitted by Stefanoni) that it should be repeated in order to be considered reliable.

Can I get a source for Stefanoni's admission? Is this from the report?

Yes (p.61).

Stefanoni did not perform negative controls, which could have indicated the presence of contamination.

Is this also from the report? Have you translated this part yet?

This is on p. 79. This section of the report, on Stefanoni's knife results, is next in line to be translated (by my collaborator katy_did, while I'll be simultaneuously doing the clasp section).

The sample was analyzed in the same laboratory at the same time as numerous samples containing Meredith Kercher's DNA.

I gave a .05 chance that, if there was a cross-contamination, it would have been of Meredith's DNA. Are you giving a different probability?

Substantially different. Stefanoni is quoted as follows on p. 102 of the report:

"...the knife was analyzed...in the course of these 50 samples attributed to the victim, some were prior to the analysis of the knife, of course, and others subsequent, so in these 50 I don't know if the knife was, I don't know now, a fourth, a third of the way through this flux of analyses..."

This puts me at 0.5 or more. Also note that the number 50 itself gives a probability of 0.1 if we use your estimate of 500 total samples in the lab.

Comment author: rolf_nelson 22 July 2011 07:18:08AM 0 points [-]

Let's talk about the dna some more once you guys have finished translating the relevant parts of the independent report, then, if your argument hinges on details of the independent report rather than just the conclusions.

We're talking here not about the time it takes for the stomach to emtpy completely, but rather the time it takes for ingesta to begin passing into the duodenum ("T_lag").

Sounds good. In your case, for one particular meal where the subjects had probably fasted beforehand, the lag is just under 2/3 of the half-time. If you accept Umani Rochi's half-time of 360-420 minutes, then the lag could be 2/3 of that, or 240+ minutes. Of course, for all I know Umani Rochi could have been referring to the lag time, or the final gastric emptying time, rather than the half-time. Lags could easily be much smaller, or larger, than 2/3 of the half-time in this case.

It sounds like you might disagree with not just with Umani Rochi (a court-appointed expert), and Raffaele's consultant Vinci, but also with another of Raffaele's consultants, Introna, who placed the start of attack between 21:30 and 22:30.

Note that stress (such as being attacked) can increase lag time, so we might be talking about the time the attack started rather than the time of death.

In addition to the starchiness of the meal, I would claim that:

  1. Alcohol (or drug use) may increase lag time, studies differ as to how significant this is though.

  2. Subjects in studies usually fast before the study, which means in the real world I expect lag times to be longer. Meredith also returned home after the meal, which may be more physical activity than the subjects did, though I could be wrong about that.

  3. Subjects in studies don't usually go and eat a snack after the meal, as I believe Meredith did. I would expect this to also increase Meredith's lag time.

Anyway, what's your model here: What do you personally estimate the lag to be based solely on digestion (assuming no slippage)? Maybe you can give a mean and a standard deviation, and we can start by modeling it as a normal distribution?

Ronchi claimed that the coroner, Lalli, had failed to seal the duodenum via ligature, as is apparently the standard procedure; this was the basis for his claim that food could have slipped into the small intestine. However, video of the autopsy revealed that Ronchi was wrong, and that Lalli had indeed properly sealed the duodenum. (Sollecito appeal, p. 165)

How much does application of the ligatures reduce the probability of slippage? If ligatures were not applied, how likely do you think complete slippage would be? If they are applied, what are the odds that (1) the slippage occurs before the ligatures are applied, or (2) the slippage occurs anyway after the ligatures are applied, perhaps due to improper application?

Comment author: rolf_nelson 12 July 2011 04:18:16AM 0 points [-]

So we're probably talking about an order of magnitude increase in the base rate: more like 1/5 instead of 1/50.

I'll posit a factor of three.

(Incidentally, there was never any "first independent report" prior to this one...)

My bad, that was quite confusing of me, I mean the Massei(+Cristiani) report, which is independent of the police lab. Will fix. Thanks!

The sample in question (Trace B) tested negative for blood, as did every other sample taken from the blade.

IMHO irrelevant, Meridith's DNA is incriminating whether it's from blood or tissue, and whether any blood chemicals remained after cleaning in sufficient quantity to test positive.

(Various lab procedural criticisms)

Do you want to formally introduce a hypothesis that, LCN aside, this test was sloppier than the average test at the average lab? (Call this "ddk.slop") If so, one way to numerically assess would be to establish:

  1. How sloppy do you think the test was, in terms of percentile? Are 10% of lab tests as or more sloppy than this one? 1%? Less?

  2. How certain are you that the ddk.slop and its associated percentile is true? (FWIW I currently think this lab test is less sloppy than average.)

Then we can quantify your estimated shift to the base rate.

Comment author: komponisto 12 July 2011 06:08:50PM *  1 point [-]

The sample in question (Trace B) tested negative for blood, as did every other sample taken from the blade.

IMHO irrelevant, Meridith's DNA is incriminating whether it's from blood or tissue, and whether any blood chemicals remained after cleaning in sufficient quantity to test positive.

Nonetheless, conditioned on Meredith's DNA being present, a negative blood test is surely significant evidence in favor of contamination over guilt, isn't it? If it was contamination, you would expect this with near certainty; whereas if the knife had been used to kill Meredith, what are the chances that any cleaning job was good enough to leave no trace of blood, yet bad enough to leave not only Meredith's DNA, but also granules of starch?

(Various lab procedural criticisms)

Do you want to formally introduce a hypothesis that, LCN aside, this test was sloppier than the average test at the average lab?

I'm not sure what you mean by "LCN aside"; LCN would seem integral to such a claim, since ordinary standards would already constitute sloppiness in the LCN context.

But anyway I'm not sure I need to introduce such a hypothesis, since my beliefs about the unreliability of the knife result do not particularly depend on any beliefs I have about reliability of results in the average lab. It suffices for my purposes that the facts mentioned by Conti and Vecchiotti are Bayesian evidence of contamination given general scientific "common sense". While I currently think it unlikely that Stefanoni's level of sloppiness is typical, if it turned out that it was, that would sooner undermine my confidence in forensic lab results in general than my beliefs about this case.

Formally, in other words, I view the base rate of contamination as being screened off by the detailed information about lab procedures, rather than an independent piece of evidence to be weighed.

Comment author: Wei_Dai 11 July 2011 03:17:55AM 0 points [-]

If there's a single lab contamination, the odds that it contaminates a likely murder weapon at the cottage is about .002.

I'm having trouble understanding this. Can you explain a bit more? How did you get .002?

Comment author: komponisto 11 July 2011 07:18:49PM 1 point [-]

My guess is it's probably figured by assuming there are ~500 objects being examined by the lab. However, I would point out that contaminations are unlikely to be independent of each other, and are instead probably correlated via the general level of standards in the laboratory. In other words, if there is a single contamination, there are likely to be others as well (reported or not).

Comment author: rolf_nelson 12 July 2011 04:03:19AM 0 points [-]

My guess is it's probably figured by assuming there are ~500 objects being examined by the lab.

Correct, I'm guessing a mean of roughly 500 examinations per homicide investigation.

...if there is a single contamination, there are likely to be others as well (reported or not).

Agree.

Comment author: rolf_nelson 09 July 2011 07:26:38PM 1 point [-]

I tentatively disagree on the interpretation of the experts' report, 'reliability' means a different thing for court DNA procedure vs. Bayesian analysis. I doubt they would say, "this kind of thing is 99% likely to point to the correct suspect, so it is mostly reliable by our standards".

Comment author: komponisto 09 July 2011 08:28:14PM 1 point [-]

I doubt they would say, "this kind of thing is 99% likely to point to the correct suspect, so it is mostly reliable by our standards".

If they didn't have the option of saying that, then it's hard for me to see what the point of the review was. Is it your position that Conti and Vecchiotti would have been likely to use the same strong language as they did, regardless of the position of this evidence along the continuum of reliability? Could they have said anything harsher if it were truly unreliable? Do you think a similar review of the DNA evidence against Guede would have produced a similar level of criticism?