Rational Justice

geebee2

I'm interested in how courts and juries might use rational techniques to arrive at correct decisions on guilt.

In a complex case, it would seem to sensible to assess each component of the prosecution and defence case, and estimate the relative likelihood. If the prosecution case is (say) 100 times more likely than the defence case, then you can say the defendant is guilty beyond reasonable doubt.

I never heard of this being done though. I recently made an analysis of the Massei report into the Amanda Knox case. It looked like this ( see http://massei-report-analysis.wikispaces.com/ for the entire analysis and some insight into the numbers below ).

Event	Prosecution Probability	Defence Probability
Phone at cottage at 22:13	1%	99%
DNA evidence correct	50%	50%
Break in staged to look like Rudy did it vs unstaged break-in	10%	90%
Conspiracy among 3 near strangers with no apparent motive vs burglary gone wrong	5%	95%
Murder weapon was two knives rather than one	10%	90%
Time of death 23:30 vs 21:10 according to empty duodenum	10%	90%

This is perhaps a bit vague. It's not a great example, because in the end I didn't find any credible prosecution evidence. It's not entirely clear what the "probability" numbers here actually are, and whether two columns are needed. But hopefully it shows that the Massei's account of the murder is quite improbable, and there is considerable doubt.

I'm interested in possibly devising a more complete framework for how such an assessment should be done, the pitfalls that need to be guarded against (how uncertain are the probability estimates?), and even views as to how "reasonable doubt" should be quantified.

Perhaps readers would like to make an assessment of other interesting cases, to explore the issues.

Or how would you approach this problem?

I'm interested in how courts and juries might use rational techniques to arrive at correct decisions on guilt.

Event	Prosecution Probability	Defence Probability
Phone at cottage at 22:13	1%	99%
DNA evidence correct	50%	50%
Break in staged to look like Rudy did it vs unstaged break-in	10%	90%
Conspiracy among 3 near strangers with no apparent motive vs burglary gone wrong	5%	95%
Murder weapon was two knives rather than one	10%	90%
Time of death 23:30 vs 21:10 according to empty duodenum	10%	90%

Perhaps readers would like to make an assessment of other interesting cases, to explore the issues.

Or how would you approach this problem?

It's not entirely clear what the "probability" numbers here actually are, and whether two columns are needed.

The simplest is just to use odds ratios. It doesn't matter if the likelihoods you give are correct, so long as they are proportional to each other. It would look like follows:

Prior: 1:1 Evidence:

1:99

1:1

1:9

1:19

1:9

product:

1:1371249

So there's 1371249 to one odds in favor of the defense.

There are a few problems with this:

First, the prior shouldn't be 1:1. You'd probably use something about how likely it is for someone that close to commit the crime. If you really want to be rigorous, use 1:6840507000 as the prior and use the relation and all that as evidence, but that's probably overkill.

Second, there are biases that will cause problems exponentially with evidence. You will tend to notice evidence more on one side, and you will tend to think evidence is more likely for one side than it really is. This would be hard to account for. The best I can figure is to keep track of an error term that grows exponentially with each step. You might end up with

(1/64 to 64):(1371249/64 to 1371249*64) = 1:(334.777588 to 5616635904)

From there, you could integrate in some way I can't think of right now, and you'd probably get something on the order of 2000:1 in favor of the defendant.