wedrifid comments on The Amanda Knox Test: How an Hour on the Internet Beats a Year in the Courtroom - Less Wrong

42 13 December 2009 04:16AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Sort By: Best

You are viewing a single comment's thread.

Comment author: 13 December 2009 05:26:48AM 5 points [-]

How did Less Wrong do by comparison? The average estimated probability of Amanda Knox's guilt was 0.35 (thanks to Yvain for doing the calculation).

Yvain! How could you? What did the probabilities do to deserve that kind of abuse? (I strongly assert that averaging the probabilities is not a good way to combine such estimates.)

Comment author: 13 December 2009 03:01:12PM 3 points [-]

...and why do you assert that? If you have good reasons, I'd like to see a top level post on the subject, since this is my natural response to a bunch of probability estimates given by different people with the same information who are rational enough that I care what they think.

Comment author: 13 December 2009 05:51:55PM 3 points [-]

Not a top level post because I don't think I have the definitive say on the matter, but I made an article in the wiki that illustrates why the mean of the log-odds makes more sense.

Unfortunately, the wiki appears to have issues with math right now, so the article has an ugly error message in it. The formula works fine in the Wikipedia sandbox. If anyone knows what is going on or has any other changes, feel free to edit.

Comment author: 13 December 2009 09:52:02PM *  1 point [-]

...and why do you assert that?

Somebody with no information does not so effectively counterbalance ten people who can describe the positions of every atom on the planet.

I calculated an example involving 0.99, 0.5, 0.745 and (1-10^20) but then I noticed badger's link beat me to it.

Comment author: 13 December 2009 09:09:55AM 2 points [-]

What would the right thing look like? Averaging the log-odds ratio?

Comment author: 13 December 2009 05:33:23PM 1 point [-]

That's what I would do. If one person is almost certain (say 1/(10^10^10)) then the strength of their view would be represented. Of course if anyone gives an irrationally low or high answer, or puts <=0 or >=1, then it overweights their views/blows up.

Comment author: [deleted] 13 December 2009 06:27:28AM 2 points [-]

I wonder what implications this has for the method of choosing priors I came up with that is "ask everybody in the world what they think the priors should be, normalize the invalid ones, and take the average of all of them".