simplicio comments on Information theory and the symmetry of updating beliefs - Less Wrong

45 Post author: Academian 20 March 2010 12:34AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (28)
Tags: information log odds bayes theorem bayesian beliefs updating theory

Comments (28)

You are viewing a single comment's thread. Show more comments above.

Comment author: simplicio 20 March 2010 07:21:44AM *  2 points [-]

Okay, so let's take an example. Suppose there's a disease with prevalence P(D) = 6%, and a test with true positive rate P(+|D) = 90% and false positive rate P(+|~D) = 10%.

We have seen a positive result on the test.

We take the information-theoretic version of Bayes theorem:

inf(D|+) = inf(D) - iev(D,+)

 = inf(D) - inf(D) - inf(+) + inf(D and +)
= -inf(+) + inf(rate of TRUE positives)
= log2[ P(+|D)P(D) + P(+|~D)P(~D) ] - log2[P(+|D)P(D)]

inf(D|+) = log2[0.148] - log2[0.9*0.06] ~= 1.45 bits (= 36%)

Now suppose the prevalence of the disease was 70%; then we find

inf(D|+) ~= 0.07 bits (= 95%)

Which makes sense, because the second test merely confirmed what was already likely; hence it is less informative (but even the first is not terribly, due to low prior).

Yeah, I can definitely see the appeal of this method. Great post, thanks!

Comment author: RobinZ 20 March 2010 12:02:48PM 5 points [-]

An even more convenient example:

1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammographies. 9.6% of women without breast cancer will also get positive mammographies. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?

inf(C) = -log2(0.01) = 6.64
iev(+,C) = inf(+) + inf(C) - inf(+ & C)
= - log2(0.01*0.8+0.99*0.096) - log2(0.01) + log2(0.01*0.8)
= 3.28 + 6.64 - 6.97 = 2.95
inf(C|+) = inf(C) - iev(+,C) = 6.64 - 2.95 = 3.69
P(C|+) = 2^(-3.69) = 7.7%

It works! That was fun.

Powered by Reddit Powered by Reddit