= 27d567bc2d48d9f40e9ccc20ea370b15)
DanielVarga comments on Confidence levels inside and outside an argument - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (174)
It might help to work an example.
Suppose we are interested in an event B with prior probability P(B) = 1/2 which is prior log odds L(B) = 0, and have observed evidence E which is worth 1 bit, so L(B|E) = 1 and P(B|E) = 2/3 ~= .67. But if we are meta uncertain of the strength of evidence E such that we assign probability 1/2 that it is worth 0 bits, and probability 1/2 that it is worth 2 bits, then the expected log odds is EL(B|E) = 1, but the expected probability EP(B|E) = (1/2)*(1/2) + (1/2)*(4/5) = (.5 + .8)/2 = .65, decreasing towards 1/2 from P(B|E) ~= .67.
But what if instead the prior probability was P(B) = 1/5, or L(B) = -2. Then, with the same evidence with the same meta uncertainty, EL(B|E) = L(B|E) = -1, P(B|E) = 1/3 ~= .33, and EP(B|E) = .35, this time increasing towards 1/2.
Note this did not even require meta uncertainty over the prior, only the uncertainty over the total posterior log-odds is important. Also note that even though uncertainty moves the expected probability towards 1/2, it does not move the expected log-odds towards 0.
Note that your observation does not generalize to more complex logodds-distributions. Here is a simple counterexample:
Let's say that L(B|E)=1+x with chance 2/3, and L(B|E)=1-2x with chance 1/3. It still holds that EL(B|E)=1. But the expected probability EP(B|E) is now not a monotone function of x. It has a global minimum at x=2.
Indeed. It looks like the effect I described occurs when the meta uncertainty is over a small range of log-odds values relative to the posterior log-odds, and there is another effect that could produce arbitrary expected probabilities given the right distribution over an arbitrarily large range of values. For any probability p, let L(B|E) = average + (1-p)*x with probability p and L(B|E) = average - p*x with probability (1-p), and then the limit of the expected probability as x approaches infinity is p.
I notice that this is where |1 + x| = |1 - 2x|. That might be interesting to look into.
(Possible more rigorous and explicit math to follow when I can focus on it more)
I let L(B|E) be uniform from x-s/2 to x+s/2 and got that P(B|E) =
where A is the odds if L(B|E)=x.
In the limit as s goes to infinity, it looks like the interesting pieces are a term that's the log of the prior probability dropping off as s grows linearly, plus a term that eventually looks like (1/s)*ln(e^(s/2))=1/2 which means we approach 1/2.