CCC comments on Confidence levels inside and outside an argument - Less Wrong

129 Post author: Yvain 16 December 2010 03:06AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (174)

You are viewing a single comment's thread. Show more comments above.

Comment author: Wes_W 29 December 2014 04:41:03AM *  0 points [-]

The easy-but-not-very-rigorous method is to use the principle of indifference, since there's no particular reason a tie +/-1 should be much less likely than any other result.

If the election is balanced (the mean of the distribution is a tie), and the distribution looks anything like normal or binomial, 1/X is an underestimate of P(tie | election is within vote margin of X), since a tie is actually the most likely result. A tie +/- 1 is right next to the peak of the curve, so it should also be more than 1/X.

The 10^-90 figure cited in the paper was an example of how the calculation is very sensitive to slight imbalances - a 50/50 chance for each voter gave a .00006 chance of tie, while 49.9/50.1 gave the 10^-90. But knowing that an election will be very slightly imbalanced in one direction is a hard epistemic state to get to. Usually we just know something like "it'll be close", which could be modeled as a distribution over possible near-balances. If that distribution is not itself skewed either direction, then we again find that individual results near the mean should be at least 1/X.

Comment author: CCC 29 December 2014 10:02:37AM 1 point [-]

If the election is balanced (the mean of the distribution is a tie)...

That's an important and non-obvious assumption to make.

a 50/50 chance for each voter gave a .00006 chance of tie, while 49.9/50.1 gave the 10^-90

So, in short, the 10^-90 figure is based on the explicit assumption that the election is not balanced?

That's why the two methods you mention produce such wildy different figures; they base their calculations on different basic assumptions. One can argue back and forth about the validity or lack thereof of a given set of assumptions, of course...

Comment author: Wes_W 29 December 2014 04:49:37PM 1 point [-]

That's an important and non-obvious assumption to make.

Yes, I agree.

I'm much more sympathetic to the 10^-90 estimate in the paper than Gelman's quote is; I think he misrepresents the authors in claiming they asserted that probability, when actually they offered it as a conditional (if you model it this way, then it's 10^-90).

Comment author: gwern 29 December 2014 04:37:06PM 0 points [-]

One can argue back and forth about the validity or lack thereof of a given set of assumptions, of course...

That is why I posted it as a comment on this particular post, after all. It's clear that our subjective probability of casting a tie-breaking vote is going to be far less extreme than 10^-90 because our belief in the binomial idealization being correct puts a much less extreme bound on the tie-breaking vote probability than just taking 10^-90 at face value.