gwern comments on Confidence levels inside and outside an argument - Less Wrong

129 Post author: Yvain 16 December 2010 03:06AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (174)
Tags: overconfidence

Comments (174)

You are viewing a single comment's thread.

Comment author: gwern 29 December 2014 12:06:54AM 2 points [-]

Another voting example; "Common sense and statistics", Andrew Gelman:

A paper* was published in a political science journal giving the probability of a tied vote in a presidential election as something like 10^-90**. Talk about innumeracy! The calculation, of course (I say “of course” because if you are a statistician you will likely know what is coming) was based on the binomial distribution with known P. For example, Obama got something like 52% of the vote, so if you take n=130 million and P=0.52 and figure out the probability of an exact tie, you can work out the formula etc etc.

From empirical grounds that 10^-90 thing is ludicrous. You can easily get an order-of-magnitude estimate by looking at the empirical probability, based on recent elections, that the vote margin will be within 2 million votes (say) and then dividing by 2 million to get the probability of it being a tie or one vote from a tie.

The funny thing—and I think this is a case for various bad numbers that get out there—is that this 10^-90 has no intuition behind it, it’s just the product of a mindlessly applied formula (because everyone “knows” that you use the binomial distribution to calculate the probability of k heads in n coin flips). But it’s bad intuition that allows people to accept that number without screaming. A serious political science journal wouldn’t accept a claim that there were 10^90 people in some obscure country, or that some person was 10^90 feet tall.

...To continue with the Gigerenzer idea [of turning probabilities into frequencies], one way to get a grip on the probability of a tied election is to ask a question like, what is the probability that an election is determined by less than 100,000 votes in a decisive state. That’s happened at least once. (In 2000, Gore won Florida by only 20-30,000 votes.***) The probability of an exact tie is of the order of magnitude of 10^(-5) times the probability of an election being decided by less than 100,000 votes...Recent national elections is too small a sample to get a precise estimate, but it’s enough to make it clear that an estimate such as 10^-90 is hopelessly innumerate.

  1. * "Is it Rational to Vote? Five Types of Answer and a Suggestion", Dowding 2005; fulltext: https://pdf.yt/d/5veaHe6F5j-k6oNQ / https://www.dropbox.com/s/fxgfa04hmpfntgh/2005-dowding.pdf / http://libgen.org/scimag/get.php?doi=10.1111%2Fj.1467-856x.2005.00188.x
  2. ** 1/1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 ; or to put it in context, 'inside' the argument, the claim is that you could hold a presidential election for every atom in the universe, and still not ever have a candidate win by one vote

  3. *** From the comments:

    It happened five or six times in 2000 alone, depending on how you think about electoral–college ties. http://en.wikipedia.org/wiki/United_States_presidential_election,_2000#Votes_by_state.

Comment author: Jiro 29 December 2014 03:13:59AM 0 points [-]

What leads you to conclude that the chance of a vote margin of 1 is anywhere near 1/X of the chance of a vote margin of X? That's not obvious, and your quote doesn't try to derive it.

Comment author: Wes_W 29 December 2014 04:41:03AM *  0 points [-]

The easy-but-not-very-rigorous method is to use the principle of indifference, since there's no particular reason a tie +/-1 should be much less likely than any other result.

If the election is balanced (the mean of the distribution is a tie), and the distribution looks anything like normal or binomial, 1/X is an underestimate of P(tie | election is within vote margin of X), since a tie is actually the most likely result. A tie +/- 1 is right next to the peak of the curve, so it should also be more than 1/X.

The 10^-90 figure cited in the paper was an example of how the calculation is very sensitive to slight imbalances - a 50/50 chance for each voter gave a .00006 chance of tie, while 49.9/50.1 gave the 10^-90. But knowing that an election will be very slightly imbalanced in one direction is a hard epistemic state to get to. Usually we just know something like "it'll be close", which could be modeled as a distribution over possible near-balances. If that distribution is not itself skewed either direction, then we again find that individual results near the mean should be at least 1/X.

Comment author: Jacobian 30 December 2014 05:55:58AM *  2 points [-]

I recently wrote about why voting is a terrible idea and fell into the same error as Gelman (I assumed 49.9-50.1 a priori is conservative). Wes and gwern, thanks for correcting me! In fact, due to the Median Voter Theorem and with better and better polling and analysis we may assume that the distribution of voter distributions should have a peak at 50-50.

Of course, there are other great reasons not to vote (mainly to avoid "enlisting in the army" and letting your mind be killed. My suggestion is always to find a friend who is a credible threat to vote for the candidate you despise most and invite him to a beer on election day under the condition that neither of you will vote and you will not talk about politics. Thus, you maintain your friendship while cancelling out the votes. I call it the VAVA (voter anti-voter annihilation) principle.

Comment author: CCC 29 December 2014 10:02:37AM 1 point [-]

If the election is balanced (the mean of the distribution is a tie)...

That's an important and non-obvious assumption to make.

a 50/50 chance for each voter gave a .00006 chance of tie, while 49.9/50.1 gave the 10^-90

So, in short, the 10^-90 figure is based on the explicit assumption that the election is not balanced?

That's why the two methods you mention produce such wildy different figures; they base their calculations on different basic assumptions. One can argue back and forth about the validity or lack thereof of a given set of assumptions, of course...

Comment author: Wes_W 29 December 2014 04:49:37PM 1 point [-]

That's an important and non-obvious assumption to make.

Yes, I agree.

I'm much more sympathetic to the 10^-90 estimate in the paper than Gelman's quote is; I think he misrepresents the authors in claiming they asserted that probability, when actually they offered it as a conditional (if you model it this way, then it's 10^-90).

Comment author: gwern 29 December 2014 04:37:06PM 0 points [-]

One can argue back and forth about the validity or lack thereof of a given set of assumptions, of course...

That is why I posted it as a comment on this particular post, after all. It's clear that our subjective probability of casting a tie-breaking vote is going to be far less extreme than 10^-90 because our belief in the binomial idealization being correct puts a much less extreme bound on the tie-breaking vote probability than just taking 10^-90 at face value.

Powered by Reddit Powered by Reddit