Summary: You want to find the truth? You want to win? You're gonna have to learn the right way to vote.
That seems to be a wrong summary.
The post talks about a variety of voting systems and their theoretical underpinnings, pointing out the failure modes of each and how do attempts to fix them turn out.
Significant changes in the real-life voting system in the US look very low-probability to me (barring a revolution or something equally cataclysmic). Thus the post doesn't tell me how to find the truth, doesn't tell me how to win (in real life) and doesn't tell me how to vote (thank goodness).
This is still in-progress, and I'm going to get to some of that later. Here's my defense of the current summary:
- First, it's just a summary. If it could include all the subtleties of the article, I wouldn't need to write the article.
- Second, even if the public voting systems (muni, state, and national) wherever you happen to live continue to be stupid ones, understanding voting systems better is useful knowledge. You should understand bad voting systems if they affect you, and good voting systems if you're in organizations that could use them.
- Third, I don't agree that changing voting systems is a negligible priority. For instance: various cities nationwide, including most of the SF bay area, use IRV for city elections (though this isn't actually the best system, it is certainly a change from 15 years ago.) A number of states (at least 10 to my knowledge) have revamped their primary systems in this time. An approval voting initiative for primaries is currently in the signature stage in Oregon, and legislative study commissions of approval voting are underway in Rhode Island and Arizona, with Colorado considering one. States representing 136 electoral votes have signed the National Popular Vote interstate compact, which is about halfway to it taking effect. Obviously, these various facts affect a small minority of Americans, but that small minority is still millions of people. So I'd estimate that a nationwide change (accomplished at the state-by-state level and NOT through a constitutional amendment) is an outside chance but not a negligible one.
As to telling you how to find truth, how to win, and how to vote: obviously the goal here is not to tell you which way to vote, but to help deepen your understanding of the utility of voting mechanisms, both at the public scale and in private contexts.
....
On the other hand, I understand that you're telling me that this sounds grating to you, like overblown rhetoric. I'll see what I can do to improve that while keeping it succinct and intriguing. So thank you.
I worry that this sort of analysis puts process ahead of results.
In large-scale decisionmaking, such as regional or national politics, most voters are confused and inattentive. I think this is inevitable and even proper. The world is too complicated for most people to have informed and thoughtful opinions on most topics. As a result, I don't particularly care if the process delivers results most voters want. Instead, I care if the process delivers decent results. And in particular, I want decent results for impatient voters and potentially-dishonest election apparatus. First-past-the-post has the important benefit that as a voter I have to indicate one preference, rather than an ordering. This requires strictly less input from me, and therefore probably less attention and thought, which is a Good Thing.
I would be interested to hear an argument for why all the voting theory stuff is useful in practice, given the constraints and goals of practical politics.
We have some examples of cities and countries that use systems other than first-past-the-post. Which of these actually are better governed as a result?
As a result, I don't particularly care if the process delivers results most voters want. Instead, I care if the process delivers decent results.
You seem to want some kind of a hidden technocracy where what the (ignorant and confused) voters say they want doesn't matter much.
This requires strictly less input from me, and therefore probably less attention and thought, which is a Good Thing.
Which quickly goes to requiring zero input from you.
Most people wouldn't call the result a democracy.
You seem to want some kind of a hidden technocracy where what the (ignorant and confused) voters say they want doesn't matter much.
This is how most of the First World is run. I think Belgium's 589 days without an elected government, during which Belgium was not much misgoverned at all, provide a nice natural experiment demonstrating this. To quote the blogger Foseti:
The odd thing about this period of no government was that Belgium’s government was very busy. For example, the non-government nationalized one of the country’s largest banks. Less importantly, your humble blogger had numerous meetings that were attended by representatives of Belgium’s non-government.
It isn't great but I expect it outperforms popular input. In the third world in particular, experiments with having popular input into actual governance seem to end badly, though I am the first to admit there are important confounding variables.
When it comes to managing cities, the more technocratic the city is the more prosperous it tends to be, now of course correlation isn't causation, but overall I'd rather live in a prosperous city than a non-technocratic one, and if something can scale for cities with millions of inhabitants, why not small countries?
Most people wouldn't call the result a democracy.
Except most people do. And the case can be made it is a democracy in the same sense say Sweden is a monarchy.
This is how most of the First World is run.
Kinda but not really. The mandarins / professional bureaucracy aren't a technocracy to start with and the voters aren't quite that powerless. But that's a fairly big discussion, probably for another thread.
if something can scale for cities with millions of inhabitants, why not small countries?
A large subset of political ideas/solutions/proposals suffer precisely from the problem that they scale badly. For example, democracy.
the case can be made it is a democracy in the same sense say Sweden is a monarchy.
Which is not an interesting or meaningful sense at all.
You seem to want some kind of a hidden technocracy where what the (ignorant and confused) voters say they want doesn't matter much.
I want the voters to have neither too much nor too little influence. I don't know how to characterize that amount other than by the results, unfortunately.
Which quickly goes to requiring zero input from you.
This is not the historical experience. Britain has had elections that matter, going back many centuries. In their system, the general election ballot is quite simple: "which of party's candidate do you like?" In contrast, a US general election ballot can have several pages of officials, referenda, and so forth. In California, it's routine to have 30 separate things to vote on every two years, at just the state and federal levels.
My sense is that Britain is better governed and has a more stable political system, I think in part because the voting avoids over-burdening the electorate. Simple elections don't seem to degenerate to dictatorship, and more complicated systems don't produce results I like better.
Wait until I get to explaining SODA; a voting system where you can vote for one and still get better results.
As for comparing different societies: there are of course societies with different electoral systems, and I think some systems do tend to lead to better governance than in the US/UK, but the evidence is weak and VERY confounded. It's certainly impossible to clearly demonstrate a causal effect; and would be, even assuming such an effect existed and were sizeable. I will talk about this more as I finish this post.
I could just call you a nasty, evil person, but instead I'm going to argue against you on more "rational" grounds.
One of the most important facets of democracy for good decision-making is the feedback loop between electors and elected, and thus between decision-makers and those affected by decisions. Voting is an informational signal from the ruled to the rulers, and legislation is an informational signal from the rulers to the ruled.
Now, what do we know about standard-issue humans trying to make decisions under partial certainty or great uncertainty regarding time-varying and complex pleasure/pain signals to optimize a large system for the expression of a broad system of values (and thus available good/evil stimuli)?
If you answered "they can't really do it well", I disagree. My position is: they need swift, sharp feedback in order to do it well. Humans are actually remarkably good at decision-making when given swift, sharp feedback signals.
So what's the problem with an undemocratic democracy, or a technocracy as such? It shatters the feedback loop that's signalling:
To the voters: how well their expressed policy preferences effect their expressed values, and how well their expressed values in the voting booth match their actual values as they experience themselves on an everyday basis.
To the government apparatus: what the masses prefer in public policy, how well the government is effecting those preferences, what impact actual enacted policies have on the voting masses, and the stability of the collective whole of society.
No feedback loop means the system as a whole can and will eventual spiral out of control from sheer divergence, as the people in power no longer know or care about the experience of the masses, leaving the masses with no reason to support the system itself, setting the stage for either a mass exodus or a revolt and thus destabilization of the State.
If you answered "they can't really do it well", I disagree. My position is: they need swift, sharp feedback in order to do it well. Humans are actually remarkably good at decision-making when given swift, sharp feedback signals.
I agree with this. The place where I don't follow your analysis is the assumption that voting is the primary means of feedback. We only hold elections every few years. If you want "swift" feedback, elections are not the tool for you.
Moreover, they don't give very precise feedback in any real democracy. In, say, the UK, the election sends basically two bits of information per voter -- LibDem, Tory, Labor, or other. Even in the US, the ballot just doesn't encode a lot of information about voters; it's single or double-digit bits per year. And each additional bit per voter is hugely expensive to collect. It costs millions of dollars to pay for all the analysis, campaigning, advocacy and so forth that goes into a national or regional election.
Happily, we don't rely on elections as our primary mechanism for sending signals to the government. Functioning democracies have an amazingly rich set of mechanisms for getting prompt feedback from the public. We have opinion surveys. We have newspaper editorial pages. We have constituent letters and phone calls. We have donations. We have private meetings between government officials and lobbyists. We also have the whole panoply of economic indicators -- politicians do notice when the stock market crashes or when bond rates spike.
I think trying to optimize the information content of the election system is missing the point. Elections aren't the right tool for preference elicitation. Elections are there to give force to voter preference -- they're not only a measurement tool or even primarily a measurement tool. They're a civic ritual to demonstrate the presence or lack of voter support for the government and they're a system to designate a specific alternative in the event that the incumbents are voted out. it's important that the election produces a clear outcome and an outcome with enough political support to sustain itself until the next election. It's not important that it extracts the maximum information about the voter preferences on election day.
In conclusion: If you want swift and detailed feedback from the public, run a telephone poll. Rounding up millions of people and asking them about their preferences every few years is a lousy way to measure sentiment.
I agree with this. The place where I don't follow your analysis is the assumption that voting is the primary means of feedback. We only hold elections every few years. If you want "swift" feedback, elections are not the tool for you.
Moreover, they don't give very precise feedback in any real democracy. In, say, the UK, the election sends basically two bits of information per voter -- LibDem, Tory, Labor, or other. Even in the US, the ballot just doesn't encode a lot of information about voters; it's single or double-digit bits per year. And each additional bit per voter is hugely expensive to collect. It costs millions of dollars to pay for all the analysis, campaigning, advocacy and so forth that goes into a national or regional election.
Well yes. This is a very good criticism of our current democracies.
This is an exceptionally good article, one which deserves many more upvotes than I could give, and I eagerly anticipate it's ascension to the main page.
That said, would you be interested in analysis of other types of non-democratic voting systems; i.e. where there is no assumption that all votes are equal? Prediction markets like Hanson's Futarchy proposal would seem to dodge several of the problems here by virtue of allowing the best voters to gradually gain increasing voting power while punishing those gaming the system and speculating, not to mention the income-IQ correlation would increase decision-making power on it's own. An indirect voting system based on the joint stock company such as in Moldbug's Neocameralism would also be interesting to look at, especially since the game theory of corporate governance is a well-examined field. It would also be interesting to see the existing options compared against a rationally self-interested dictator; if nothing else, it gives us a good baseline to compare against as to whether the outcomes are actually worth implementing a voting system to begin with.
Again, incredible article and I can't wait to watch it develop.
Understanding voting processes a little bit can help you navigate that road; understanding them deeply opens the possibility of improving that road and thus winning more often.
The post might do more to reduce people's understanding of voting processes than help increase it. Democratic elections are complex system in which communication matters. You can model democratic elections effectively ignoring the most important part of democracy.
Democracy is about civic debate and people vote as a reflection of that debate.
When it comes to elections in organisations one of the most important lessons is to count voters before pushing any proposal. Speak with people and form alliance before you vote.
Arrow's theorem considers your options holding others fixed - and does its analysis knowing them. But when you're actually filling out your ballot, you don't have access to that kind of information. So it doesn't prove that there aren't systems where the risk/reward from going strategic is poor under more realistic conditions.
Is there such a theorem?
This is a key question. The general answer is:
For realistic cases, there is no such theorem, and so the task of choosing a good system is a lot about choosing one which doesn't reward strategy in realistic cases.
Roughly speaking, my educated intuition is that strategic payoffs grow insofar as you know that the distinctions you care about are orthogonal to what the average/modal/median voter cares about. So insofar as you are average/modal/median, your strategic incentive should be low; which is a way of saying that a good voting system can have low strategy for most voters in most elections.
2a. It may be possible to make this intuition rigorous, and prove that no system can make strategy non-viable for the orthogonal-preferenced voter. However, that would involve a lot of statistics and random variables.... I guess that's what I'm learning in my PhD so eventually I may be up to taking on this proof.
- The exception, the realistic case where there are a number of voters who have an interest that's orthogonal to the average voter, is a case called the chicken dilemma, which I'll talk about a lot more in section 6. Chicken strategy is by far the trickiest realistic strategy to design away.
I would work on making the existing writing clearer before expanding further. There were a number of points where I did not follow you as a casual reader, like with the "circular money pump" and Sen, or things just appeared random, like going from has specific theoretical pitfall to BAD.
Second, if you are going to expand, you may want to address whether all this theory actually predicts much accurately. Beyond saying, yeah, there were these guys Duncan and Black who looked at specific cases--what were those cases?
A couple of years ago, I tried imagining democracy from the point of view of some oligarchs at the top of their society - and, oddly enough, noticed that there seems to be a path by which a hierarchical, authoritarian society might decide to implement something resembling democratic reforms. Certain preconditions seem to be necessary:
- The oligarchs need to be divided into competing factions, each struggling for dominance. (Nearly a certainty.)
- The people need to be able to make a difference in the dominance struggle. (Eg, by being well-armed - longbows, crossbows, and rifles seem likely candidates)
- The oligarchs need to be smart enough to avoid /unnecessary/ fights, to conserve their strength for the necessary ones.
Which leads to:
- Some bright boy coming up with the idea of counting up the popular support for the oligarchical factions, so their input into the struggles can be figured out without having to go to all the trouble and muss of actually fighting.
I'm not a professional historian, so I'm not sure how well this can be argued to map to historical developments of democracies. But, at the least, it seems to be within the general bounds of plausibility, and doesn't make it into classroom civics classes, so it provides an alternate perspective to look at the whole shebang from, so if nothing else, it seems to be a useful mental tool, even if it's not an accurate description.
The idea that oligarchy naturally evolves (declines?) into democracy is an old one, famously expressed in Plato's Republic (still a good read, in my opinion).
I'm not sure about 'naturally'; it seems more likely that the various factors which /could/ combine to spur democracy will naturally wax and wane in different patterns, occasionally all lining up at the same time and providing would-be democrats an opportunity.
Then again, I could be wrong. Anyone have some decent references comparing how democracies have historically started off?
Traditionally, the birth of democracy is attributed to Ancient Greece. I don't have references handy but I'm sure it's very googleable.
The original idea for it may come from there, yes; but why did that idea get implemented at any particular time, instead of earlier or later?
Oh, boy. That's a topic about which there is little consensus, if any. You're basically asking for a comprehensive theory which explains how and why various political structures arose and fell in the course of history. That's a very contentious field filled with mindkilling mines...
General comments: I discovered rangevoting.org years ago and have been a fan since. But it's not quite clear to me what the next action is, here. Yes, score voting is superior to plurality, but is it superior enough that people should join CRV? I use score voting to determine what restaurant to go to or what board game to play if there's any collective uncertainty, and I can see many applications for rationalist group houses or other organizations, but the statement "I would be happier if the US Presidential Election used score voting" misunderstands the nature of politics.
many less-wrong'ers
I would use "many LWers".
Arrows theorem shows that any voting system which can consistently give the same winner (or, in ties, winners) for the same voter preferences; which does not make one voter the effective dictator; which is sure to elect a candidate if all voters prefer them; and which will switch the results for two candidates if you switch their names on all the votes... must exhibit, in at least some situation, the pathology that befell the Rationalist Marching Band above, or in other words, must fail "independence of irrelevant alternatives".
"Arrow's". Later you use Arrows', which is also incorrect, and so I would do a search-replace on the whole thing.
[EDIT- I was wrong about Arrow's theorem. You can safely ignore the following thread, unless you had the same confusion I did, in which case I'll leave it as an example.]
Also, I never found this theorem that troublesome- the "dictator" axiom is necessary for the theorem but I find it untroubling. (The proof suggests that the "dictator" is not an individual selected before the voting process, but a 'pivotal voter'- the last person to move from coalition A to coalition B. But if you never allow a changed individual vote to change the group vote, then how can the group vote ever change?) It also doesn't apply to cardinal voting systems, just ordinal voting systems, so range voting is immune. (Which I see you bring up later.)
In one sense, score voting reducing to approval OK.
I think this sentence is missing an 'is'. (It's also much easier to proofread things in Google Docs, or similar software, than discussion posts.)
But in another sense, it's a problem. If one side of an issue is more inclined to be strategic than the other side, the more-strategic faction could win even if it's a minority. That clashes with many people's ideals of democracy; and worse, it encourages mind-killing political attitudes, where arguments are used as soldiers rather than as ways to seek the truth.
I think this is a feature, personally, since it allows you to encode how much you care about a particular issue. There are lots of ways to implement tradeoffs here, which can lead to overall more efficient decisions.
I'll certainly have more content that addresses these questions as the post develops. For now, I'll simply respond to your misunderstanding about Arrow. The problem is not that there will always be an a posteori pivotal voter, but that (to satisfy the other criteria) there must be an a priori dictator. In other words, you would get the same election result by literally throwing away all ballots but one without ever looking at them. This is clearly not democracy.
This is kind of late and not exactly on point, but I wanted to add in two points about Arrow's Theorem that I hope will help clarify things somewhat.
There is a generalization of Arrow's Theorem to the case where the set of voters may be infinite. In this more general case, the conclusion of the theorem (where here I'm considering the premises to be IIA and unanimity) is not that there must be a dictator, but rather that the voting system is given by an ultrafilter on the set of voters. That is to say, there exists an ultrafilter U on the set of voters such that the candidate elected is precisely the candidate such that the set of people voting for that candidate is large with respect to U. The fact that if there are only finitely many voters there must be a dictator then follows as any ultrafilter on a finite set is principal.
In a nephew/niece comment, homunq mentions sortition (select a voter at random to be dictator) as an example. I don't think this is correct? That is to say, Arrow's Theorem only applies to deterministic voting systems, and "dictatorship" refers to having a pre-specified dictator. There may be a generalization to nondeterministic systems for all I know, but if so, I don't think it's normally considered part of the theorem per se, and I don't think sortition is normally considered an example of dictatorship.
For now, I'll simply respond to your misunderstanding about Arrow.
It's not clear to me why you think that's a misunderstanding; the statement of the theorem is not that the dictator is an a priori dictator, just that there never exists a situation where an individual can completely determine society's preferences. The proof is a construction of a situation given the first two fairness axioms and at least three alternatives, where one voter will be a pivotal voter who can completely determine society's preferences.
But if you don't care about the third axiom, you don't care about the proof. Okay, in a deeply divided but balanced situation, the one non-partisan can pick whether we go left, right, or to the middle; this isn't a huge tragedy.
(The collapse of the scale of preferences is a huge tragedy.)
Again, you're simply not understanding the theorem. If a system fails non-dictatorship, that really does mean that there is an a priori dictator. That could be that one vote is chosen by lot after the ballots are in, or it could be that everybody (or just some special group or person) knows beforehand that Mary's vote will decide it. But it's not that Mary just happens to turn out to be the pivotal voter between a sea of red on one side and blue on the other.
I realize that this is counterintuitive. Do you think I have to be clearer about it in the post?
one vote is chosen by lot
I would say that Arrow simply excludes that possibility. As you said, he only considers systems that "consistently give the same winner...for the same voter preferences."
Nothing wrong with your analysis, but I think it's an important disclaimer.
I realize that this is counterintuitive. Do you think I have to be clearer about it in the post?
Yes, please.
That could be that one vote is chosen by lot after the ballots are in
This is the case that doesn't sound like an a-priori dictator to me, because you don't know who the dictator will be, and thus can't do anything to manipulate the outcome by dint of there being a dictator.
Under Arrow's terms, this still counts as a dictator, as long as the other ballots have no effect. (Not "no net effect", but no effect at all.)
In other words: if I voted for myself, and everyone else voted for Kanye, and my ballot happened to get chosen, then I would win, despite being 1 vote against 100 million.
It may not be the traditional definition of dictatorship, but it sure ain't democracy.
By an a priori dictatorship, I mean there is some individual 1 such that =R_1\%20\forall\%20(R_2,\ldots,R_N)\in%20L(A)%5E{N-1}).
By an a posteriori dictatorship, I mean there is some individual 1 such that \in%20L(A)%5EN\%20s.t.\%20F(R_1,\ldots,R_N)=R_1\%20\forall\%20R_1)
There is obviously not an a priori dictationship for all voting environments under all aggregation rules that satisfy unanimity and IIA. For example, if 9 people prefer A>B>C, and 1 person prefers B>C>A, then society prefers A, regardless of how any specific individual changes their vote (so long as only one vote is changed).
(Note the counterfactual component of my statement- there needs to be an individual who can change the social preference function, not just identify the social preference function.)
But it's not that Mary just happens to turn out to be the pivotal voter between a sea of red on one side and blue on the other.
Every proof of the theorem that I can see operates exactly this way; I'm still not seeing what specific step you think I misunderstand.
I'm sorry, you really are wrong here. You can't make up just one scenario and its result and say that you have a voting rule; a rule must give results for all possible scenarios. And once you do, you'll realize that the only ones which pass both unanimity and IIA are the ones with an a priori dictatorship. I'm not going to rewrite Arrow's whole paper here but that's really what he proved.
You can't make up just one scenario and its result and say that you have a voting rule; a rule must give results for all possible scenarios.
I think I see how the grandparent was confusing. I was assuming that the voting rule was something like plurality voting, with enough sophistication to make it a well-defined rule.
What I meant to do was define two dictatorship criteria which differ from Arrow's, which apply to individuals under voting rules, rather than applying to rules. Plurality voting (with a bit more sophistication) is a voting rule. Bob choosing for everyone is a voting rule. But the rule where Bob chooses for everyone has an a priori dictator- Bob. (He's also an a posteriori dictator, which isn't surprising.)
Plurality voting as a voting rule does not empower an a priori dictator as I defined that in the grandparent. But it is possible to find a situation under plurality voting where an a posteriori dictator exists; that is, we cannot say that plurality voting is free from a posteriori dictators. That is what the nondictatorship criterion (which is applied to voting rules!) means- for a rule to satisfy nondictatorship, it must be impossible to construct a situation where that voting rule empowers an a posteriori dictator.
Because unanimity and IIA imply not nondictatorship, for any election which satisfies unanimity and IIA, you can carefully select a ballot and report just that ballot as the group preferences. But that's because it's impossible for the group to prefer A>B>C with no individual member preferring A>B>C, and so there is guaranteed to be an individual who mirrors the group, not an individual who determines the group. Since individuals determining group preferences is what is politically dangerous, I'm not worried about the 'nondictatorship' criterion, because I'm not worried about mirroring.
I'm not going to rewrite Arrow's whole paper here but that's really what he proved.
I've read it; I've read Yu's proof; I've read Barbera's proof, I've read Geanakoplos's proof, I've read Hansen's proof. (Hansen's proof does follow a different strategy from the one I discussed, but I came across it after writing the grandparent.) I'm moderately confident I know what the theorem means. I'm almost certain that our disagreement stems from different uses of the phrase "a priori dictator," and so hope that disagreement will disappear quickly.
Funnily enough, I asked Amartya Sen and Eric Maskin about this earlier today, in addition to a similarly extensive reading list. You definitely have your quantifiers backwards.
You claim that Arrow's dictatorship criterion is "for all preferences, there exists some k such that the social preference is voter k's preference". In fact, the criterion is the stronger statement that "there exists some k such that for all preferences, the social preference is voter k's preference".
If you read the proof on Wikipedia carefully, you'll notice that it proves that:
Pivotal Voter dictates society's decision for B over C. That is, we show that no matter how the rest of society votes, if Pivotal Voter ranks B over C, then that is the societal outcome.
(Note the order of the quantifiers). They then show
There can be at most one dictator.
Funnily enough, I asked Amartya Sen and Eric Maskin about this earlier today,
I'm hella envious.
And also convinced. If Sen says Arrow meant non-dictatorship in the strong sense, I'm pretty much willing to take Sen's word for it.
I, too, hope that our disagreement will soon disappear. But as far as I can see, it's clearly not a semantic disagreement; one of us is just wrong. I'd say it's you.
So. Say there are 3 voters, and without loss of generality, voter 1 prefers A>B>C. Now, for every one of the 21 distinct combinations for the other two, you have to write down who wins, and I will find either an (a priori, determinative; not mirror) dictator or a non-IIA scenario.
ABC ABC: A
ABC ACB: A
ABC BAC: ?... you fill in these here
ABC BCA: ?
ABC CAB: .
ABC CBA: .
ACB ACB: .
ACB BAC:
ACB BCA:
ACB CAB:
ACB CBA:
BAC BAC:
BAC BCA:
BAC CAB:
BAC CBA:
BCA BCA:
BCA CAB: .... this one's really the key, but please fill in the rest too.
BCA CBA:
CAB CAB:
CAB CBA:
CBA CBA:
Once you've copied these to your comment I will delete my copies.
I, too, hope that our disagreement will soon disappear. But as far as I can see, it's clearly not a semantic disagreement; one of us is just wrong. I'd say it's you.
Thanks; I see it now. Editing my earlier posts.
I'm moderately confident I know what the theorem means. I'm almost certain that our disagreement stems from different uses of the phrase "a priori dictator," and so hope that disagreement will disappear quickly.
I'm glad you've opened a discussion about this, BTW, even if it turns out you are wrong. I wondered myself what exactly the non-dictatorship criterion meant when I started reading the commentary on "Arrow's Paradox" in Ken Binmore's Playing Fair. After I read all of it I was fairly sure non-dictatorship referred to what you call an a priori dictator, but couldn't be totally sure because I didn't have the patience to sit down and puzzle through it systematically.
I'm glad you've opened a discussion about this, BTW, even if it turns out you are wrong.
I was wrong. Hansen's 2002 proof was much, much easier for me to understand than the other ones, but I was reluctant to generalize it past 2 voters until I wrestled with homunq's example. Even after seeing it in the 3 voter case, I had to resist an impulse that said "but surely there's an n such that an n voter situation works!"
No words for, against or mentioning the works of Sir Karl Popper seems an oversight. He wrote at length on democracy and rationality. Tl;dr - democracy is a peacable way to get the wrong leaders out and perhaps move to a slightly better society while having no power to get the right leaders in. Science is a contentious way to get false ideas out and perhaps move to a slightly more accurate world view while having no power to get the right ideas in. In both politics and science, question convention.
Lengthier piece on the compatability of Popper's falsification and Less Wrong Bayesian: do.
i think most people here see falsificationism as a special, more specific case of bayesianism - in the same sense that scientific evidence is a proper subset of rational evidence.
Thanks, I'll try to work him in. (I assume your last sentence is directed to yourself, though.)
Note: This is a draft; so far, about the first half is complete. I'm posting it to Discussion for now; when it's finished, I'll move it to Main. In the mean time, I'd appreciate comments, including suggestions on style and/or format. In particular, if you think I should(n't) try to post this as a sequence of separate sections, let me know.
Summary: You want to find the truth? You want to win? You're gonna have to learn the right way to vote. Plurality voting sucks; better voting systems are built from the blocks of approval, medians (Bucklin cutoffs), delegation, and pairwise opposition. I'm working to promote these systems and I want your help.
Contents: 1. Overblown¹ rhetorical setup ... 2. Condorcet's ideals and Arrow's problem ... 3. Further issues for politics ... 4. Rating versus ranking; a solution? ... 5. Delegation and SODA ... 6. Criteria and pathologies ... 7. Representation, Proportional representation, and Sortition ... 8. What I'm doing about it and what you can ... 9. Conclusions and future directions ... 10. Appendix: voting systems table ... 11. Footnotes
1.
This is a website focused on becoming more rational. But that can't just mean getting a black belt in individual epistemic rationality. In a situation where you're not the one making the decision, that black belt is just a recipe for frustration.
Of course, there's also plenty of content here about how to interact rationally; how to argue for truth, including both hacking yourself to give in when you're wrong and hacking others to give in when they are. You can learn plenty here about Aumann's Agreement Theorem on how two rational Bayesians should never knowingly disagree.
But "two rational Bayesians" isn't a whole lot better as a model for society than "one rational Bayesian". Aspiring to be rational is well and good, but the Socratic ideal of a world tied together by two-person dialogue alone is as unrealistic as the sociopath's ideal of a world where their own voice rules alone. Society needs structures for more than two people to interact. And just as we need techniques for checking irrationality in one- and two-person contexts, we need them, perhaps all the more, in multi-person contexts.
Most of the basic individual and dialogical rationality techniques carry over. Things like noticing when you are confused, or making your opponent's arguments into a steel man, are still perfectly applicable. But there's also a new set of issues when n>2: the issues of democracy and voting. For a group of aspiring rationalists to come to a working consensus, of course they need to begin by evaluating and discussing the evidence, but eventually it will be time to cut off the discussion and just vote. When they do so, they should understand the strengths and pitfalls of voting in general and of their chosen voting method in particular.
And voting's not just useful for an aspiring rationalist community. As it happens, it's an important part of how governments are run. Discussing politics may be a mind-killer in many contexts, but there are an awful lot of domains where politics is a part of the road to winning.² Understanding voting processes a little bit can help you navigate that road; understanding them deeply opens the possibility of improving that road and thus winning more often.
2. Collective rationality: Condorcet's ideals and Arrow's problem
Imagine it's 1785, and you're a member of the French Academy of Sciences. You're rubbing elbows with most of the giants of science and mathematics of your day: Coulomb, Fourier, Lalande, Lagrange, Laplace, Lavoisier, Monge; even the odd foreign notable like Franklin with his ideas to unify electrostatics and electric flow.
One day, they'll put your names in front of lots of cameras (even though that foreign yokel Franklin will be in more pictures)
And this academy, with many of the smartest people in the world, has votes on stuff. Who will be our next president; who should edit and schedule our publications; etc. You're sure that if you all could just find the right way to do the voting, you'd get the right answer. In fact, you can easily prove that, or something like it: if a group is deciding between one right and one wrong option, and each member is independently more than 50% likely to get it right, then as the group size grows the chance of a majority vote choosing the right option goes to 1.
But somehow, there's still annoying politics getting in the way. Some people seem to win the elections simply because everyone expects them to win. So last year, the academy decided on a new election system to use, proposed by your rival, Charles de Borda, in which candidates get different points for being a voters first, second, or third choice, and the one with the most points wins. But you're convinced that this new system will lead to the opposite problem: people who win the election precisely because nobody expected them to win, by getting the points that voters strategically don't want to give to a strong rival. But when people point that possibility out to Borda, he only huffs that "my system is meant for honest men!"
So with your proof of the above intuitive, useful result about two-way elections, you try to figure out how to reduce an n-way election to the two-candidate case. Clearly, you can show that Borda's system will frequently give the wrong results from that perspective. But frustratingly, you find that there could sometimes be no right answer; that there will be no candidate who would beat all the others in one-on-one races. A crack has opened up; could it be that the collective decisions of intelligent individual rational agents could be irrational?
Of course, the "you" in this story is the Marquis de Condorcet, and the year 1785 is when he published his Essai sur l’application de l’analyse à la probabilité des décisions rendues à la pluralité des voix, a work devoted to the question of how to acheive collective rationality. The theorem referenced above is Condorcet's Jury Theorem, which seems to offer hope that democracy can point the way from individually-imperfect rationality towards an ever-more-perfect collective rationality. Just as Aumann's Agreement Theorem shows that two rational agents should always move towards consensus, the Condorcet Jury Theorem apparently shows that if you have enough rational agents, the resulting consensus will be correct.
But as I said, Condorcet also opened a crack in that hope: the possibility that collective preferences will be cyclical. If the assumptions of the jury theorem don't hold — if each voter doesn't have a >50% chance of being right on a randomly-selected question, OR if the correctness of two randomly-selected voters is not independent and uncorrelated — then individually-sensible choices can lead to collectively-ridiculous ones.
What do I mean by "collectively-ridiculous"? Let's imagine that the Rationalist Marching Band is choosing the colors for their summer, winter, and spring uniforms, and that they all agree that the only goal is to have as much as possible of the best possible colors. The summer-style uniforms come in red or blue, and they vote and pick blue; the winter-style ones come in blue or green, and they pick green; and the spring ones come in green or red, and they pick red.
Obviously, this makes us doubt their collective rationality. If, as they all agree they should, they had a consistent favorite color, they should have chosen that color both times that it was available, rather than choosing three different colors in the three cases. Theoretically, the salesperson could use such a fact to pump money out of them; for instance, offering to let them "trade up" their spring uniform from red to blue, then to green, then back to red, charging them a small fee each time; if they voted consistently as above, they would agree to each trade (though of course in reality human voters would probably catch on to the trick pretty soon, so the abstract ideal of an unending circular money pump wouldn't work).
This is the kind of irrationality that Condorcet showed was possible in collective decisionmaking. He also realized that there was a related issue with logical inconsistencies. If you were take a vote on 3 logically related propositions — say, "Should we have a Minister of Silly Walks, to be appointed by the Chancellor of the Excalibur", "Should we have a Minister of Silly Walks, but not appointed by the Chancellor of the Excalibur", and "Should we in fact have a Minister of Silly Walks at all", where the third cannot be true unless one of the first is — then you could easily get majority votes for inconsistent results — in this case, no, no, and yes, respectively. Obviously, there are many ways to fix the problem in this simple case — probably many less-wrong'ers would suggest some Bayesian tricks related to logical networks and treating votes as evidence⁸ — but it's a tough problem in general even today, especially when the logical relationships can be complex, and Condorcet was quite right to be worried about its implications for collective rationality.³
And that's not the only tough problem he correctly foresaw. Nearly 200 years later and an ocean away, in the 1960s, Kenneth Arrow showed that it was impossible for a preferential voting system to avoid the problem of a "Condorcet cycle" of preferences. Arrows theorem shows that any voting system which can consistently give the same winner (or, in ties, winners) for the same voter preferences; which does not make one voter the effective dictator; which is sure to elect a candidate if all voters prefer them; and which will switch the results for two candidates if you switch their names on all the votes... must exhibit, in at least some situation, the pathology that befell the Rationalist Marching Band above, or in other words, must fail "independence of irrelevant alternatives".
Arrow's theorem is far from obvious a priori, but proof is not hard to understand intuitively using Condorcet's insight. Say that there are three candidates, X, Y, and Z, with roughly equal bases of support; and that they form a Condorcet cycle, because in two-way races, X would beat Y with help from Z supporters, Y would beat Z with help from X supporters, and Z would beat X with help from Y supporters. So whoever wins in the three-way race — say, X — just remove the one who would have lost to them — Y in this case — and that "irrelevant" change will change the winner to be the third — Z in this case.
Summary of above: Collective rationality is harder than individual or two-way rationality. Condorcet saw the problem and tried to solve it, but Arrow saw that Condorcet had been doomed to fail.
3. Further issues for politics
So Condorcet's ideals of better rationality through voting appear to be in ruins. But at least we can hope that voting is a good way to do politics, right?
Not so fast. Arrow's theorem quickly led to further disturbing results. Alan Gibbard (and also Mark Satterthwaite) extended that there is no voting system which doesn't encourage voting strategy. That is, if you view an voting system as a class of games where the finite players and finite available strategies are fixed, no player is effectively a dictator, and the only thing that varies are the payoffs for each player from each outcome, there is no voting system where you can derive your best strategic vote purely by looking "honestly" at your own preferences; there is always the possibility of situations where you have to second-guess what others will do.
Amartya Sen piled on with another depressing extension of Arrows' logic. He showed that there is no possible way of aggregating individual choices into collective choice that satisfies two simple criteria. First, it shouldn't choose pareto-dominated outcomes; if everyone prefers situation XYZ to ABC, that they don't do XYZ. Second, it is "minimally liberal"; that is, there are at least two people who each get to freely make their own decision on at least one specific issue each, no matter what, so for instance I always get to decide between X and A (in Gibbard's⁴ example, colors for my house), and you always get to decide between Y and B (colors for your own house). The problem is that if you nosily care more about my house's color, the decision that should have been mine, and I nosily care about yours, more than we each care about our own, then the pareto-dominant situation is the one where we don't decide our own houses; and that nosiness could, in theory, be the case for any specific choice that, a priori, someone might have labelled as our Inalienable Right. It's not such a surprising result when you think about it that way, but it does clearly show that unswerving ideals of Democracy and Liberty will never truly be compatible.
Meanwhile, "public choice" theorists⁵ like Duncan Black, James Buchanan, etc. were busy undermining the idea of democratic government from another direction: the motivations of the politicians and bureaucrats who are supposed to keep it running. They showed that various incentives, including the strange voting scenarios explored by Condorcet and Arrow, would tend open a gap between the motives of the people and those of the government, and that strategic voting and agenda-setting within a legislature would tend to extend the impact of that gap. Where Gibbard and Sen had proved general results, these theorists worked from specific examples. And in one aspect, at least, their analysis is devastatingly unanswerable: the near-ubiquitous "democratic" system of plurality voting, also known as first-past-the-post or vote-for-one or biggest-minority-wins, is terrible in both theory and practice.
So, by the 1980s, things looked pretty depressing for the theory of democracy. Politics, the theory went, was doomed forever to be a worse than sausage factory; disgusting on the inside and distasteful even from outside.
Should an ethical rationalist just give up on politics, then? Of course not. As long as the results it produces are important, it's worth trying to optimize. And as soon as you take the engineer's attitude of optimizing, instead of dogmatically searching for perfection or uselessly whining about the problems, the results above don't seem nearly as bad.
From this engineer's perspective, public choice theory serves as an unsurprising warning that tradeoffs are necessary, but more usefully, as a map of where those tradeoffs can go particularly wrong. In particular, its clearest lesson, in all-caps bold with a blink tag, that PLURALITY IS BAD, can be seen as a hopeful suggestion that other voting systems may be better. Meanwhile, the logic of both Sen's and Gibbard's theorems are built on Arrow's earlier result. So if we could find a way around Arrow, it might help resolve the whole issue.
Summary of above: Democracy is the worst political system... (...except for all the others?) But perhaps it doesn't have to be quite so bad as it is today.
4. Rating versus ranking
So finding a way around Arrow's theorem could be key to this whole matter. As a mathematical theorem, of course, the logic is bulletproof. But it does make one crucial assumption: that the only inputs to a voting system are rankings, that is, voters' ordinal preference orders for the candidates. No distinctions can be made using ratings or grades; that is, as long as you prefer X to Y to Z, the strength of those preferences can't matter. Whether you put Y almost up near X or way down next to Z, the result must be the same.
Relax that assumption, and it's easy to create a voting system which meets Arrow's criteria. It's called Score voting⁶, and it just means rating each candidate with a number from some fixed interval (abstractly speaking, a real number; but in practice, usually an integer); the scores are added up and the highest total or average wins. (Unless there are missing values, of course, total or average amount to the same thing.) You've probably used it yourself on Yelp, IMDB, or similar sites. And it clearly passes all of Arrow's criteria. Non-dictatorship? Check. Unanimity? Check. Symmetry over switching candidate names? Check. Independence of irrelevant alternatives? In the mathematical sense — that is, as long as the scores for other candidates are unchanged — check.
So score voting is an ideal system? Well, it's certainly a far sight better than plurality. But let's check it against Sen and against Gibbard.
Sen's theorem was based on a logic similar to Arrow. However, while Arrow's theorem deals with broad outcomes like which candidate wins, Sen's deals with finely-grained outcomes like (in the example we discussed) how each separate house should be painted. Extending the cardinal numerical logic of score voting to such finely-grained outcomes, we find we've simply reinvented markets. While markets can be great things and often work well in practice, Sen's result still holds in this case; if everything is on the market, then there is no decision which is always yours to make. But since, in practice, as long as you aren't destitute, you tend to be able to make the decisions you care the most about, Sen's theorem seems to have lost its bite in this context.
What about Gibbard's theorem on strategy? Here, things are not so easy. Yes, Gibbard, like Sen, parallels Arrow. But while Arrow deals with what's written on the ballot, Gibbard deals with what's in the voters head. In particular, if a voter prefers X to Y by even the tiniest margin, Gibbard assumes (not unreasonably) that they may be willing to vote however they need to, if by doing so they can ensure X wins instead of Y. Thus, the internal preferences Gibbard treats are, effectively, just ordinal rankings; and the cardinal trick by which score voting avoided Arrovian problems no longer works.
How does score voting deal with strategic issues in practice? The answer to that has two sides. On the one hand, score never requires voters to be actually dishonest. Unlike the situation in a ranked system such as plurality, where we all know that the strategic vote may be to dishonestly ignore your true favorite and vote for a "lesser evil" among the two frontrunners, in score voting you never need to vote a less-preferred option above a more-preferred option. At worst, all you have to do is exaggerate some distinctions and minimize others, so that you might end giving equal votes to less- and more-preferred options.
Did I say "at worst"? I meant, "almost always". Voting strategy only matters to the result when, aside from your vote, two or more candidates are within one vote of being tied for first. Except in unrealistic, perfectly-balanced conditions, as the number of voters rises, the probability that anyone but the two a priori frontrunner candidates is in on this tie falls to zero.⁷ Thus, in score voting, the optimal strategy is nearly always to vote your preferred frontrunner and all candidate above at the maximum, and your less-preferred frontrunner and all candidates below at the minimum. In other words, strategic score voting is basically equivalent to approval voting, where you give each candidate a 1 or 0 and the highest total wins.
In one sense, score voting reducing to approval OK. Approval voting is not a bad system at all. For instance, if there's a known majority Condorcet winner — a candidate who could beat any other by a majority in a one-on-one race — and voters are strategic — they anticipate the unique strong Nash equilibrium, the situation where no group of voters could improve the outcome for all its members by changing their votes, whenever such a unique equilibrium exists — then the Condorcet winner will win under approval. That's a lot of words to say that approval will get the "democratic" results you'd expect in most cases.
But in another sense, it's a problem. If one side of an issue is more inclined to be strategic than the other side, the more-strategic faction could win even if it's a minority. That clashes with many people's ideals of democracy; and worse, it encourages mind-killing political attitudes, where arguments are used as soldiers rather than as ways to seek the truth.
But score and approval voting are not the only systems which escape Arrow's theorem through the trapdoor of ratings. If score voting, using the average of voter ratings, too-strongly encourages voters to strategically seek extreme ratings, then why not use the median rating instead? We know that medians are less sensitive to outliers than averages. And indeed, median-based systems are more resistant to one-sided strategy than average-based ones, giving better hope for reasonable discussion to prosper. That is to say, in a simple model, a minority would need twice as much strategic coordination under median as under average, in order to overcome a majority; and there's good reason to believe that, because of natural factional separation, reality is even more favorable to median systems than that model.
There are several different median systems available. In the US during the 1910-1925 Progressive Era, early versions collectively called "Bucklin voting" were used briefly in over a dozen cities. These reforms, based on counting all top preferences, then adding lower preferences one level at a time until some candidate(s) reach a majority, were all rolled back soon after, principally by party machines upset at upstart challenges or victories. The possibility of multiple, simultaneous majorities is a principal reason for the variety of Bucklin/Median systems. Modern proposals of median systems include Majority Approval Voting, Majority Judgment, and Graduated Majority Judgment, which would probably give the same winners almost all of the time. An important detail is that most median system ballots use verbal or letter grades rather than numeric scores. This is justifiable because the median is preserved under any monotonic transformation, and studies suggest that it would help discourage strategic voting.
Serious attention to rated systems like approval, score, and median systems barely began in the 1980s, and didn't really pick up until 2000. Meanwhile, the increased amateur interest in voting systems in this period — perhaps partially attributable to the anomalous 2000 US presidential election, or to more-recent anomalies in the UK, Canada, and Australia — has led to new discoveries in ranked systems as well. Though such systems are still clearly subject to Arrow's theorem, new "improved Condorcet" methods which use certain tricks to count a voter's equal preferences between to candidates on either side of the ledger depending on the strategic needs, seem to offer promise that Arrovian pathologies can be kept to a minimum.
With this embarrassment of riches of systems to choose from, how should we evaluate which is best? Well, at least one thing is a clear consensus: plurality is a horrible system. Beyond that, things are more controversial; there are dozens of possible objective criteria one could formulate, and any system's inventor and/or supporters can usually formulate some criterion by which it shines.
Ideally, we'd like to measure the utility of each voting system in the real world. Since that's impossible — it would take not just a statistically-significant sample of large-scale real-world elections for each system, but also some way to measure the true internal utility of a result in situations where voters are inevitably strategically motivated to lie about that utility — we must do the next best thing, and measure it in a computer, with simulated voters whose utilities are assigned measurable values. Unfortunately, that requires assumptions about how those utilities are distributed, how voter turnout is decided, and how and whether voters strategize. At best, those assumptions can be varied, to see if findings are robust.
In 2000, Warren Smith performed such simulations for a number of voting systems. He found that score voting had, very robustly, one of the top expected social utilities (or, as he termed it, lowest Bayesian regret). Close on its heels were a median system and approval voting. Unfortunately, though he explored a wide parameter space in terms of voter utility models and inherent strategic inclination of the voters, his simulations did not include voters who were more inclined to be strategic when strategy was more effective. His strategic assumptions were also unfavorable to ranked systems, and slightly unrealistic in other ways. Still, though certain of his numbers must be taken with a grain of salt, some of his results were large and robust enough to be trusted. For instance, he found that plurality voting and instant runoff voting were clearly inferior to rated systems; and that approval voting, even at its worst, offered over half the benefits compared to plurality of any other system.
Summary of above: Rated systems, such as approval voting, score voting, and Majority Approval Voting, can avoid the problems of Arrow's theorem. Though they are certainly not immune to issues of strategic voting, they are a clear step up from plurality. Starting with this section, the opinions are my own; the two prior sections were based on general expert views on the topic.
5. Delegation and SODA
Rated systems are not the only way to try to beat the problems of Arrow and Gibbard (/Satterthwaite).
Summary of above:
6. Criteria and pathologies
do.
Summary of above:
7. Representation, proportionality, and sortition
do.
Summary of above:
8. What I'm doing about it and what you can
do.
Summary of above:
9. Conclusions and future directions
do.
Summary of above:
10. Appendix: voting systems table
Compliance of selected systems (table)
The following table shows which of the above criteria are met by several single-winner systems. Note: contains some errors; I'll carefully vet this when I'm finished with the writing. Still generally reliable though.
MMC
Majority Condorcet
loser
Participation
symmetry
Resolvable
allowed
prefs
allowed
Later-no-help
favorite
betrayal
arbitrary winner[nb 25]
"Yes/No", in a column which covers two related criteria, signifies that the given system passes the first criterion and not the second one.
11. Footnotes
¹ When I call my introduction "overblown", I mean that I reserve the right to make broad generalizations there, without getting distracted by caveats. If you don't like this style, feel free to skip to section 2.
² Of course, the original "politics is a mind killer" sequence was perfectly clear about this: "Politics is an important domain to which we should individually apply our rationality—but it's a terrible domain in which to learn rationality, or discuss rationality, unless all the discussants are already rational." The focus here is on the first part of that quote, because I think Less Wrong as a whole has moved too far in the direction of avoiding politics as not a domain for rationalists.
³ Bayes developed his theorem decades before Condorcet's Essai, but Condorcet probably didn't know of it, as it wasn't popularized by Laplace until about 30 years later, after Condorcet was dead.
⁴ Yes, this happens to be the same Alan Gibbard from the previous paragraph.
⁵ Confusingly, "public choice" refers to a school of thought, while "social choice" is the name for the broader domain of study. Stop reading this footnote now if you don't want to hear mind-killing partisan identification. "Public choice" theorists are generally seen as politically conservative in the solutions they suggest. It seems to me that the broader "social choice" has avoided taking on a partisan connotation in this sense.
⁶ Score voting is also called "range voting" by some. It is not a particularly new idea — for instance, the "loudest cheer wins" rule of ancient Sparta, and even aspects of honeybees' process for choosing new hives, can be seen as score voting — but it was first analyzed theoretically around 2000. Approval voting, which can be seen as a form of score voting where the scores are restricted to 0 and 1, had entered theory only about two decades earlier, though it too has a history of practical use back to antiquity.
⁷ OK, fine, this is a simplification. As a voter, you have imperfect information about the true level of support and propensity to vote in the superpopulation of eligible voters, so in reality the chances of a decisive tie between other than your two expected frontrunners is non-zero. Still, in most cases, it's utterly negligible.
⁸ This article will focus more on the literature on multi-player strategic voting (competing boundedly-instrumentally-rational agents) than on multi-player Aumann (cooperating boundedly-epistemically-rational agents). If you're interested in the latter, here are some starting points: Scott Aaronson's work is, as far as I know, the state of the art on 2-player Aumann, but its framework assumes that the players have a sophisticated ability to empathize and reason about each others' internal knowledge, and the problems with this that Aaronson plausibly handwaves away in the 2-player case are probably less tractable in the multi-player one. Dalkiran et al deal with an Aumann-like problem over a social network; they find that attempts to "jump ahead" to a final consensus value instead of simply dumbly approaching it asymptotically can lead to failure to converge. And Kanoria et al have perhaps the most interesting result from the perspective of this article; they use the convergence of agents using a naive voting-based algorithm to give a nice upper bound on the difficulty of full Bayesian reasoning itself. None of these papers explicitly considers the problem of coming to consensus on more than one logically-related question at once, though Aaronson's work at least would clearly be easy to extend in that direction, and I think such extensions would be unsurprisingly Bayesian.