In grade school, I read a series of books titled Sideways Stories from Wayside School by Louis Sachar, who you may know as the author of the novel Holes which was made into a movie in 2003. The series included two books of math problems, Sideways Arithmetic from Wayside School and More Sideways Arithmetic from Wayside School, the latter of which included the following problem (paraphrased):

The students have Mrs. Jewl's class have been given the privilege of voting on the height of the school's new flagpole. She has each of them write down what they think would be the best hight for the flagpole. The votes are distributed as follows:

• 1 student votes for 6 feet.
• 1 student votes for 10 feet.
• 7 students vote for 25 feet.
• 1 student votes for 30 feet.
• 2 students vote for 50 feet.
• 2 students vote for 60 feet.
• 1 student votes for 65 feet.
• 3 students vote for 75 feet.
• 1 student votes for 80 feet, 6 inches.
• 4 students vote for 85 feet.
• 1 student votes for 91 feet.
• 5 students vote for 100 feet.

At first, Mrs. Jewls declares 25 feet the winning answer, but one of the students who voted for 100 feet convinces her there should be a runoff between 25 feet and 100 feet. In the runoff, each student votes for the height closest to their original answer. But after that round of voting, one of the students who voted for 85 feet wants their turn, so 85 feet goes up against the winner of the previous round of voting, and the students vote the same way, with each student voting for the height closest to their original answer. Then the same thing happens again with the 50 foot option. And so on, with each number, again and again, "very much like a game of tether ball."

Question: if this process continues until it settles on an answer that can't be beaten by any other answer, how tall will the new flagpole be?

Answer (rot13'd): fvkgl-svir srrg, orpnhfr gung'f gur zrqvna inyhr bs gur bevtvany frg bs ibgrf. Naq abj lbh xabj gur fgbel bs zl svefg rapbhagre jvgu gur zrqvna ibgre gurberz.

Why am I telling you this? There's a minor reason and a major reason. The minor reason is that this shows it is possible to explain little-known academic concepts, at least certain ones, in a way that grade schoolers will understand. It's a data point that fits nicely with what Eliezer has written about how to explain things. The major reason, though, is that a month ago I finished my systematic read-through of the sequences and while I generally agree that they're awesome (perhaps moreso than most people; I didn't see the problem with the metaethics sequence), I thought the mini-discussion of political parties and voting was on reflection weak and indicative of a broader nerd failure mode.

TLDR (courtesy of lavalamp):

1. Politicians probably conform to the median voter's views.
2. Most voters are not the median, so most people usually dislike the winning politicians.
3. But people dislike the politicians for different reasons.
4. Nerds should avoid giving advice that boils down to "behave optimally". Instead, analyze the reasons for the current failure to behave optimally and give more targeted advice.

Advance warning for heavy US slant, at least in terms of examples, though the theory is applicable everywhere.

The median voter theorem

The median voter theorem was first laid out in a paper by Duncan Black titled "On the Rationale of Group Decision-Making," which imagine's a situation very much like Mrs. Jewls' class voting on the flagpole height: a committee passes a motion by majority vote, and then it considers various motions to amend the original motion, each of which itself needs a simple majority to pass. Each member of the committee has preferences over the range of possible motions, and furthermore:

While a member’s preference curve may be of any shape whatever, there is reason to expect that, in some important practical problems, the valuations actually carried out will tend to take the form of isolated points on single-peaked curves. This would be particularly likely to happen were the committee considering different possible sizes of a numerical quantity and choosing one size in preference to the others. It might be reaching a decision, say, with regard to the price of a product to be marketed by a firm, or the output for a future period, or the wage rate of labor, or the height of a particular tax, or the legal school-leaving age, and so on.

Or, for that matter, the height of a flagpole. Black shows that on his assumptions, the committee will eventually settle on the version of the motion favored by the median committee member.

Again, you may be asking, so what? Most people don't care about understanding the behavior of committees, especially not compared to their passion for national presidential elections. And elections for political office don't use a tether ball-like system of having head-to-head matchup after head-to-head matchup until you've finally found the candidate the median voter wants. There's one election with two (or if you're lucky, three) major candidates and that's it.

The relevance to electoral politics comes in when you allow for the possibility of candidates shaping themselves and their platforms to appeal to the median voter. The candidate who does this should be invincible - at least, until the other candidate does the same thing, at which point the election becomes a closer call. The idea of candidates shaping themselves to voter preferences is key; I started off this post with the flagpole example partly to emphasize that. And there are other assumptions you have to make to get to the conclusion that candidates will actually behave this way.

But before we get in to that, let's compare the median voter picture to the picture Eliezer put forward in the posts linked above:

Forget that Congresspeople on both sides of the "divide" are more likely to be lawyers than truck drivers.  Forget that in training and in daily life, they have far more in common with each other than they do with a randomly selected US citizen from their own party. Forget that they are more likely to hang out at each other's expensive hotel rooms than drop by your own house.  Is there a political divide - a divide of policies and interests - between Professional Politicians on the one hand, and Voters on the other?

Well, let me put it this way.  Suppose that you happen to be socially liberal, fiscally conservative.  Who would you vote for?

Or simplify it further:  Suppose that you're a voter who prefers a smaller, less expensive government - should you vote Republican or Democratic?  Or, lest I be accused of color favoritism, suppose that your voter preference is to get US troops out of Iraq.  Should you vote Democratic or Republican?

One needs to be careful, at this point, to keep track of the distinction between marketing materials and historical records.  I'm not asking which political party stands for the idea of smaller government - which football team has "Go go smaller government!  Go go go!" as one of its cheers.  (Or "Troops out of Iraq!  Yay!")  Rather, over the last several decades, among Republican politicians and Democratic politicians, which group of Professional Politicians shrunk the government while it was in power?

And by "shrunk" I mean "shrunk".  If you're suckered into an angry, shouting fight over whether Your Politicians or Their Politicians grew the government slightly less slowly, it means you're not seeing the divide between Politicians and Voters. There isn't a grand conspiracy to expand the government, but there's an incentive for each individual politician to send pork to campaign contributors, or borrow today against tomorrow's income.  And that creates a divide between the Politicians and the Voters, as a class, for reasons that have nothing to do with colors and slogans.

Eliezer observes that there doesn't seem to be much difference between the two parties, and concludes that they are colluding (albeit probably not by explicit agreement) to advance their own interests at the expense of the voters'. The politicians don't offer the voters any real choice, but get voters to vote for them anyway though misleading party labels and the argument that, if they don't vote for a major-party candidate, they're "throwing their vote away."

However, the observation that there doesn't seem to be much difference between the two parties can also be explained by the hypothesis that politicians are shaping themselves to appeal to the median voter. This fact alone doesn't show that the median voter model is right... but it does show that the mere fact of there not being much difference between the two parties doesn't show the "colluding politicians" model is right either.

So how well does the median voter theorem capture reality? One problem for the model is that it potentially breaks down if the choices don't fit onto a nice, linear spectrum. Suppose, for the sake of a simplified example, that only three people vote in a particular presidential election. Suppose, furthermore, that the three voters have the following set of preferences:

• Alice prefers Obama to Romney, and Romney to Ron Paul
• Bob prefers Romney to Ron Paul, and Ron Paul to Obama
• Carol prefers Ron Paul to Obama, and Obama to Romney

Given this set of voters and their preferences, in an Obama vs. Romney contest, Obama will win; in a Romney vs. Ron Paul contest, Romney will win; but in a Ron Paul vs. Obama contest, Ron Paul will win.

However, the median voter theorem seems to be a pretty good model in practice in spite of such problems. Roger Congleton, in an article in the Encyclopedia of Public Choice, writes:

Although theoretical arguments suggest that the applicability of the median voter model may be very limited, the empirical evidence suggests otherwise. There is a large body of evidence that suggests median voter preferences over policies are (largely) of the sort which can be mapped into a single issue space while retaining "single peakedness" Poole and Daniels (1985) find that 80-90% of all the recorded votes in the US Congress can be explained with a one dimensional policy space. Stratmann (1996) finds little evidence of cycling across Congressional votes over district specific grants.

Moreover, the median voter model has a very good empirical track record in public finance as a model of fiscal policy across states and through time. Recent studies show that the median voter model can explain federal, state, and local spending, as well as international tariff policies. Congleton and Shughart (1990) Congleton and Bennett (1995) suggest that the median voter model provides a better explanation of large scale public programs than comparable interest group models. This is not to suggest that the median voter always exercises the same degree of control over public policy irrespective of political institutions. Holcombe (1980) and Frey (1994) report significant policy difference between representative and direct forms of democracy that would not exist unless significant agency problems exist within representative government. Moreover, statistical tests can never prove that a particular model is correct, only that it is more likely to be correct than false. However, in general, the median voter model appears to be quite robust as a model of public policy formation in areas where the median voter can credibly be thought to understand and care about public policy.

The empirical evidence suggests that the median voter model can serve as a very useful first approximation of governance within democratic polities. As a consequence, the median voter model continues to function as an analytical point of departure for more elaborate models of policy formation within democracies in much the same way that the competitive model serves the micro economics literature.

In the American political system, the effect of the median voter theorem is blunted somewhat by the primary system. It's a commonplace among American political commentators that politicians must appeal to the "base" during the primaries, then swing towards the center for the general election. Of course, politicians can't suddenly become perfectly centrist once they secure their party's nomination; their swing towards the center has to be done in a way that's at least superficially consistent with their previous pandering to their base. These observations suggest that, while reality doesn't perfectly match the idealized model, there's still a lot of truth to it. 

(Note: a site search for previous discussion of the median voter theorem on LessWrong turned up a comment by Carl Shulman that mentioned "the need to motivate one's base to vote/volunteer/contribute the ideological lumpiness" as probably having an effect similar to the effect of primaries. I wouldn't have thought they were as important as primaries but I can believe Carl here.)

The tendency of politicians to position themselves wherever the center of public opinion is currently at can be striking on specific issues. For example, public support for gay rights has increased greatly in the past two decades. In that time period, positions which once got Bill Clinton demonized by the religious right as an agent of the homosexual agenda (like Don't Ask Don't Tell) became the "conservative" position. Progress, but in terms of the public stances of politicians, it's progress that came not in the form of dramatic shifts but cautious adjustments.

By the time of the 2008 campaign, Republican nominee John McCain was voicing vague support for "legal agreements" between same sex couples, while rejecting same-sex marriage. At the same time, he suggested the issue could be punted to the states. Meanwhile, Obama's position was only slightly more liberal: clearer support for civil unions (but again not full marriage equality), and similar suggestions that the issue could be left to the states.

Four years later in 2012, Obama finally mentioned in an interview that he'd changed his mind and now supported same-sex marriage. By that time, figures from Rick Santorum to Rick Warren to Sarah Palin had begun telling the press that they, too, have gay friends. Since that time, the Obama administration has only taken modest concrete steps to support gay marriage: a narrowly-worded brief opposing California's Proposition 8, a decision not to defend the Defense of Marriage Act in court, and that's about it.

From the point of view of the median voter model, the way to explain both the behavior of liberals like Obama and conservatives like McCain and Santorum is that both groups are trying to avoid straying very far from the position of the median voter, so as to not alienate them and lose their vote. It's significant that in 2008, the polls showed that public opinion was roughly divided into thirds on gay marriage, with about a third totally opposed, a third supporting civil unions, and a third supporting full marriage equality. Obama's announcement that he supported gay marriage came after numerous polls showed 50-some percent of Americans supporting gay marriage.

Some readers may be wondering how this analysis fits with the current polarization in Congress. The answer is, "perfectly." The median voter theorem leads us to expect that politicians running against each other should adopt similar views, but even in its most idealized form, it says nothing about members of the same legislature should have similar views. In fact, it predicts polarized legislatures in situations where (1) members of the legislature are elected by geographic region and (2) the electorate itself is polarized by geographic region.

This is what we see in the US, where a big-city congressional district can be much more liberal than a rural one. Many members of the House of Representatives probably have more to worry about from a more-extreme primary challenger within their own party than from a general election challenger from the other party. Caveat: I've tried looking up data on the voting records of various House members, and while there's clearly a correlation between the tendencies of their respective districts, the correlation is not as strong as I expected. I'd be curious to hear if anyone out there knows more about this issue of polarization and geography. 

Voting systems, voting strategies, and knowing your fellow voters

So elections in the US may not offer voters much choice, but that's better explained by the median voter theorem than by politicians colluding against voters. Political science also provides a second objection to Eliezer's analysis of the two-party system in America: Duverger's Law, which says that in a system like ours (where everyone votes for one candidate and whoever gets the most votes wins), the system will tend to converge on having two main political parties, due to standard reasoning about not throwing your vote away. A corollary is that you can get a multiparty system by using proportional representation, which is used in many countries around the world including Spain, Portugal, Italy, Germany, and Israel.

There are some apparent exceptions to Duverger's Law, such as Canada, which has long had a multiparty system in spite of using a voting system similar to that of the US. However, a friend of mine who follows Canadian politics tells me that what really happens in Canada isn't that far from what you would expect given Duverger's Law. Currently, the three largest parties are the Conservative Party, the New Democratic Party (NDP), and the Liberal Party. It used to be that the NDP was a relatively small party with positions well to the left of the Liberals, but this is no longer true. Instead of offering Canadian voters two different flavors of liberalism, the current situation is that in any given election for any given seat in parliament, the NDP candidate and the Liberal candidate put a lot of effort into arguing over who has the best chance of beating the Conservatives.¹

So suppose you're an American or Canadian or British voter, looking at the major-party candidates in the next election, and finding that none of them are a good fit for your political views, what you should conclude? First, given that the median voter theorem is a pretty good model of how elections actually work, you should probably take your as evidence that your views are a good ways away from those of the median voter. And if the views of the voters are sufficiently varied, a majority of voters could find themselves in the same position as you.

In the flagpole problem at the start of this post, the only one student originally wanted the height that ends up winning. Actually, there's a subtle joke I left out of my paraphrase: the student who wanted 65 feet was Kathy, who elsewhere in the series was established as hating everyone and loving to see bad things happen. Or, to use the gay marriage example: in the 2008 election, the ~1/3 of voters who supported gay marriage didn't have a major party candidate who supported their views (and voters totally opposed to gay marriage and civil unions may not have been terribly happy with their choices either).

To throw off the yoke of the existing major parties, it isn't enough for most voters to reject their platforms. They need to reject their platforms in more or less the same direction. In "Stop Voting for Nincompoops," Eliezer mentions having anti-interventionist foreign policy views, and based on that, maybe he would say Obama is a nincompoop for being too interventionist, too willing to kill foreigners in the name of fighting terrorism. If so, I'd be sympathetic. But even if a majority of Americans agreed that Obama is a nincompoop, it wouldn't follow that they agree he is a nincompoop for being too willing to kill foreigners in the name of fighting terrorism. Many of them probably think he's a nincompoop for not doing nearly enough to fight terrorism, and maybe even being secretly on the side of the terrorists.²

That's because median voter analysis suggests that if none of the main candidates in an election are a good fit for your views, this is a sign that your views are a good ways from those of the median voter, and as a corollary there must be people out there whose views differ from the median voter's in the opposite direction, and therefore would seem even more repugnant to you. (Never forget that half the population is below average.)

In "Stop Voting for Nincompoops," Eliezer quotes from Douglas Adams' novel So Long And Thanks For All The Fish:

"The leaders are lizards.  The people hate the lizards and the lizards rule the people."

"Odd," said Arthur, "I thought you said it was a democracy."

"I did," said Ford, "It is."

"So," said Arthur, hoping he wasn't sounding ridiculously obtuse, "why don't the people get rid of the lizards?"

"It honestly doesn't occur to them," said Ford. "They've all got the vote, so they all pretty much assume that the government they've voted in more or less approximates to the government they want."

"You mean they actually vote for the lizards?"

"Oh yes," said Ford with a shrug, "of course."

"But," said Arthur, going for the big one again, "why?"

"Because if they didn't vote for a lizard," said Ford, "the wrong lizard might get in. Got any gin?"

In light of all the above, let me suggest a modified allegory: the people hate the lizards, and have thought of getting rid of them, but there's disagreement about what to do after getting rid of the lizards. Many people favor self-rule, but a very nearly equal number of people favor replacing the lizards with the Demon Acolytes of Yog-Sothoth. Since a few people actually like the lizards, and almost everyone agrees lizards are better than what those other people want, lizards are what they get. 

Of course, since very few people consider themselves villains, to make the story as realistic as possible, we should imagine that the partisans of Demon Acolytes believe the demons are actually Angels of the Light, and that anyone prideful enough to think autonomy is better than being ruled by angels must be profoundly wicked. Either way, the point is that widespread dislike of the current political situation does not imply widespread support for any particular alternative.

Moving back to the real world again, here's an explanation for US foreign policy under both Bush II and Obama, which I suspect Eliezer would think too cynical, but which I'll mention anyway: maybe the reason the US government is so quick to kill foreigners in the name of fighting terrorism is because the median voter fears terrorism more than they care about the lives of foreigners. I suppose you could argue it isn't so, and the real reason is the median voter doesn't know what impact US foreign policy has on foreigners, but if they cared to know, couldn't they start paying less attention to CNN and more to Al-Jazeera?

Given all this, how should you vote? Well, you shouldn't vote for a third party candidate because you think a lot of our problems could be solved if everyone just simultaneously resolved to never vote for (anyone they believed to be) a nincompoop. If somehow you actually manage to persuade people to everyone to adopt that policy, don't be surprised if disagreements about who the nincompoops are result in nothing really changing, or worse result in a bunch bizarre elections decided by small pluralities.

Beyond that though, I'm not actually sure what the proper strategy is. In spite of everything I've said, maybe the "vote third party to send a message" argument is (sometimes) right. Or maybe there's something to be said for the argument that your vote isn't going to make a difference anyway so you may as well do whatever makes you feel good. So far in my relatively short time as a voter, I've adopted a mixed approach, protest-voting in my two presidential elections but voting for major-party candidates otherwise. But I'm honestly not sure what I'll do in the future. Maybe a seemingly-infinitesimal chance of affecting the election outcome is worth it.

That is not a very exciting way to end an essay this long. Which is why I'm happy to report that that is not how I'm ending this essay, and in fact have been building up to a different general point.

A nerd failure mode regarding human affairs

So at last, I'm ready to explain what I think the broader nerd failure mode here is: they have a tendency to notice that people are failing to behave optimally and then propose, as a solution to this problem, that people switch to behaving optimally.

This is related to, if not quite the same as, the problem Randall Munroe pokes at here. The problem is that if you don't first make a serious effort to figure out why people are failing to behave optimally, that can get in the way of figuring out what a better course of action would be. And it makes it almost impossible to figure out how to get people to actually follow the better course of action.

If the reason people elect bad leaders is that half the people have views even crazier than those of the leaders they elect, you will not make much progress changing things if you think the problem is a two-party conspiracy against the voters. Or, if you to get people to stop voting for nincompoops, convincing them they should never vote for nincompoops may give you a very different result than you were expecting if they have different ideas from you about who the nincompoops are and what it is about them that qualifies them as nincompoops.

Many readers of LessWrong will have heard of Chesterton's fence already, but let me quote Chesterton's original words at somewhat greater length than is usual:

In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say for the sake of simplicity, a fence or gate erected across a road.  The more modern type of reformer goes gaily up to it and says, "I don't see the use of this; let us clear it away." To which the more intelligent type of reformer will do well to answer: "If you don't see the use of it, I certainly won't let you clear it away. Go away and think.  Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it."

This paradox rests on the most elementary common sense. The gate or fence did not grow there.  It was not set up by somnambulists who built it in their sleep.  It is highly improbable that it was put there by escaped lunatics who were for some reason loose in the street. Some person had some reason for thinking it would be a good thing for somebody.  And until we know what the reason was, we really cannot judge whether the reason was reasonable. It is extremely probable that we have overlooked some whole aspect of the question, if something set up by human beings like ourselves seems to be entirely meaningless and mysterious. There are reformers who get over this difficulty by assuming that all their fathers were fools; but if that be so, we can only say that folly appears to be a hereditary disease. But the truth is that nobody has any business to destroy a social institution until he has really seen it as an historical institution. If he knows how it arose, and what purposes it was supposed to serve, he may really be able to say that they were bad purposes, or that they have since become bad purposes, or that they are purposes which are no longer served.  But if he simply stares at the thing as a senseless monstrosity that has somehow sprung up in his path, it is he and not the traditionalist who is suffering from an illusion.

In spite of being a conservative Catholic apologist, what Chesterton is saying here isn't crazy. Certainly it helps to know what people's reasons for something were before trying to judge whether they were good ones. I wouldn't go quite as far as Chesterton, since sometimes there's such good evidence something's a bad idea that you can reject it without knowing what people were originally thinking.

But even on much weaker assumptions than Chesterton's, something in the vicinity turns out to be good advice. Even if the fence was built by lunatics, that's worth knowing. It's especially worth knowing whether they're still out there, and whether they're likely to try to rebuild the fence after it's been taken down. If they are likely to try that, you need to know so they can be recaptured before taking the fence down, so that the lunatics don't just rebuild it, making the taking-down a waste of effort.

Notes

1. Someone might read this and conclude that, since the two-party system is so awful, and Duverger's Law implies it's a necessary result of our voting system, shouldn't we switch voting systems to something like proportional representation? I'm willing to believe that other systems might be slightly better than what we have in the US. Countries that use proportional representation tend to have higher voter turnout, though it's unclear whether the one causes the other. But does anyone think that proportional representation and more major parties makes, say, Germany's government that much better than the UK's? For more on voting systems, see Yvain's summary of why no voting system is perfect.

2. Some people reading this might be skeptical of the idea many people would believe something as crazy-sounding as "Obama is secretly on the side of the terrorists." While I think we should be careful about phantom lizardmen and partisan media selectively reporting on the other side's crazies to gin up outrage, sadly, from what I can tell there genuinely are a large number of people out there who believe such right-wing conspiracy theories about Obama. I'm not trying to make a partisan point here, and say this with full awareness of things like 9/11 conspiracy theories on the left. 

   Remember, first, that hardly any of us come into contact with a random sampling of our fellow voters on a daily basis. Furthermore, I grew up in a smallish (pop. ~60k), conservative-leaning town, and occasionally people I barely interacted with in high school will friend me on Facebook, I'll accept because why not, and then I'll start getting their thoughts on politics in my Facebook feed. That may give me a somewhat clearer perspective on this than the averge resident of a liberal big city. I remember when the NSA scandal broke and one girl posted a status update which, while containing civil-libertarian thoughts that I approved of, also contained references to Obama being an illegal president (because, as far as I could tell, birtherism), as well as a reference to Obama's "terroristic ways," whatever that means.