Game theory. You've studied the posts, you've laughed at the comics, you've heard the music¹. But the best way to make it Truly Part Of You is to play a genuine game, and I have yet to find any more effective than Diplomacy.

Diplomacy is a board game for seven people played on a map of WWI Europe. The goal is to capture as many strategic provinces ("supply centers") as possible; eighteen are needed to win. But each player's country starts off with the same sized army, and there is no luck or opportunity for especially clever tactics. The most common way to defeat an enemy is to form coalitions with other players. But your enemies will also be trying to form coalitions, and the most profitable move is often to be a "double agent", stringing both countries along as long as you can. All game moves are written in secret and revealed at the same time and there are no enforcement mechanisms, so alliances, despite their central importance, aren't always worth the paper they're printed on.

The conditions of Diplomacy - competition for scarce resources, rational self-interested actors, importance of coalitions, lack of external enforcement mechanisms - mirror the conditions of game theoretic situations like the Prisoner's Dilemma (and the conditions of most of human evolution!) and so make a surprisingly powerful laboratory for analyzing concepts like trust, friendship, government, and even religion.

Over the past few months, I've played two online games of Diplomacy. One I won through a particularly interesting method; the other I lost quite badly, but with an unusual consolation. This post is based on notes I took during the games about relevant game theoretic situations. You don't need to know the rules of Diplomacy to understand the post, but if you want a look you can find them here.

Study One: The Prisoner's Dilemma

The Prisoner's Dilemma is a classic case in game theory in which two players must decide whether or not to cooperate for a common goal. If both players cooperate, they both do better than if both defect, but one player can win big by defecting when the other cooperates. This situation is at the heart of Diplomacy.

Germany and France have agreed to ally against Britain. Both countries have demilitarized their mutual border, and are concentrating all of their forces to the north, where they take province after province of British territory.

But Britain is fighting back; not successfully, but every inch of territory is hard-won. France is doing well for itself and has captured a few British cities, but it could be doing better. The French player thinks to eirself: I could either continue battering against the heavily defended British lines, or I could secretly ally with Britain, stab Germany in the back, and waltz in along our undefended mutual border before the Germans even know what hit them. Instead of fighting for each inch of British land, I could be having dinner in Berlin within a week.

Meanwhile, in Berlin, the German player is looking towards France's temptingly undefended border and thinking the exact same thing.

If both France and Germany are honorable, and if both countries know the other is honorable, the two of them can continue fighting Britain with a two-to-one numerical advantage and probably divide England's lucrative territory among the two of them.

If Germany is naively trusting and France is a dishonest backstabber, then France can get obscene rewards by rolling over Germany while the Kaiser's armies are tied up on the fields of England.

If both countries are suspicious of the other, or if both countries try to backstab each other simultaneously, then they will both divert forces away from the war on England to guard their mutual border. They will not gain any territory in England, and they will not gain any territory along their border. They've not only stabbed each other in the back, they've shot themselves in the foot.

Study Two: Parfit's Hitch-Hiker

The wiki describes Derek Parfit's famous hitchhiker problem as:

Suppose you're out in the desert, running out of water, and soon to die - when someone in a motor vehicle drives up next to you. Furthermore, the driver of the motor vehicle is a perfectly selfish ideal game-theoretic agent, and even further, so are you; and what's more, the driver is Paul Ekman, who's really, really good at reading facial microexpressions. The driver says, "Well, I'll convey you to town if it's in my interest to do so - so will you give me $100 from an ATM when we reach town?"

 

Now of course you wish you could answer "Yes", but as an ideal game theorist yourself, you realize that, once you actually reachtown, you'll have no further motive to pay off the driver. "Yes," you say. "You're lying," says the driver, and drives off leaving you to die.

 

The so-called Key Lepanto opening is one of the more interesting opening strategies in Diplomacy, and one that requires guts of steel to pull off. It goes like this: Italy and Austria decide to ally against Turkey. This is common enough, and hindered by the fact that Turkey is probably expecting it and Italy's kind of far away from Turkey anyway.

So Italy and Austria do something unexpected. Italy swears loudly and publicly that ey's allied with Austria. Then, the first turn, Italy moves deep into undefended Austrian territory! Austria is incensed, and curses loud and long at Italy's betrayal and at eir own stupidity for leaving the frontier unguarded. Turkey laughs and leaves the two of them to their war when - boom - Austria and Italy launch a coordinated attack against Turkey from Italy's base deep in Austrian territory. The confused Turkey has no chance to organize a resistance before combined Italo-Austrian forces take Constantinople.

It's frequently a successful strategy, especially for Italy. You know what else is a successful strategy for Italy? Doing this up to the point where they take over lots of Austrian territory, forgetting the part where it was all just a ploy, and then ending up in control of lots of Austrian territory, after which they can fight Turkey at their leisure.

It's very much in Italy's advantage to play a Key Lepanto opening, and they may beg the Austrian player to go for it, saying correctly that it would benefit both of them. But the Austrian player very often refuses, telling Italy that ey would have no incentive not to just keep the conquered territory.

This problem resembles the Hitchhiker: Italy is the lost man, and Austria is the driver. Italy really wants Austria to help em play the awesome Key Lepanto opening, but Austria knows that ey would have no incentive not to break his promise once Austria's given him the help he needs. As a result, neither country gets what they want. The Key Lepanto opening is played only rarely, and this is one of the reasons.

Study Three: Enforceable Side Contracts

The Prisoner's Dilemma is nontrivial because there's no enforcement mechanism. In the presence of an enforcement mechanism, it becomes much simpler. Say two mobsters are about to be arrested, and expect to be put in a Prisoner's Dilemma type situation. They approach the mob boss with a contract with both of their names on it, saying that they have both agreed that if either of them testifies against the other, the mob boss should send his goons to shoot the rat.

For many payoff matrices, signing this contract will be a no-brainer. It ensures your opponent will cooperate at the relatively low cost of forcing you to cooperate yourself, and almost guarantees you safe passage into the desirable (C,C) square. Not only does it prevent your opponent doesn't defect out of sheer greed, but it prevents your opponent from worrying that you're going to defect and then defecting emself to save emself from being the chump.

The game of Diplomacy I won, I won through an enforceable side contract (which lost me a friend and got me some accusations of cheating, but this is par for the course for a good Diplomacy game). I was Britain; my friend H was France. H and I knew each other from an medieval times role-playing game, in which we both held land and money. The medieval kingdom of this game had a law on the books that any oath witnessed by a noble was binding on both parties and would be enforced by the king. So H and I went into our role-playing game and swore an oath before a cooperative noble, declaring that we would both aid each other in a permanent alliance in Diplomacy, or else all our in-game lands and titles would be forfeit.

A lot of people made fun of me for this, including H, but in my defense I did end up winning the game. H and I were able to do things that would otherwise have been impossible; for example, in order to convince our enemy Germany that we were at war, I took over the French city of Brest. Normally, this would be almost impossible for two allies to coordinate, even as a red herring, for exactly the reasons listed in the Hitchhiker problem above. Since the two of us were able to trust each other absolutely, this otherwise difficult maneuver became easy.

One of the advantages to strong central government is that it provides an enforcement mechanism for contracts, which benefits all parties.

Study Four: Religion As Enforcement

Religion is a special case of the enforceable side-contract in which God is doing the enforcing. God doesn't have to exist for this to work; as long as at least one party believes He does, the threat of punishment will be credible. The advantage of being able to easily make enforceable side contracts even in the absence of social authority may be one reason religion became so popular, and if humans do turn out to have a genetic tendency toward belief, the side contracts might have provided part of the survival advantage that spread the gene.

In a Youngstown Variant game (like Diplomacy, but with Eurasia instead of just Europe), I was playing Italy and after colonizing Africa was trying to juggle my forces around to defend borders with Germany, France, Turkey, and India.

India was played by my friend A, who I sometimes have philosophical discussions with and who I knew to be an arch-conservative religion-and-family-values type. I decided to try something which, as far as I know, no one's ever tried in a Diplomacy game before. "Do you swear in the name of God and your sacred honor that you won't attack me?" I asked.

"Yes," said A, and I knew he meant it, because he takes that sort of thing really seriously. I don't know if he thought he would literally go to Hell if he broke his oath, but I'm pretty sure he wasn't willing to risk it over a board game. So I demilitarized my border with India. I concentrated my forces to the west, he concentrated them to the east, and both avoided a costly stalemate in the Indian Ocean and had more forces to send elsewhere. In the future, I will seek out A for alliances more often, since I have extra reason to believe he won't betray me; this will put A in an unusually strong position.

This is not a unique advantage of religion; any strongly held philosophy that trumps self-interest would do. I would have made the same deal with Alicorn, who has stated loudly and publicly that she is a deontologist who has a deep personal aversion to lying². I would have made it with Eliezer, who has a consequentialist morality but, on account of the consequences, has said he would not break an oath even for the sake of saving the world.

But I only trust Alicorn and Eliezer because I've discussed morality with both of them in a situation where they had no incentive to lie; it was only in the very unusual conditions of Less Wrong that they could send such a signal believably. Religion is a much easier signal to send and receive without being a moral philosopher.

Study Five: Excuses as Deviations from a Rule

My previous post, Eight Short Studies on Excuses, was inspired by a maneuver I pulled during a Diplomacy game.

I was Italy, and Turkey and I had formed a mutual alliance against Austria. As part of the alliance, we had decided not to fight over who got the lucrative neutral territories in between our empires. I would get Egypt, Turkey would get Greece and Yemen, and we would avoid the resource drain of fighting each other for them so we could both concentrate on Austria.

Both Turkey and I would have liked to grab the centers that had been promised to the other. But both Turkey and I knew that maintaining the general rule of alliance between us was higher utility than getting one extra territory. BUT both Turkey and I knew that the other would be loathe to break off the alliance between just because their partner had committed one little infraction. BUT both Turkey and I knew that we would have to do exactly that, or else our ally would have a carte blanche to violate whatever terms of the alliance they wanted.

Then India (from whom I had not yet extracted his oath) made a move towards Yemen, threatening to take it from both of us. I responded by moving a navy to Yemen, supposedly to see off the Indian menace. I then messaged Turkey, saying that although I still respected the terms of our alliance, he was clearly too weak to keep Yemen out of Indian hands, so I would be fortifying it for him, and I hoped he would have the maturity to see this as a mutually beneficial move to prevent Indian expansionism, and not get too hung up on the exact terms of our alliance.

The gambit worked: Turkey decided that maintaining our alliance was more important than keeping Yemen, and that because of the trouble with India my conquest of Yemen was not indicative of a general pattern of alliance-breaking that needed to be punished.

I can't claim total victory here: several years later, when the threat of Austria had disappeared, Turkey betrayed me and captured half my empire, partly because of my actions in Yemen.

Study Six: For the Sake of Revenge

This comes from the book Game Theory at Work:

Consider the emotion of revenge. At its core, revenge means hurting someone who has harmed you, even if you would be better off leaving him alone. Revenge is an irrational desire to harm others who have injured our loved ones or us.

 

To see the benefit of being known as vengeful, consider a small community living in prehistoric times. Imagine that a group of raiders stole food from this community. A rational community would hunt down the raiders only if the cost of doing so was not too high. A vengence-endowed community would hunt down the raiders regardless of the cost. Since the raiders would rather go after the rational community, being perceived as vengeful provides you with protection and therefore confers an evolutionary advantage.

I play Diplomacy often against the same people, so I decided I needed to cultivate a reputation for vengefulness. And by "decided to cultivate a reputation for vengefulness", I mean "Turkey betrayed me and I was filled with the burning rage of a thousand suns".

So my drive for revenge was mostly emotional instead of rational. But what I didn't do was suppress my anger, the way people are always telling you. Suppressing anger is a useful strategy for one-shot games, but in an iterated game, getting a reputation for anger is often more valuable than behaving in your immediate rational self-interest.

So I decided to throw the game to Germany, Turkey's biggest rival. I moved my forces away from the Italian-German border and invited Germany to take over my territory. At the same time, I used my remaining forces supporting German attacks against Turkey. The Austrians, who had been dealing with Turkey's betrayals even longer than I had, happily joined in. With our help, German forces scored several resounding victories against Turkey and pushed it back from near the top of the game down to a distant third.

Around the same time, Germany's other enemy France also betrayed me. So I told France I was throwing the game to Germany to punish him. No point in missing a perfectly good opportunity to cultivate a reputation for vengefulness.

If I had done the rational thing and excused Turkey's betrayal because it was in my self-interest to cut my losses, I could have had a mediocre end game, and Turkey's player would have happily betrayed me the next game as soon as he saw any advantage in doing so. Instead, I'm doing very poorly in the end game, but Turkey - and everyone else - will be very wary about betraying me next time around.

Study Seven: In-Group Bias as a Schelling Point

I made the mistake of moderating a game of Diplomacy at the SIAI House, which turned into one of the worst I've ever seen. The players were five SIAI Visiting Fellows and two of my non-SIAI friends who happened to be in the area.

Jasen came up with the idea of an alliance of the five SIAI players against my two friends. Although a few of the Fellows vacillated back and forth and defected a few times, he was generally able to keep the loyalty of the five Fellows until my two friends had been eliminated from the game relatively early on. Although normally the game would have continued until one of the Fellows managed to dominate the others, it was already very late and we called it a night at that point.

It's easy to explain what happened as an irrational in-group bias, or as "loyalty" or "patriotism" among the SIAI folk. Jasen himself explained it as a desire to prove that SIAI people were especially cooperative and especially good at game theory, which I suppose worked. But there's another, completely theoretical perspective from which to view the SIAI Alliance.

Imagine you are on a lifeboat with nine other people, and determine that one of the ten of you must be killed and eaten to provide sustenance to the others. You are all ready to draw lots to decide who is dinner when you shout out "Hey, instead of this whole drawing lots thing, let's kill and eat Bob!"

If your fellow castaways are rational agents, they might just agree. If they go with lots, each has a 10% chance of ending up dinner. If everyone just agrees on Bob, then everyone has a 0% chance of ending up dinner (except poor Bob). Nine out of ten people are better off, and nine out of ten of you vote to adopt the new plan. Whether your lifeboat decides things by majority vote or by physical violence, it doesn't look good for Bob.

But imagine a week later, you still haven't been rescued, and the whole situation repeats. If everyone lets you repeat your action of calling out a name, there's a 1/9 chance it'll be eir name - no better than drawing lots. In fact, since you're very unlikely to call out your own name, it's more of a 1/8 chance - worse than just drawing lots. So everyone would like to be the one who calls out the name, and as soon as the lots are taken out, everyone shouts "Hey, instead of the whole drawing lots thing, let's kill and eat X!" where X is a different person for each of the nine castaways. This is utterly useless, and you probably end up just drawing lots.

But suppose eight of the nine of you are blond, and one is a brunette. The brunette is now a Schelling point. If you choose to kill and eat the brunette, there's a pretty good chance all of your blond friends will do the same, even if none of you had a pre-existing prejudice against brunettes. Therefore, all eight of you shout out "Let's kill and eat the brunette!", since this is safer than drawing lots. Your lifeboat has invented in-group bias from rational principles.

Such alliances are equally attractive in Diplomacy. When the five SIAI Fellows allied against my two friends, they ensured there was a five-against-two alliance with themselves on the winning side, and successfully reduced the gameboard from six opponents to four. Although they could have done this with anyone (eg Jasen could have selected two other Fellows and my two friends, and forged an equivalent coalition of five), Jasen would have been at risk of five other people having the same idea and excluding him. By choosing a natural and obvious division in which he was on the majority, Jasen avoided this risk.

Rationalist Diplomacy

I'm interested in seeing what a Diplomacy game between Less Wrongers looks like. I'm willing to moderate. The first seven people to sign up get places (don't sign up if you don't expect to have enough time for about two or three turns/week), and the next few can be alternates. Doesn't matter if you've ever played before as long as you read the rules above and think you understand them. (We already have seven people. See the post in Discussion. If many more sign up, someone else may want to moderate a second game).

Footnotes

1: Source: "Nice Guys Finish First" in the Frameshift album Unweaving the Rainbow.

2. Alicorn wishes me to note that she considers anyone playing a Diplomacy game  without prior out-of-game-context agreements secured to have waived eir right to complete honesty from her, but the general principle still stands.