Summary: Rationalists should win; however, it could take a really long time before a technological singularity or uploading provide powerful technology to aid rationalists in achieving their goals. It's possible today to create assistant computer software to help direct human effort and provide "hints" for clearer thinking. We should catalog such software when it exists and create it when it doesn't.

The Problem
We may be waiting awhile for a Friendly AI or similar “world changing” technology to appear. While technology continues to improve, the process of creating a Friendly AI seems extremely tricky, and there’s no solid ETA on the program. Uploading is still years to decades away. In the meantime, aspiring rationalists still have to get on with our lives.

Rationality is hard. Merely knowing about a bias is often not enough to overcome it. Even in cases where the steps to act rationally are known, the algorithm required may be more than can be done manually, or may require information which itself is not immediately at hand. However, a lot of things that are difficult become easier when you have the right tools. Could there be tools that supplement the effort involved in making a good decision? I suspect that this is the case, and will give several examples of programs that the community could work to create -- computer software to help you win. Because a lot of software is specifically created to address problems as they come up, it would also be worthwhile to maintain an index of already available software with special usefulness and applicability to Less Wrong readers.

Meta: Influenced by a cool blog post by Kaj, which was influenced by a cool Michael Vassar (like pretty much everything else; the man sure has a lot of ideas). The name of this post is intended to be taken slightly more literally than the similarly titled Deontology for Consequentialists.

There's been a hip new trend going around the Singularity Institute Visiting Fellows house lately, and it's not postmodernism. It's virtue ethics. "What, virtue ethics?! Are you serious?" Yup. I'm so contrarian I think cryonics isn't obvious and that virtue ethics is better than consequentialism. This post will explain why.

When I first heard about virtue ethics I assumed it was a clever way for people to justify things they did when the consequences were bad and the reasons were bad, too. People are very good at spinning tales about how virtuous they are, even more so than at finding good reasons that they could have done things that turned out unpopular, and it's hard to spin the consequences of your actions as good when everyone is keeping score. But it seems that moral theorists were mostly thinking in far mode and didn't have too much incentive to create a moral theory that benefited them the most, so my Hansonian hypothesis falls flat. Why did Plato and Aristotle and everyone up until the Enlightenment find virtue ethics appealing, then? Well...

This example (and the whole method for modelling blackmail) are due to Eliezer. I have just recast them in my own words.

We join our friends, the Countess of Rectitude and Baron Chastity, in bed together. Having surmounted their recent difficulties (she paid him, by the way), they decide to relax with a good old game of prisoner's dilemma. The payoff matrix is as usual:

Were they both standard game theorists, they would both defect, and the payoff would be (1,1). But recall that the baron occupies an epistemic vantage over the countess. While the countess only gets to choose her own action, he can choose from among four more general tactics:

1. (Countess C, Countess D)→(Baron D, Baron C)   "contrarian" : do the opposite of what she does
   
2. (Countess C, Countess D)→(Baron C, Baron C)   "trusting soul" : always cooperate
   
3. (Countess C, Countess D)→(Baron D, Baron D)   "bastard" : always defect
   
4. (Countess C, Countess D)→(Baron C, Baron D)   "copycat" : do whatever she does

Recall that he counterfactually considers what the countess would do in each case, while assuming that the countess considers his decision a fixed fact about the universe. Were he to adopt the contrarian tactic, she would maximise her utility by defecting, giving a payoff of (0,5). Similarly, she would defect in both trusting soul and bastard, giving payoffs of (0,5) and (1,1) respectively. If he goes for copycat, on the other hand, she will cooperate, giving a payoff of (3,3).

Thus when one player occupies a superior epistemic vantage over the other, they can do better than standard game theorists, and manage to both cooperate.

"Isn't it wonderful," gushed the Countess, pocketing her 3 utilitons and lighting a cigarette, "how we can do such marvellously unexpected things when your position is over mine?"

This is Eliezer's model of blackmail in decision theory at the recent workshop at SIAI, filtered through my own understanding. Eliezer help and advice were much appreciated; any errors here-in are my own.

The mysterious stranger blackmailing the Countess of Rectitude over her extra-marital affair with Baron Chastity doesn't have to run a complicated algorithm. He simply has to credibly commit to the course of action:

"If you don't give me money, I will reveal your affair."

And then, generally, the Countess forks over the cash. Which means the blackmailer never does reveal the details of the affair, so that threat remains entirely counterfactual/hypothetical. Even if the blackmailer is Baron Chastity, and the revelation would be devastating for him as well, this makes no difference at all, as long as he can credibly commit to Z. In the world of perfect decision makers, there is no risk to doing so, because the Countess will hand over the money, so the Baron will not take the hit from the revelation.

Indeed, the baron could replace "I will reveal our affair" with Z="I will reveal our affair, then sell my children into slavery, kill my dogs, burn my palace, and donate my organs to medical science while boiling myself in burning tar" or even "I will reveal our affair, then turn on an unfriendly AI", and it would only matter if this changed his pre-commitment to Z. If the Baron can commit to counterfactually doing Z, then he never has to do Z (as the countess will pay him the hush money), so it doesn't matter how horrible the consequences of Z are to himself.

To get some numbers in this model, assume the countess can either pay up or not do so, and the baron can reveal the affair or keep silent. The payoff matrix could look something like this:

‘‘During race, I am going crazy, definitely,’’ he says, smiling in bemused despair. ‘‘I cannot explain why is that, but it is true.’’

The craziness is methodical, however, and Robic and his crew know its pattern by heart. Around Day 2 of a typical weeklong race, his speech goes staccato. By Day 3, he is belligerent and sometimes paranoid. His short-term memory vanishes, and he weeps uncontrollably. The last days are marked by hallucinations: bears, wolves and aliens prowl the roadside; asphalt cracks rearrange themselves into coded messages. Occasionally, Robic leaps from his bike to square off with shadowy figures that turn out to be mailboxes. In a 2004 race, he turned to see himself pursued by a howling band of black-bearded men on horseback.

‘‘Mujahedeen, shooting at me,’’ he explains. ‘‘So I ride faster.’’

This 2006 New York Times story is about Jure Robic, a Slovenian ultra long distance bicycler who goes seriously insane when he pushes himself far enough during the races. At the point he feels like dying out of fatigue he still has a major portion (estimated 50 % by his team) of his strength left. So he hands over control to his team and with their help, pushes himself into the realm of insanity and gives up control to the team:

From the New Yorker:

It was as if there were a kind of conspiracy in the basketball world about the way the game ought to be played, and Ranadivé thought that that conspiracy had the effect of widening the gap between good teams and weak teams. Good teams, after all, had players who were tall and could dribble and shoot well; they could crisply execute their carefully prepared plays in their opponent’s end. Why, then, did weak teams play in a way that made it easy for good teams to do the very things that made them so good?

[...]

David’s victory over Goliath, in the Biblical account, is held to be an anomaly. It was not. Davids win all the time. The political scientist Ivan Arreguín-Toft recently looked at every war fought in the past two hundred years between strong and weak combatants. The Goliaths, he found, won in 71.5 per cent of the cases. That is a remarkable fact. Arreguín-Toft was analyzing conflicts in which one side was at least ten times as powerful—in terms of armed might and population—as its opponent, and even in those lopsided contests the underdog won almost a third of the time.

[...] What happened, Arreguín-Toft wondered, when the underdogs likewise acknowledged their weakness and chose an unconventional strategy? He went back and re-analyzed his data. In those cases, David’s winning percentage went from 28.5 to 63.6.

[...]

Arreguín-Toft found the same puzzling pattern. When an underdog fought like David, he usually won. But most of the time underdogs didn’t fight like David. Of the two hundred and two lopsided conflicts in Arreguín-Toft’s database, the underdog chose to go toe to toe with Goliath the conventional way a hundred and fifty-two times—and lost a hundred and nineteen times.

Playing to learn

I like losing. I don't even think that losing is necessarily evil. Personally, I believe this has less to do with a desire to lose and more to do with curiosity about the game-space.

Technically, my goals are probably shifted into some form of meta-winning — I like to understand winning or non-winning moves, strategies, and tactics. Actually winning is icing on the cake. The cake is learning as much as I can about whatever subject in which I am competing. I can do that if I win; I can do that if I lose.

I still prefer winning and I want to win and I play to win, but I also like losing. When I dive into a competition I will like the outcome. No matter what happens I will be happy because I will either (a) win or (b) lose and satiate my curiosity. Of course, learning is also possible while watching someone else lose and this generally makes winning more valuable than losing (I can watch them lose). It also provides a solid reason to watch and study other people play (or play myself and watch me "lose").

The catch is that the valuable knowledge contained within winning has diminishing returns. When I fight I either (a) win or (b) lose and, as a completely separate event, (c) may have an interesting match to study. Ideally I get (a) and (c) but the odds of (c) get lower the more I dominate because my opponents could lose in a known fashion (by me winning in an "old" method). (c) should always be found next to (b). If there is a reason I lost I should learn the reason. If I knew the reason I should not have lost. Because of this, (c) offsets the negative of (b) and losing is valuable. This makes winning and losing worth the effort. When I lose, I win.

Personally, I find (c) so valuable that I start getting bored when I no longer see anything to learn. If I keep winning over and over and never learn anything from the contest I have to find someone stronger to play or start losing creatively so that I can start learning again. Both of these solutions set up scenarios where I am increasing my chances to lose. Mathematically, this starts to make sense if the value of knowledge gained and the penalty of losing combine into something greater than winning without learning anything. (c - b > a) My hunches tell me that I value winning too little and curiosity is starting to curb my desire to win. I am not playing to win; I am playing to learn.

skip to summary

Related to: Extreme Rationality: It's Not That Great

On the recent topics of "rationality is all very well but how do we translate understanding into winning?" and "isn't akrasia the most common limiting factor?", one of the best (non-recent) articles on practical rationality that I've come across is:

http://www.joelonsoftware.com/articles/fog0000000339.html

Interestingly, it uses a different kind of martial art as a metaphor. I conjecture it to be the sort of metaphor that just works well for humans.

(Most of Spolsky's posts are good reading even if you're not a programmer. I'm not in the New York real estate market but I still enjoyed his posts on that topic. He's just that good a writer.)