skeptical_lurker comments on Simulate and Defer To More Rational Selves - LessWrong

125 Post author: BrienneYudkowsky 17 September 2014 06:11PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (112)

You are viewing a single comment's thread. Show more comments above.

Comment author: skeptical_lurker 08 October 2014 02:24:27AM 2 points [-]

Its kinda obvious, but deeply counter-intuitive - I mean its a situation where stupidity is decisive advantage!

Comment author: Lumifer 08 October 2014 02:57:43PM *  5 points [-]

its a situation where stupidity is decisive advantage!

Not quite stupidity -- irrationality. And it is well-known that (credible) irrationality can be a big advantage in negotiations and other game theory scenarios. Essentially, if I'm irrational then you cannot simulate me accurately and cannot predict what I will do which means that your risk aversion pushes you towards safe choices which limit your downside at the cost of your upside. And if it's a zero-sum game, I get this upside.

Of course, I need to be credible in showing my irrationality.

The reason such a strategy is not used more often is because (a) often there is the option to walk away which many people do when faced with an irrational counterparty; and (b) when two irrational counterparties meet, bad things happen :-)

Comment author: gjm 08 October 2014 11:41:15PM *  3 points [-]

There are instances where (arguably) irrationality confers a big game-theoretic advantage even though you're predictable.

For instance, suppose you're leading a nuclear superpower. If you can make it credibly clear that you really truly would be happy to launch World War Three if the other guys don't back down, then they probably will. Not because they can't predict your actions, but because they can.

In this sort of case it's either debatable whether it's really irrationality, or debatable whether it's really a game-theoretic advantage. If you can really be sure that the other guys will back down, then maybe it's not irrationality because you never have to blow up the world. If you can't, then maybe you don't have a game-theoretic advantage after all because if you play this game often enough then the other guys call your bluff, you push the big red button, and everyone dies.

[EDITED to add: I think this sort of case is nearer to the example discussed upthread than the sort where unpredictability is key.]

Comment author: Lumifer 09 October 2014 12:14:02AM *  0 points [-]

For instance, suppose you're leading a nuclear superpower. If you can make it credibly clear that you really truly would be happy to launch World War Three

That's more like sheer bloodymindedness X-) not irrationality.

then the other guys call your bluff, you push the big red button, and everyone dies.

Yeah, it's called the game of chicken and that's a slightly different thing.