SilasBarta comments on Desirable Dispositions and Rational Actions - Less Wrong

13 Post author: RichardChappell 17 August 2010 03:20AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (180)
Tags: rationality decision_theory newcomb

Comments (180)

You are viewing a single comment's thread. Show more comments above.

Comment author: Perplexed 17 August 2010 05:23:10AM *  9 points [-]

Thanks for posting. Your analysis is an improvement over the LW conventional wisdom, but you still doesn't get it right, where right, to me, means the way it is analyzed by the guys who won all those Nobel prizes in economics. You write:

First, let's note that there definitely are possible cases where it would be "beneficial to be irrational".

But in every example you supply, what you really want is not exactly to be irrational; rather it is to be believed irrational by the other player in the game. But you don't notice this because in each of your artificial examples, the other player is effectively omniscient, so the only way to be believed irrational is to actually be irrational. But then, once the other player really believes, his strategies and actions are modified in such a way the your expected behavior (which would have been irrational if the other player had not come to believe you irrational) is now no longer irrational!

But, better yet, lets Taboo the word irrational. What you really want him to believe is that you will play some particular strategy. If he does, in fact, believe, then he will choose a particular strategy, and your own best response is to use the strategy he believes you are going to use. To use the technical jargon, you two are in a Nash equilibrium.

So, the standard Game Theory account is based on the beliefs each player has about the other player's preferences and strategies. And, because it deals with (Bayesian) belief, it is an incredibly flexible explanatory framework. Pick up a standard textbook or reference and marvel at the variety of applications that are covered rigorously, quantitatively, and convincingly.

I suspect that the LW interest in scenarios involving omniscient agents arises from considerations of one AI program being able to read another program's source code. However, I don't understand why there is an assumption of determinism. For example, in a Newcomb-type problem, suppose I decide to resolve the question of one box or two by flipping a coin? Unless I am supposed to believe that Omega can foretell the results of future coin flips, I think the scenario collapses. Has anyone written anything on LW about responding to Omega by randomizing? [Edited several times for minor cleanups]

Comment author: SilasBarta 20 August 2010 03:53:39PM *  4 points [-]

But in every example you supply, what you really want is not exactly to be irrational; rather it is to be believed irrational by the other player in the game.

I don't think that's the real problem: after all, Parfit's Hitchhiker and Newcomb's problem also eliminate this distinction by positing an Omega that will not be wrong in its predictions.

The real problem is that Chappell has delineated a failure mode that we don't care about. TDT/UDT are optimized for situations in which the world only cares about what you would do, not why you decide to do so. In Chappell's example's, there's no corresponding action that forms the basis of the failure; the "ritual of cognition" alone determines your punishment.

The EY article he linked to ("Newcomb's Problem and the Regret of Rationality") makes the irrelevance of these cases very clear:

Next, let's turn to the charge that Omega favors irrationalists. I can conceive of a superbeing who rewards only people born with a particular gene, regardless of their choices. I can conceive of a superbeing who rewards people whose brains inscribe the particular algorithm of "Describe your options in English and choose the last option when ordered alphabetically," but who does not reward anyone who chooses the same option for a different reason. But Omega rewards people who choose to take only box B, regardless of which algorithm they use to arrive at this decision, and this is why I don't buy the charge that Omega is rewarding the irrational. Omega doesn't care whether or not you follow some particular ritual of cognition; Omega only cares about your predicted decision.

...It is precisely the notion that Nature does not care about our algorithm, which frees us up to pursue the winning Way - without attachment to any particular ritual of cognition, apart from our belief that it wins. Every rule is up for grabs, except the rule of winning. [bold added]

So Chappell has not established a benefit to being irrational, and any mulitplication of his examples would be predicated on the same error.

Of course, as I said here, it's true that there are narrow circumstances where the decision theory "always jump off the nearest cliff" will win -- but it won't win on average, and any theory designed specifically for such scenarios will quickly lose.

(I really wish I had joined this conversation earlier to point this out.)

Powered by Reddit Powered by Reddit