orthonormal comments on Rationality is Systematized Winning - Less Wrong

48 Post author: Eliezer_Yudkowsky 03 April 2009 02:41PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (252)

You are viewing a single comment's thread. Show more comments above.

Comment author: thomblake 03 April 2009 10:16:12PM 1 point [-]

Simple: most situations in real life aren't like this. If you believe Omega and one-box, you'll lose when he's lying. If your decision theory works better in hypothetical situations and worse in real life, then it doesn't make you win.

Comment author: orthonormal 04 April 2009 01:33:52AM *  8 points [-]

Apply the Least Convenient Possible World principle.

Also, I don't think Eliezer keeps harping on Newcomb's problem because he anticipates experiencing precisely that scenario. I see several important points that I don't think have been clearly made (not that I'm the one to do so):

  1. We can choose whether and when to implement certain decision algorithms, including classical causal decision theory (CCDT). This choice may in fact be trivial, or it may be subtle, but it is a worthy question for a rationalist.

  2. Although, for any fixed set of options, implementing CCDT maximizes your return, there are in fact cases where the options you have depend on the outcome of a model of your decision algorithm. I'm not talking about Omega, I'm talking about human social life. We base a large portion of our interactions with others on our anticipations of how they might respond. (This isn't often done rationally by anyone's standards, but it can be.)

  3. It gets confusing (in particular, Hofstadterian) here, but a plausibly better outcome might be reached in the Prisoner's Dilemma by selfish non-strangers mutually modeling the other's likely decision process, and recognizing that only C-C and D-D are stable outcomes under mutual modeling.

Of course, I still feel a bit uncomfortable with this line of reasoning.