MoreOn comments on Newcomb's Problem and Regret of Rationality - Less Wrong

64 Post author: Eliezer_Yudkowsky 31 January 2008 07:36PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (588)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: MoreOn 11 December 2010 06:09:01PM *  3 points [-]

Let me try my own stab at a little chat with Omega. By the end of the chat I will either have 1001 K, or give up. Right now, I don’t know which.

Act I

Everything happens pretty much as it did in Polymeron’s dialogue, up until…

Me: "Aha! That means I do have a choice here, even before you have left. If I change my state so that I am unable or unwilling to two-box once you've left, then your prediction of my future "decision" will be different. In effect, I will be hardwired to one-box. And since I still want to retain my rationality, I will make sure that this hardwiring is strictly temporary."

Omega: Yup, that’ll work. So you’re happy with your 1000 K?

Act II

Whereupon I try to exploit randomness.

Me: Actually, no. I’m not happy. I want the entire 1001 K. Any suggestions for outsmarting you?

Omega: Nope.

Me: Are you omniscient?

Omega: As far as you’re concerned, yes. Your human physicists might disagree in general, but I’ve got you pretty much measured.

Me: Okay, then. Wanna make a bet? I bet I can find a <clever use of a clever formula> to get over 1000 K if I make a bet with you. You estimate your probability of being right at 100%, right? Nshepperd had a good suggestion….

Omega: I won’t play this game. Or let you play it with anyone else. I thought we’d moved past that.

Me: How about I flip a fair coin to decide between B and A+B. In fact, I’ll use <Name Drop>’s generator using the <technobable technobable magic technobable> principle to generate the outcome of a truly random coin flip. Even you can’t predict the outcome.

Omega: And what do you expect to happen as a result of this (not-as-clever-as-you-think) strategy?

Me: Since you can’t predict what I’ll do, hopefully you’ll fill both boxes. Then there’s a true 50% chance of me getting 1001 K. My expected payoff is 1000.5 K.

Omega: That, of course, is assuming I’ll fill both boxes.

Me: Oh, I’ll make you fill both boxes. I’ll bias the <Name Drop>’s <technobable> to 50+eps% chance of one-boxing for the expected winnings of 1000.5 K – eps. Then if you want to maximize your omniscience-y-ness, you’ll have to fill both boxes.

Omega: Oh, taking others’ suggestions already? Can’t think for yourself? Making edits to make it look like you’d thought of it in time? Fair enough. Attribute this one to gurgeh. As to the idea itself, I’ll disincentivize you from randomization at all. I won’t fill box B if I predict you cheating.

Me: But then there’s a 50-eps% chance of proving you wrong. I’ll take it. MWAHAHA.

Omega: What an idiot. You’re not trying to prove me wrong. You’re trying to maximize your own profit.

Me: The only reason I don’t insult you back is because I operate under Crackers Rule.

Omega: Crocker’s Rules.

Me: Uh. Right. Whoops.

Omega: Besides. Your <Name Drop>’s <technobable> random generator idea won’t work even to get you the cheaters’ utility for proving me wrong.

Me: Why not? I thought we’d established that you can’t predict a truly random outcome.

Omega: I don’t need to. I can just mess with your <Name Drop>’s <technobable> randomness generator so that it gives out pseudo-random numbers instead.

Me: You’re omnipotent now, too?

Omega: Nope. I’ll just give someone a million dollars to do something silly.

Me: No one would ever…! Oh, wait. Anyway, I’ll be able to detect tampering with randomness, the same way it’s possible with a Mersenne twister….

Omega: And I know exactly how soon you’ll give up. Oh, and don’t waste page space suggesting secondary and tertiary levels of ensuring randomness. If, to guide your behavior, you’re using the table of random numbers that I already have, then I already know what you’d do.

Me: Is there any way at all of outsmarting you and getting 1001 K?

Omega: Not one you can find.

Me: Okay then… let me consult smarter people.

This conversation is obviously not going my way. Any suggestions for Act III?