atucker comments on You're in Newcomb's Box - Less Wrong

40 Post author: HonoreDB 05 February 2011 08:46PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (172)
Tags: evolution evpsych newcomblike timeless decision_theory philosophy

Comments (172)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 05 February 2011 06:59:13AM *  19 points [-]

Azathoth wants you to maximize your number of descendants; if you fail to have descendants, Azathoth will try not to have created you.

But this seems merely false. Azathoth just creates descendants whose ancestors reproduced. Azathoth isn't exerting any sort of foresight as to whether you reproduce. I can't figure out who or what you're trying to trade with. Not having children simply does not make you retroactively less likely to have existed.

I suppose you could be in a Newcomblike situation with your parents making a similar decision to have birthed you. I don't see how you could be in one with respect to Azathoth/evolution. It's not modeling you, it doesn't contain a computation similar to you, there is no logical update on what it does after you know your own decision.

Comment author: HonoreDB 05 February 2011 08:28:13PM 3 points [-]

I suppose you could be in a Newcomblike situation with your parents making a similar decision to have birthed you.

If I'd thought of ArisKatsaris's repugnant conclusions, I probably would have used those instead of Azathoth in part 2. I'm sure there are plenty of real-world situations where one's parents both had justifiably high confidence that you would turn out a certain way, and wouldn't have birthed you if they thought otherwise. And in a few cases, at least, those expectations would be repugnant ones. The argument also suggests a truly marvelous hack for creating an AI that wants to fulfill its creators intentions.

That said, I'm not entirely convinced that changing Prometheus to Azathoth should yield different answers. We can change the Predictor in Newcomb to an evolutionary process. Omega tells you that the process has been trained using copies of every human mind that has ever existed, in chronological order--it doesn't know it's a predictor, but it sure acts like one. Or an overtly reference-classed-based version: Omega tells you that he's not a predictor at all: he just picked the past player of this game who most reminds him of you, and put the million dollars in the box if-and-only-if that player one-boxed. Neither of these changes seem like they should alter the answer, as long as the difference in payouts is large enough to swamp fluctuations in the level of logical entanglement.

Comment author: atucker 06 February 2011 08:12:47AM *  1 point [-]

That said, I'm not entirely convinced that changing Prometheus to Azathoth should yield different answers. We can change the Predictor in Newcomb to an evolutionary process. Omega tells you that the process has been trained using copies of every human mind that has ever existed, in chronological order--it doesn't know it's a predictor, but it sure acts like one. Or an overtly reference-classed-based version: Omega tells you that he's not a predictor at all: he just picked the past player of this game who most reminds him of you, and put the million dollars in the box if-and-only-if that player one-boxed. Neither of these changes seem like they should alter the answer, as long as the difference in payouts is large enough to swamp fluctuations in the level of logical entanglement.

This isn't quite the same as Evolution, because you know you exist, which means that your parents one-boxed. This is like the selector using the most similar person who happens to be guaranteed to have chosen to one-box.

Since the predictor places money based on what the most similar person chose, and you know that the most similar person one-boxed, you know that there is $1000000 in box B regardless of what you pick, and you can feel free to take both.

Comment author: HonoreDB 07 February 2011 11:43:03PM 3 points [-]

Again, that same logic would seem to lead you to two-box in any variant of transparent Newcomb.

Comment author: hairyfigment 11 February 2011 08:58:19PM 0 points [-]

I haven't studied all the details of UDT, so I may have missed an argument for treating it as the default. (I don't know if that affects the argument or not, since UDT seems a little more complicated than 'always one-box'.) So far all the cases I've seen look like they give us reasons to switch from within ordinary utility-maximizing decision theory -- for a particular case or set of cases.

Now if we find ourselves in transparent Newcomb without having made a decision, it seems too late to switch in that way. If we consider the problem beforehand, ordinary decision theory gives us reason to go with UDT iff Omega can actually predict our actions. Evolution can't. It seems not only possible but common for humans to make choices that don't maximize reproduction. That seems to settle the matter. Even within UDT I get the feeling that the increased utility from doing as you think best can overcome a slight theoretical decrease in chance of existing.

If evolution could predict the future as well as Omega then logically I'd have an overwhelming chance of "one-boxing". The actual version of me would call this morally wrong, so UDT might still have a problem there. But creating an issue takes more than just considering parents who proverbially can't predict jack.

Powered by Reddit Powered by Reddit