# wedrifid comments on Mysterious Answers to Mysterious Questions - Less Wrong

72 25 August 2007 10:27PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Sort By: Old

Comment author: 16 October 2012 11:48:38PM *  2 points [-]

In Newcomb's problem, Omega knows what you will do using their superintelligence. Since you know you cannot two-box successfully, you should one-box.

In this case you are trying (futilely) so that you, very crudely speaking, are less likely to be in the futile situation in the first place.

If Omega didn't know what you would do with a fair degree of accuracy, two-boxing would work, obviously.

Yes, then it wouldn't be Newcomb's Problem. The important feature in the problem isn't boxes with arbitrary amounts of money in them. It is about interacting with a powerful predictor whose prediction has already been made and acted upon. See, in particular, the Transparent Newcomb's Problem (where you can ourtright see how much money is there). That makes the situation seem even more like this one.

Even closer would be the Transparent Newcomb's Problem combined with an Omega that is only 99% accurate. You find yourself looking at an empty 'big' box. What do you do? I'm saying you still one box the empty box. That makes it far less likely that you will be in a situation where you see an empty box at all.

Comment author: 17 October 2012 04:35:05PM 0 points [-]

Being a person who avoids plane crashes makes it less likely that you will be told "you will die in a plane crash", yes.

But probability is subjective - once you have the information that you will die in a car crash, your subjective estimate of this should vastly increase, regardless of the precautions you take.

Comment author: 17 October 2012 10:42:58PM 1 point [-]

But probability is subjective - once you have the information that you will die in a car crash, your subjective estimate of this should vastly increase, regardless of the precautions you take.

Absolutely. And I'm saying that you update that probability, perform a (naive) expected utility function calculation that says "don't bother trying to prevent plane crashes" then go ahead and try to avoid plane crashes anyway. Because in this kind of situation maximising expected utility is actually a mistake.

(To those who consider this claim to be bizarre without seeing context, note that we are talking situations such as within time-loops.)

Comment author: 18 October 2012 08:09:16AM 0 points [-]

Because in this kind of situation maximising expected utility is actually a mistake.

So ... I should do things that result in less expected utility ... why?

Comment author: 18 October 2012 09:28:20AM 0 points [-]

So ... I should do things that result in less expected utility ... why?

I am happy to continue the conversation if you are interested. I am trying to unpack just where your intuitions diverge from mine. I'd like to know what your choice would be when faced with Newcomb's Problem with transparent boxes and an imperfect predictor when you notice that the large box is empty. I take the empty large box, which isn't a choice that maximises my expected utility and in fact gives me nothing, which is the worst possible outcome from that game. What do you do?

Comment author: 18 October 2012 12:36:02PM *  0 points [-]

Oh, so you pay counterfactual muggers?

All is explained.

Comment author: 18 October 2012 01:20:17PM 2 points [-]

The counterfactual mugging isn't that strange if you think of it as a form of entrance fee for a positive-expected-utility bet -- a bet you happened to lose in this instance, but it is good to have the decision theory that will allow you to enter it in the abstract.

The problem is that people aren't that good in understanding that your specific decision isn't separate from your decision theory under a specific context ... DecisionTheory(Context)=Decision. To have your decision theory be a winning decision theory in general, you may have to eventually accept some individual 'losing' decisions: That's the price to pay for having a winning decision theory overall.

Comment author: 19 October 2012 09:09:05AM 0 points [-]

I doubt that a decision theory that simply refuses to update on certain forms of evidence can win consistently.

Comment author: 19 October 2012 09:40:29AM *  1 point [-]

If Parfit's hitchhiker "updates" on the fact that he's now reached the city and therefore doesn't need to pay the driver, and furthermore if Parfit's hitchhiker knows in advance that he'll update on that fact in that manner, then he'll die.

If right now we had mindscanners/simulators that could perform such counterfactual experiments on our minds, and if this sort of bet could therefore become part of everyday existence, being the sort of person that pays the counterfactual mugger would eventually be seen by all to be of positive-utility -- because such people would eventually be offered the winning side of that bet (free money in the tenfold of your cost).

While the sort of person that wouldn't be paying the counterfactual mugger would never be given such free money at all.

Comment author: 19 October 2012 10:29:51AM 0 points [-]

If, and only if, you regularly encounter such bets.

Comment author: 19 October 2012 01:51:33AM *  0 points [-]

Oh, so you pay counterfactual muggers?

If the coin therein is defined as a quantum one then yes, without hesitation. If it is a logical coin then things get complicated.

All is explained.

This is more ambiguous than you realize. Sure, the dismissive part came through but it doesn't quite give your answer. ie. Not all people would give the same response to counterfactual mugging as Transparent Probabilistic Newcomb's and you may notice that even I had to provide multiple caveats to provide my own answer there despite for most part making the same kind of decision.

Let's just assume your answer is "Two Box!". In that case I wonder whether the problem is that you just outright two box on pure Newcomb's Problem or whether you revert to CDT intuitions when the details get complicated. Assuming you win at Newcomb's Problem but two box on the variant then I suppose that would indicate the problem is in one of:

• Being able to see the money rather than being merely being aware of it through abstract thought switched you into a CDT based 'near mode' thought pattern.
• Changing the problem from a simplified "assume a spherical cow of uniform density" problem to one that actually allows uncertainty changes things for you. (It does for some.)
• You want to be the kind of person who two-boxes when unlucky even though this means that you may actually not have been unlucky at all but instead have manufactured your own undesirable circumstance. (Even more people stumble here, assuming they get this far.)

The most generous assumption would be that your problem comes at the final option---that one is actually damn confusing. However I note that your previous comments about always updating on the free money available and then following expected utility maximisation are only really compatible with the option "outright two box on simple Newcomb's Problem". In that case all the extra discussion here is kind of redundant!

I think we need a nice simple visual taxonomy of where people fall regarding decision theoretic bullet-biting. It would save so much time when this kind of thing. Then when a new situation comes up (like this one with dealing with time traveling prophets) we could skip straight to, for example, "Oh, you're a Newcomb's One-Boxer but a Transparent Two-Boxer. To be consistent with that kind of implied decision algorithm then yes, you would not bother with flight-risk avoidance."

Comment author: 18 October 2012 01:51:24PM 0 points [-]

Two boxes, sitting there on the ground, unguarded, no traps, nobody else has a legal claim to the contents? Seriously? You can have the empty one if you'd like, I'll take the one with the money. If you ask nicely I might even give you half.

I don't understand what you're gaining from this "rationality" that won't let you accept a free lunch when an insane godlike being drops it in your lap.

Comment author: 18 October 2012 02:54:50PM 0 points [-]

I don't understand what you're gaining from this "rationality" that won't let you accept a free lunch when an insane godlike being drops it in your lap.

A million dollars.

Comment author: 18 October 2012 03:11:35PM 0 points [-]

No, you're not. You're getting an empty box, and hoping that by doing so you'll convince Omega to put a million dollars in the next box, or in a box presented to you in some alternate universe.

Comment author: 18 October 2012 03:32:40PM *  0 points [-]

And by this exact reasoning, which Omega has successfully predicted, you will one-box, and thus Omega has successfully predicted that you will one-box and made the correct decision to leave the box empty.

Remember to trace your causal arrows both ways if you want a winning CDT.

Remember also Omega is a superintelligence. The recursive prediction is exactly why it's rational to "irrationally" one-box.

Comment author: 18 October 2012 04:05:44PM 0 points [-]

And by this exact reasoning, which Omega has successfully predicted, you will one-box, and thus Omega has successfully predicted that you will one-box and made the correct decision to leave the box empty.

Yes, that's why I took the one box with more money in it.

Strictly speaking the scenario being discussed is one in which Omega left a transparent box of money and another transparent box which was empty in front of Wedrifid, then I came by, confirmed Wedrifid's disinterest in the money, and left the scene marginally richer. I personally have never been offered money by Omega, don't expect to be any time soon, and am comfortable with the possibility of not being able to outwit something that's defined as being vastly smarter than me.

Remember also Omega is an insane superintelligence, with unlimited resources but no clear agenda beyond boredom. If appeasing such an entity was my best prospect for survival, I would develop whatever specialized cognitive structures were necessary; it's not, so I don't, and consider myself lucky.

Comment author: 18 October 2012 04:27:12PM 2 points [-]

In the specific "infallible oracle says you're going to die in a plane crash" scenario, you might live considerably longer by giving the cosmos fewer opportunities to throw plane crashes at you.

Comment author: 19 October 2012 09:01:43AM 0 points [-]

I was assuming a time was given. wedrifid was claiming that you should avoid plane-crash causing actions even if you know that the crash will occur regardless.

Comment author: 19 October 2012 09:20:15AM 0 points [-]

If you know the time, then that becomes even easier to deal with - there's no particular need to avoid plane crash opportunities that do not take place at that time. In fact, it then becomes possible to try to avoid it by other means - for example, faking your own plane-crash-related demise and leaving the fake evidence there for the time traveller to find.

If you know the time of your death in advance, then the means become important only at or near that time.

Comment author: 19 October 2012 10:26:35AM 2 points [-]

Let's take this a step further. (And for this reply I will neglect all acausal timey-wimey manipulation considerat ions.)

If you know the time of your death you have the chance to exploit your temporary immortality. Play Russian Roulette for cash. Contrive extreme scenarios that will either result in significant gain or certain death. The details of ensuring that it is hard to be seriously injured without outright death will take some arranging but there is a powerful "fixed point in time and space" to be exploited.

Comment author: 19 October 2012 10:49:34AM 1 point [-]

Problem with playing russian roulette under those circumstances is that you might suffer debilitating but technically nonfatal brain damage. It's actually surprisingly difficult to arrange situations where there's a chance of death but no chance of horrific incapacitation.

Comment author: 19 October 2012 02:57:13PM 2 points [-]

Problem with playing russian roulette under those circumstances is that you might suffer debilitating but technically nonfatal brain damage.

Yes, that was acknowledged as the limiting factor. However it is not a significant problem when playing a few of rounds of Russian Roulette. In fact, even assuming you play the Roulette to the death with two different people in sequence you still only create two bits of selection pressure towards the incapacitation. You can more than offset this comparatively trivial amount of increased injured-not-dead risk (relative to the average Russian Roulette player) by buying hollow point rounds for the gun and researching optimal form for suicide-by-handgun.

The point is, yes exploiting death-immunity for optimization other outcomes increases the risk of injury in the same proportion that the probability of the desired outcome is increased but this doesn't become a significant factor for something as trivial as a moderate foray into Russian Roulette. It would definitely become a factor if you started trying to brute force 512 bit encryption with a death machine. That is, you would still end up with practically 0 chance of brute forcing the encryption and your expected outcome would come down to whether it is more likely for the machine to not work at all or for it to merely incapacitate you.

This is a situation where you really do have to shut up and multiply. If you try to push the anti-death too far you will just end up with (otherwise) low probability likely undesirable outcomes occurring. On the other hand if you completely ignore the influence of "death" outcomes being magically redacted from the set of possible outcomes you will definitely make incorrect expected utility calculations when deciding what is best to do. This is particularly the case given that there is a strict upper bound on how bad a "horrific incapacitation" can be. ie. It could hurt a bit for a few hours till your certain death.

This scenario is very different and far safer than many other "exploit the impossible physics" scenarios in as much as the possibility of bringing disaster upon yourself and others is comparatively low. (ie. In other scenarios it is comparatively simple/probable for the universe to just to throw a metaphorical meteor at you and kill everyone nearby as a way to stop your poorly calibrated munchkinism.)

It's actually surprisingly difficult to arrange situations where there's a chance of death but no chance of horrific incapacitation.

I shall assume you mean "sufficiently low chance of horrific incapacitation for your purposes".

It isn't especially difficult for the kind of person who can get time travelling prophets to give him advice to also have a collaborator with a gun.

Comment author: 19 October 2012 12:26:17PM 0 points [-]

You can do the same thing if you know only the means of your death and not the time in advance; merely set up your death-stunts to avoid that means of death. (For example, if you know with certainty that you will die in a plane crash but not when, you can play Russian Roulette for cash on a submarine).

Comment author: 19 October 2012 12:58:18PM 1 point [-]

And then the experimental aqua-plane crashes into you.

Comment author: 19 October 2012 02:02:04PM 0 points [-]

If you're the sort of person who would take advantage of such knowledge to engage in dangerous activities, does that increase the probability that your reported time of death will be really soon?

Comment author: 19 October 2012 02:10:59PM 0 points [-]

If you're the sort of person who would take advantage of such knowledge to engage in dangerous activities, does that increase the probability that your reported time of death will be really soon?

Absolutely. Note the parenthetical. The grandparent adopted the policy of ignoring this kind of consideration for the purpose of exploring the implied tangent a little further. I actually think not actively avoiding death, particularly death by the means predicted, is a mistake.

Comment author: 19 October 2012 05:24:50PM 1 point [-]

On the other hand, if you're the kind of person who (on discovering that you will die in a plane crash) takes care to avoid plane crashes, wouldn't that increase your expected life span?

Moreover, these two attitudes - avoiding plane crashes and engaging in non-plane-related risky activities - are not mutually exclusive.

Comment author: 19 October 2012 10:27:16AM 0 points [-]

... True.

But you could still be injured by a plane crash or other mishap at another time, at standard probabilities.

And you should still charter your own plane to avoid collateral damage.

Comment author: 19 October 2012 10:06:41AM 0 points [-]

Yes, you are correct. Or at least it is true that I am not trying to make a "manipulate time of death" point. Let's say we have been given a reliably predicted and literal "half life" that we know has already incorporated all our future actions.

Comment author: 19 October 2012 10:28:58AM 0 points [-]

OK.

So the odds of my receiving that message are the same as the odds of my death by plane, but having recieved it I can freely act to increase the odds of my plane-related death without repercussions. I think.