Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

MugaSofer comments on Mysterious Answers to Mysterious Questions - Less Wrong

72 Post author: Eliezer_Yudkowsky 25 August 2007 10:27PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (152)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: MugaSofer 18 October 2012 12:36:02PM *  0 points [-]

Oh, so you pay counterfactual muggers?

All is explained.

Comment author: ArisKatsaris 18 October 2012 01:20:17PM 2 points [-]

The counterfactual mugging isn't that strange if you think of it as a form of entrance fee for a positive-expected-utility bet -- a bet you happened to lose in this instance, but it is good to have the decision theory that will allow you to enter it in the abstract.

The problem is that people aren't that good in understanding that your specific decision isn't separate from your decision theory under a specific context ... DecisionTheory(Context)=Decision. To have your decision theory be a winning decision theory in general, you may have to eventually accept some individual 'losing' decisions: That's the price to pay for having a winning decision theory overall.

Comment author: MugaSofer 19 October 2012 09:09:05AM 0 points [-]

I doubt that a decision theory that simply refuses to update on certain forms of evidence can win consistently.

Comment author: ArisKatsaris 19 October 2012 09:40:29AM *  1 point [-]

If Parfit's hitchhiker "updates" on the fact that he's now reached the city and therefore doesn't need to pay the driver, and furthermore if Parfit's hitchhiker knows in advance that he'll update on that fact in that manner, then he'll die.

If right now we had mindscanners/simulators that could perform such counterfactual experiments on our minds, and if this sort of bet could therefore become part of everyday existence, being the sort of person that pays the counterfactual mugger would eventually be seen by all to be of positive-utility -- because such people would eventually be offered the winning side of that bet (free money in the tenfold of your cost).

While the sort of person that wouldn't be paying the counterfactual mugger would never be given such free money at all.

Comment author: MugaSofer 19 October 2012 10:29:51AM 0 points [-]

If, and only if, you regularly encounter such bets.

Comment author: ArisKatsaris 19 October 2012 11:11:45AM *  2 points [-]

The likelihood of encountering the winning side of the bet is proportional to the likelihood of encountering its losing side. As such, whether you are likely to encounter the bet once in your lifetime, or to encounter it a hundred times, doesn't seem to significantly affect the decision theory you ought possess in advance if you want to maximize your utility.

In addition to Omega asking you to give him 100$ because the coin came up tails, also imagine Omega coming to you and saying "Here's 100,000$, because the coin came up heads and you're the type of person that would have given me 100$ if it had come up tails."

That scenario makes it obvious to me that being the person that would give Omega 100$ if it had come up heads is the winning type of person...

Comment author: wedrifid 19 October 2012 01:51:33AM *  0 points [-]

Oh, so you pay counterfactual muggers?

If the coin therein is defined as a quantum one then yes, without hesitation. If it is a logical coin then things get complicated.

All is explained.

This is more ambiguous than you realize. Sure, the dismissive part came through but it doesn't quite give your answer. ie. Not all people would give the same response to counterfactual mugging as Transparent Probabilistic Newcomb's and you may notice that even I had to provide multiple caveats to provide my own answer there despite for most part making the same kind of decision.

Let's just assume your answer is "Two Box!". In that case I wonder whether the problem is that you just outright two box on pure Newcomb's Problem or whether you revert to CDT intuitions when the details get complicated. Assuming you win at Newcomb's Problem but two box on the variant then I suppose that would indicate the problem is in one of:

  • Being able to see the money rather than being merely being aware of it through abstract thought switched you into a CDT based 'near mode' thought pattern.
  • Changing the problem from a simplified "assume a spherical cow of uniform density" problem to one that actually allows uncertainty changes things for you. (It does for some.)
  • You want to be the kind of person who two-boxes when unlucky even though this means that you may actually not have been unlucky at all but instead have manufactured your own undesirable circumstance. (Even more people stumble here, assuming they get this far.)

The most generous assumption would be that your problem comes at the final option---that one is actually damn confusing. However I note that your previous comments about always updating on the free money available and then following expected utility maximisation are only really compatible with the option "outright two box on simple Newcomb's Problem". In that case all the extra discussion here is kind of redundant!

I think we need a nice simple visual taxonomy of where people fall regarding decision theoretic bullet-biting. It would save so much time when this kind of thing. Then when a new situation comes up (like this one with dealing with time traveling prophets) we could skip straight to, for example, "Oh, you're a Newcomb's One-Boxer but a Transparent Two-Boxer. To be consistent with that kind of implied decision algorithm then yes, you would not bother with flight-risk avoidance."