= 27d567bc2d48d9f40e9ccc20ea370b15)
Psy-Kosh comments on Don't be Pathologically Mugged! - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (30)
Yes, I know the explicit negation is what makes it pathological in this case. My problem above was deliberately contrived. The point is that I'm not sure how to more precisely separate such pathological cases from non pathological cases. I'm pretty sure we wouldn't be able to establish a theorem to the effect of "any situation that lacks an explicit 'inverter' will be well behaved and compatible with the ability to make reasonable decisions in reasonable situations"
I think the important part is that the agent doesn't know about it, not the negation. Throw in information we assume the agent can't fully take into account and the agent fails when the information would have been important. Maybe your point isn't that simple but I'm not seeing it.
Maybe that's sufficient to take care of the issue, but I'm not sure I can prove that it is. ie, my point was more "As obvious as the pathology of this one was, can we be sure that we won't run into analogous... but subtler and harder to detect incompatibilities in, say, whatever next week's souped up extension of counterfactual mugging someone comes up with or whatever?"
My worry isn't about this specific instance of a pathological decision problem. This specific instance was simply meant to be a trivial illustration of the issue. Maybe I'm being silly, and we'll be able to detect any of these cases right away, but maybe not. I haven't really thought yet of a way of precisely stating the problem, but basically it's a "while we're having all this fun figuring out how to make a decision algorithm that wins in these weird mindbending cases... let's make sure doing so doesn't sacrifice its ability to win for more ordinary cases."
Pathological cases are unlikely. Non-pathological cases are likely. This is because rational agents do whatever makes non-pathologicality likely rather than unlikely. If people are getting Pathologically Mugged on a regular basis, you end up with a more normal decision problem as people adapt to the pathological mugging.
So how often does actual Counterfactual Mugging come up?
Homeowner's insurance.
ETA: Actually, insurance of any kind. I'll explain with a simple dialogue:
"Hey, give me $1000."
"WTF? Why?"
"What's with the attitude? If I had found out your house was going to burn down in the next year, I was going to give you $300,000. Well, turns out it's not. But if you weren't willing to give me $1000 in every safe year, I wouldn't plan on giving you $300,000 in the years your house burns down! Now, are you going to give me $1000, or are you going to condemn to poverty all of your copies living in worlds where your house burns down?"
If no one's pointed out this insight before, and it really is an insight, please pass this analogy along.
If insurance companies predict that my house will burn down next year, they do not simulate me to determine whether I would have paid the $1,000 had they counterfactually mugged me. They just ask me to pay $1,000 every year, before my house potentially burns. This is a critical difference.
Or are insurance companies more psychic than I thought?
That's not necessary for the parallel to work. In my post, the insurer is stating how things look from your side of the deal, in a way that shows the mapping to the counterfactual mugger. (And by the way, if insurers predict your house will burn down they don't offer you a policy -- not one as cheap as $1000. If they sell you one at all, they sell at a price equal to the payout, in which case it's just shuffling money around.)
What creates a mapping to Newcomb's problem (and by transition, the counterfactual mugging) is the inability to selectively set a policy so that it only applies at just the right time to benefit you. With a perfect predictor (Omega), you can't "have a policy of one-boxing" yet conceal your intent to actually two-box.
This same dilemma arises in insurance, without having to assume a perfect, near-acausal predictor: you can't "decide against buying insurance" and then make an exception over the time period where the disaster occurs. All that's necessary for you to be in that situation is that you can't predict the disaster significantly better than the insurer (assuming away for now the problems of insurance fraud and liability insurance, which introduce other considerations).
I see your point.
The analogy is that both situations use expected utility to make a decision. The principal difference is that when you are buying insurance, you do expect your house to burn to the ground, with a small probability. Counterfactual mugging suggests an improbable conclusion that the updated probability distribution is not what you should base your decisions on, and this aspect is clearly absent from normal insurance.
No, normal insurance has this aspect too: you don't regret buying insurance once you learn the updated probability distribution, so you shouldn't base your decision on it either.
I don't regret it, because I remember that it was the best I could do given the information I had at the time. But if I knew when deciding whether or not to buy insurance that I would not be sued or become liable for large amounts, then I wouldn't buy it. And I don't want to change my decision algorithm to ignore information just because I didn't have it some other time.
Where did I suggest that throwing away information is somehow optimal or of different optimality than in Newcomb's problem or the counterfactual mugging?
That's not really a counterfactual mugging though, is it? ie, it doesn't fit the template of "I decided to flip a fair coin give you ten dollars if it came up heads if I predicted (and I'm really good at predicting) that if it came up tails you would give me five dollars. I flipped the coin yesterday and it came up tails. So... do you give me five dollars?"
EDIT: to respond to your edit... what insurance company would actually do that? ie, you first have to sign up with them, etc etc. And if their actuaries compute that there's a reasonable likelihood that they'll have to pay out to you... They avoid you or give you nastier premiums or such in the first place. I guess I could see an argument that insurance could be viewed as RELATED to something like an iterated counterfactual mugging, though...
There are a few ways you can look at this to make it seem more relevant. I think you can transform the counterfactual mugging into insurance such that each step shouldn't change your answer.
But let me put it this way instead: Imagine that you're going to insert your consciousness into a random "you" across the multiverse. In some of those your house (or other valuable) burns down (or otherwise descends into entropy). Would you rather be thrown into a "you" who had bought insurance, or hadn't bought insurance?
Remember, it's not an option to only buy insurance in the ones where your house burns down, i.e. to separate the "yous" into a) those whose houses didn't burn down and didn't buy insurance, vs. b) those whose houses did burn down and did buy insurance. This inseparability, I think, captures the salient aspects of the counterfactual mugging because it's (presumably) not an option to "be the type to pay the mugger" only in those cases where the coin flip favors you.
(I daydreamed once about some guy whose house experiences a natural disaster, so he goes to an insurance company with which he has no policy, and when it's explained to him that they only pay out to people who have a policy with them, he rolls his eyes and tries to give them money equal to a month's premium, as if that will somehow make them pay out.)