Okay, maybe not me, but someone I know, and that's what the title would be if he wrote it. Newcomb's problem and Kavka's toxin puzzle are more than just curiosities relevant to artificial intelligence theory. Like a lot of thought experiments, they approximately happen. They illustrate robust issues with causal decision theory that can deeply affect our everyday lives.
Yet somehow it isn't mainstream knowledge that these are more than merely abstract linguistic issues, as evidenced by this comment thread (please no Karma sniping of the comments, they are a valuable record). Scenarios involving brain scanning, decision simulation, etc., can establish their validy and future relevance, but not that they are already commonplace. For the record, I want to provide an already-happened, real-life account that captures the Newcomb essence and explicitly describes how.
So let's say my friend is named Joe. In his account, Joe is very much in love with this girl named Omega… er… Kate, and he wants to get married. Kate is somewhat traditional, and won't marry him unless he proposes, not only in the sense of explicitly asking her, but also expressing certainty that he will never try to leave her if they do marry.
Now, I don't want to make up the ending here. I want to convey the actual account, in which Joe's beliefs are roughly schematized as follows:
- if he proposes sincerely, she is effectively sure to believe it.
- if he proposes insincerely, she will 50% likely believe it.
- if she believes his proposal, she will 80% likely say yes.
- if she doesn't believe his proposal, she will surely say no, but will not be significantly upset in comparison to the significance of marriage.
- if they marry, Joe will 90% likely be happy, and will 10% likely be unhappy.
He roughly values the happy and unhappy outcomes oppositely:
- being happily married to Kate: 125 megautilons
- being unhapily married to Kate: -125 megautilons.
So what should he do? What should this real person have actually done?1 Well, as in Newcomb, these beliefs and utilities present an interesting and quantifiable problem…
- ExpectedValue(marriage) = 90%·125 - 10%·125 = 100,
- ExpectedValue(sincere proposal) = 80%·100 = 80,
- ExpectedValue(insincere proposal) = 50%·80%·100 = 40.
No surprise here, sincere proposal comes out on top. That's the important thing, not the particular numbers. In fact, in real life Joe's utility function assigned negative moral value to insincerity, broadening the gap. But no matter; this did not make him sincere. The problem is that Joe was a classical causal decision theorist, and he believed that if circumstances changed to render him unhappily married, he would necessarily try to leave her. Because of this possibility, he could not propose sincerely in the sense she desired. He could even appease himself by speculating causes2 for how Kate can detect his uncertainty and constrain his options, but that still wouldn't make him sincere.
Seeing expected value computations with adjustable probabilities for the problem can really help feel its robustness. It's not about to disappear. Certainties can be replaced with 95%'s and it all still works the same. It's a whole parametrized family of problems, not just one.
Joe's scenario feels strikingly similar to Newcomb's problem, and in fact it is: if we change some probabilities to 0 and 1, it's essentially isomorphic:
- If he proposes sincerely, she will say yes.
- If he proposes insincerely, she will say no and break up with him forever.
- If they marry, he is 90% likely to be happy, and 10% likely to be unhappy.
The analogue of the two boxes are marriage (opaque) and the option of leaving (transparent). Given marriage, the option of leaving has a small marginal utility of 10%·125 = 12.5 utilons. So "clearly" he should "just take both"? The problem is that he can't just take both. The proposed payout matrix would be:
Joe \ Kate | Say yes | Say no |
---|---|---|
Propose sincerely |
Marriage | Nothing significant |
Propose insincerely |
Marriage + option to leave | Nothing significant |
The "principal of (weak3) dominance" would say the second row is the better "option", and that therefore "clearly" Joe should propose insincerely. But in Newcomb some of the outcomes are declared logically impossible. If he tries to take both boxes, there will be nothing in the marriage box. The analogue in real life is simply that the four outcomes need not be equally likely.
So there you have it. Newcomb happens. Newcomb happened. You might be wondering, what did the real Joe do?
In real life, Joe actually recognized the similarity to Newcomb's problem, realizing for the first time that he must become updateless decision agent, and noting his 90% certainty, he self-modified by adopting a moral pre-commitment to never leaving Kate should they marry, proposed to her sincerely, and the rest is history. No joke! That's if Joe's account is accurate, mind you.
Footnotes:
1 This is not a social commentary, but an illustration that probabilistic Newcomblike scenarios can and do exist. Although this also does not hinge on whether you believe Joe's account, I have provided it as-is nonetheless.
2 If you care about causal reasoning, the other half of what's supposed to make Newcomb confusing, then Joe's problem is more like Kavka's (so this post accidentally shows how Kavka and Newcomb are similar). But the distinction is instrumentally irrelevant: the point is that he can benefit from decision mechanisms that are evidential and time-invariant, and you don't need "unreasonable certainties" or "paradoxes of causality" for this to come up.
3 Newcomb involves "strong" dominance, with the second row always strictly better, but that's not essential to this post. In any case, I could exhibit strong dominance by removing "if they do get married" from Kate's proposal requirement, but I decided against it, favoring instead the actual account of events.
It's an interesting situation, and I can see the parallel to Newcombe's Problem. I'm not certain that it's possible for a person to self-modify to the extent that he will never leave his wife, ever, regardless of the very real (if small) doubts he has about the relationship right now. I don't think I could ever simultaneously sustain the thoughts "There's about a 10% chance that my marriage to my wife will make me very unhappy" and "I will never leave her no matter what". I could make the commitment financially - that, even if the marriage turns awful, I will still provide the same financial support to her - but not emotionally. If Joe can modify his own code so that he can do that, that's very good of him, but I don't think many people could do it, not without pre-commitment in the form of a marital contract with large penalties for divorce, or at least a very strong mentality that once the vows are said, there's no going back.
Perhaps the problem would be both more realistic and more mathematically tractable if "sincerity" were rated between 0 and 1, rather than being a simple on/off state? If 1 is "till death do us part" and 0 is "until I get a better offer", then 0.9 could be "I won't leave you no matter how bad your cooking gets, but if you ever try to stab me, I'm out of here". Then Kate's probability of accepting the proposal could be a function of sincerity, which seems a much more reasonable position for her.
Could this be an example where rationality and self-awareness really do work against an actor? If Joe were less self-aware, he could propose with complete sincerity, having not thought through the 10% chance that he'll be unhappy. If he does become unhappy, he'd then feel justified in this totally unexpected change inducing him to leave. The thing impeding Joe's ability to propose with full sincerity is his awareness of the possibility of future unhappiness.
Also, it's worth pointing out that, by the formulation of the original problem, Kate expects Joe to stay with her even if she is causing him -125 megautilons of unhappiness by forcing him to stay. That seems just a touch selfish. This is something they should talk about.