= 27d567bc2d48d9f40e9ccc20ea370b15)
The Smoking Lesion: A problem for evidential decision theory
This is part of a sequence titled "An introduction to decision theory". The previous post was Newcomb's Problem: A problem for Causal Decision Theories
For various reasons I've decided to finish this sequence on a seperate blog. This is principally because there were a large number of people who seemed to feel that this sequence either wasn't up to the Less Wrong standard or felt that it was simply covering ground that had already been covered on Less Wrong.
The decision to post it on another blog rather than simply discontinuing it came down to the fact that other people seemed to feel that the sequence had value. Those people can continue reading it at "The Smoking Lesion: A problem for evidential decision theory".
Alternatively, there is a sequence index available: Less Wrong and decision theory: sequence index
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Comments (100)
I see you've moved this discussion off-site. FWIW, I commend you for trying to organize the various decision theory issues into a more accessible and organized sequence. I'd like to suggest that you take some of this and use it to improve the (almost comically sparse) decision theory articles on the LW Wiki. If that's really going to be the go-to place for LW knowledge, your efforts to summarize and present this info could really be useful there, and any redundancy with existing blog posts would be a non-issue.
I'm confused as to why you said you weren't continuing this on Less Wrong, then posted it on Less Wrong.
The conclusion turned out to be that some people thought it should be posted here and some didn't. GIven that some people found the sequence useful it would have been silly to not give them a chance to read it.
There are three more posts in the sequence so by doing it this way I cut down on the amount that I bombard Less Wrong with the post (I agree that your reasoning would make sense if I made a post which linked to only one post but this post links to three posts - ie. once you read the first post, at the bottom of that it links to the next and so on).
I've read the smoking lesion thing before, and what occurred to be is that even under EDT, the reasoning in there is wrong. What I mean was that one shouldn't simply reason it out by comparing to the average stats, but take into account the fact that they're using EDT itself. ie, they should say "given that a person is using EDT, then what's the correlation between etc etc..."
Yes, this is correct. However, in principle there could still be a correlation when people used EDT. That was my point and that would make the case equivalent to Newcomb.
If the population of EDT rationalists was sufficiently large however, the correlation would necessarily be small enough that, for those with the largest desire to smoke, it would still be rational to smoke, even within the EDT paradigm.
Note: In your 100% scenario, it is actually perfectly rational to smoke if you desire to do so; your desire to do so is perfect evidence that you have the lesion and hence the decision to smoke provides no further evidence.
In the 100% scenario, it is not rational to smoke even if you desire to do so. The desire is evidence that you have the lesion, but not conclusive evidence. The decision to smoke would be conclusive evidence, while the decision not to smoke is conclusive evidence that you don't have the lesion. So you shouldn't smoke.
If that was true, then the correlation among people using EDT wouldn't even be close to 100%, what with the fact that people can't do something that's irrational under EDT while actually obeying EDT.
So either it's impossible to use EDT on this subject if you have the lesion, or it must be rational to smoke under EDT if you have the lesion.
Do you not see how creating contradictions is a problem?
It's not impossible to use EDT if you have the lesion. You use EDT, and you don't smoke. But then after a while, you change your mind, without using EDT, and start smoking. So you're right to the extent that if you have the lesion, you won't consistently use EDT at all times. This is no different from any other decision theory: people don't use them consistently in real life.
Worth referencing:
The Smoking Lesion on the wiki.
Timeless Decision Theory and Meta-Circular Decision Theory, where Eliezer discusses this problem (among others)
(By the way, your blog has some interesting posts!)
Thanks. I added the two links suggested by the way. Should be following up on the causality post in the next few days.
I don't understand why the Smoking Lesion is a problem for evidential decision theory. I would simply accept that in the scenario given, you shouldn't smoke. And I don't see why you assert that this doesn't lessen your chances of getting cancer, except in the same sense that two-boxing doesn't lessen your chances of getting the million.
I would just say: in the scenario give, you should not smoke, and this will improve your chances of not getting cancer.
If you doubt this, consider if the correlation were known to be 100%; every person who ever smoked up till now, had the lesion and developed cancer, while every person who did not smoke, did not have the lesion. This was true also of people who knew about the Lesion. Do you still say it's a good idea to smoke?
You're correct that if the correlation were known to be 100% then the only meaningful advice one could give would be not to smoke. However, it's important to understand that "100% correlation" is a degenerate case of the Smoking Lesion problem, as I'll try to explain:
Imagine a problem of the following form: Y is a variable under our control, which we can either set to k or -k for some k >= 0 (0 is not ruled out). X is an N(0, m^2) random variable which we do not observe, for some m >= 0 (again, 0 is not ruled out). Our payoff has the form (X + Y) - 1000(X + rY) for some constant r with 0 <= r <= 1. Working out the optimal strategy is rather trivial. But anyway, in the edge cases: If r = 0 we should put Y = k and if r = 1 we should put Y = -k.
Now I want to say that the case r = 0 is analogous to the Smoking Lesion problem and the case r = 1 is analogous to Newcomb's problem (with a flawless predictor):
ETA: Perhaps this analogy can be developed into an analysis of the original problems. One way to do it would be to define random variables Z and W taking values 0, 1 such that log(P(Z = 1 | X and Y)) / log(P(Z = 0 | X and Y)) = a linear combination of X and Y (and likewise for W, but with a different linear combination), and then have Z be the "player's decision" and W be "Omega's decision / whether person gets cancer". But I think the ratio of extra work to extra insight would be quite high.
If the correlation is 100%, it doesn't mean that you can choose whether or not you'll have cancer. It means that if you have the lesion, then some combination of logic, rationalisation or impulse will make you decide to smoke (and if you don't, then similarly you'll end up not smoking). You can then tell from your decision whether you'll get cancer or not, but you couldn't have made the other decision, no matter what.
(Either that, or you can be the first person to try using EDT for it, and that way you get to be the person who breaks the 100% correlation and gets cancer without smoking)
You can say the same thing about Newcomb's problem. It doesn't mean you can choose whether or not there will be a million in one of the boxes. It means that if there is a million in one of the boxes, then "some combination of logic, rationalisation or impulse will make you decide" to choose only one of the boxes (and if there's no million, then similarly you'll end up taking both boxes.) "You can then tell from your decision whether" you'll get the million or not, "but you couldn't have made the other decision, no matter what."
Either that, or you can be the first to outguess Omega and get the million as well as the thousand...
Nope, this reasoning doesn't work with Newcomb, and it doesn't work with the Smoking Lesion. If you want to win, you one-box, and you don't smoke.
Yes I can, right now.
I think the difference is that your disposition to one-box or two-box is something you can decide to change. Whether you were born with a lesion is not.
When you are standing there, and there is either a million in the box or there isn't, can you change whether or not there is a million in the box?
No, no more than whether you were born with a lesion or not. The argument, "I should smoke, because if I have the lesion I have it whether or not I smoke" is exactly the same as the argument "I should take two boxes, because if the million is there, it is there whether or not I take two boxes."
I agree, insofar as I think "I should not smoke" is true as long as I'm also allowed to say "I should not have the lesion".
The problem is I think running into the proper use of 'should'. We'd need to draw very sharp lines around the things we pretend that we can or cannot control for purposes of that word.
Basically, you end up with a black-box concept containing some but not all of the machinery that led up to your decision such that words like 'should' and 'control' apply to the workings of the black box and not to anything else. And then we can decide whether it's sensible to ask 'should I smoke' in Smoking Lesion and 'should I one-box' in Newcomb.
Right now I don't have a good enough handle on this model to draw those lines, and so don't have an answer to this puzzle.
One potentially-significant difference: in Newcomb, it is precisely the fact that you're disposed to two-box that causes you to lose out. (Omega is detecting and responding to this very disposition.) In Smoking Lesion, the disposition to smoke is intrinsically harmless; it merely happens to be correlated (due to a common cause) with a disposition to get cancer.
(But if you're right that the two cases are on a par, then that would significant increase my confidence that two-boxing is rational. The smoking lesion case is by far the more obvious of the two.)
Responding to the supposed difference between the cases:
Omega puts the million in the box or not before the game has begun, depending on your former disposition to one-box or two-box.
Then the game begins. You are considering whether to one-box or two-box. Then the choice to one-box or two-box is intrinsically harmless; it merely happens to be correlated with your previous disposition and with Omega's choice. Likewise, your present disposition to one-box or two-box is also intrinsically harmless. It is merely correlated with your previous disposition and with Omega's choice.
You can no more change your previous disposition than you can change whether you have the lesion, so the two cases are equivalent.
And if people's actions are deterministic, then in theory there could be an Omega that is 100% accurate. Nor would there be a need for simulation; as cousin_it has pointed out, it could "analyze your source code" and come up with a proof that you will one-box or two-box. In this case the 100% correlated smoking lesion and Newcomb would be precisely equivalent. The same is true if each has a 90% correlation, and so on.
If some subset of the information contained within you is sufficient to prove what you will do, simulating that subset is a relevant simulation of you.
I'm not sure what kind of proof you could do without going through the steps such that you essentially produced a simulation.
Could you give an example of the type of proof you're proposing, so I can judge for myself whether it seems to involve running through the relevant steps?
See cousin_it's post: http://lesswrong.com/lw/2ip/ai_cooperation_in_practice/
Many programs can be proven to have a certain result without any simulation, not even of a subset of the information. For example, think of a program that discovers the first 10,000 primes, increasing a counter by one for each prime it finds, and then stops. You can prove that the counter will equal 10,000 when it stops, without simulating this program.
See, to me that is a mental simulation of the relevant part of the program.
The counter will increase, point by point, it will remain an integer at each point and pass through every integer, and upon reaching 10,000 this will happen.
The fact that the relevant part of the program is as ridiculously simple as a counter just means that the simulation is easy.
So would you smoke even if the previous correlation were 100%, and included those who knew about the Lesion?
This could happen in reality, if everyone who smoked, smoked because he wanted to, and if everyone who sufficiently desired it did so, and if the sufficient desire for smoking was completely caused by the lesion. In other words, by choosing to smoke, you would be showing that you had sufficient desire, and therefore the lesion, and by choosing not to smoke, you would be showing that you did not have sufficient desire, and therefore not the lesion.
Under these circumstances, if you chose not to smoke, would you expect to get cancer, since you knew that you had some desire for smoking? (Presumably whether the desire was sufficient or not would not be evident to introspection, but only from whether or not you ended up smoking.) Or choosing to smoke, would you expect not to get cancer, since you say it doesn't make any difference to whether you have the lesion?
For the correlation to be 100%, smoking would have to be ABSOLUTELY IRRESISTIBLE to people with the lesion.
Hence, if I had the lesion, I would smoke. I wouldn't be able to resist doing so.
And of course smoking would have to be ABSOLUTELY UNTHINKABLE for people without the lesion.
Hence, if I didn't have the lesion, I wouldn't smoke, I wouldn't be able to even try it.
I think that the "ABSOLUTELY IRRESISTIBLE" and "ABSOLUTELY UNTHINKABLE" language can be a bit misleading here. Yes, someone with the lesion is compelled to smoke, but his experience of this may be experience of spending days deliberating about whether to smoke - even though, all along, he was just running along preprepared rails and the end-result was inevitable.
If we assume determinism, however, we might say this about any decision. If someone makes a decision, it is because his brain was in such a state that it was compelled to make that decision, and any other decision was "UNTHINKABLE". We don't normally use language like that, even if we subscribe to such a view of decisions, because "UNTHINKABLE" implies a lot about the experience itself rather than just implying something about the certainty of particular action or compulsion towards it.
I could walk to the nearest bridge to jump off, and tell myself all along that, to someone whose brain was predisposed to jumping off the bridge, not doing it was unthinkable, so any attempt on my part to decide otherwise is meaningless. Acknowledging some kind of fatalism is one thing, but injecting it into the middle of our decision processes seems to me to be asking for trouble.
For the correlation with Omega to be 100%, one-boxing would have to be ABSOLUTELY IRRESISTABLE when there was a million in the box...
Hence, if there was a million, the person would one-box. He wouldn't be able to resist doing so...
And of course taking only one box would have to be ABSOLUTELY UNTHINKABLE for people when the million wasn't there.
And so on.
Well, yeah, which is why people resist the story about Omega, think it must be nonsense, and decide to two-box (although it would be better to explicitly reject the story). Or interpret it to imply backwards causality (in which case even CDT makes you one-box) or something else that violates the laws of physics as I know them.
This is one reason to stick with probabilistic versions of Newcomb's Paradox.
In both cases (Newcomb's Paradox and the Smoking Lesion), this seems to another example of the difficulty with 0 and 1 as probabilities.
Nope. In the Newcombian situation the lines of causality are different.
What's in the box is explicitly caused by what you will choose, whereas in the smoking lesion example they simply share a cause.
Different lines of causality, different scenario.
Every event has multiple causes, and what causes you point out is not such important as you seem to think. In Newcomb, Omega's decision and your one-or-two-boxing are both ultimately consequences of the state of the world before the scenario has started.
The only difference between Newcomb and the lesion is that in case of 100% effective lesion, there will be no correlation between having read about EDT and smoking. And in a world where there was such a correlation, one should start believing in fate.
I find that the term "cause" or "causality" can be very misleading in this situation.
As a matter of terminology, I actually agree with you: in lay speech, I see nothing wrong with saying that "One-boxing causes the sealed box to be filled", because this is exactly how we perceive causality in the world.
However, when speaking of these problems, theorists nail down their terminology as best they can. And in such problems, standard usage is such that the concept of causality only applies to cases where an event changes things solely in the future[1], not merely where it reveals you to be in a situation in which a past event has happened.
When speaking of decision-theoretic problems, it is important to stick to this definition of causality, counter-intuitive though it may be.
Another example of the distinction is in Drescher's Good and Real. Consider this: if you raise your hand (in a deterministic universe), you are setting the universe's state 1 billion years ago to be such that a chain of events will unfold in a way that, 1 billion years later, you will raise your hand. In a (lay) sense, raising your hand "caused" that state.
However, because that state is in the past, it violates decision-theoretic usage to say that you caused that state; instead, you should simply say that either:
a) there is an acausal relationship between your choice to raise your hand and that state of the universe, or
b) by choosing to raise your hand, you have learned about a past state of universe. (Just as deciding whether to exit in the Absent-Minded Driver problem tells you something about which exit you are at.)
[1] or, in timeless formalisms, where the cause screens off that which it causes.
I think you've misunderstood me. "What you will choose" is a fact that exists before omega fills the boxes.
This fact determines how the boxes are filled.
"What you will choose" (some people seem to refer to this, or something similar, as your "disposition", but I find my terminology more immediately apparent) causes the future event "how the boxes are filled"
Someone should wrap it up with a problem where what you choose is determined by what's in the box. Any ideas, anyone?
Actually, this is excellent. We could rewrite Newcomb's problem like this:
Omega places in the box together with the million or non-million, a device that influences your brain, programming the device so that you are caused to take both if it does not place the million, and programming the device so that you are caused to one-box if it places the million. In other words, Omega decides in advance whether you are going to get the million or not, then sets up the situation so you will make the choice that gets you what it wanted you to get.
However, the influence on your brain is quite subtle; to you, it still feels like you are deciding in the normal way, using some decision theory or other.
Now, do you one-box or two-box? This is certainly exactly the same as the smoking lesion. Nor can you answer "I don't have to decide because my actions are determined" because your actions might well be determined in real life anyway, and you still have to decide.
If you one-box here, you should not smoke in the lesion problem. If you don't one-box here... well, too bad for you.
No. What is in the box is not caused by what you will choose. It is caused by Omega after analyzing your original disposition, before the game begins. After you start the game, your choice and the million share a cause, namely your original disposition. So the cases are the same-- same lines of causality, same scenario.
You can no more change your original disposition (which causes the million), than you can change the lesion that causes cancer.
You can control your original disposition in exactly the same way you usually control your decisions. Even normally when you consider a decision the outcome is already settled and the measure of all Everett branches involved already determined. Just because you consider the counterfactual of local miracles that result in a different decision when evaluating your preferences doesn't mean any such local miracles actually happen. Your original disposition is caused by your preferences between the two "possible" actions, just like with any other decision. The lesion example is different because your preferences are at no point involved in the causal history of the cancer.
Even going on that basis, which I disagree with (I disagree with the "lack of simulation" hypothesis; see the other thread of comments in a second)
Right now, I could precommit myself to winning in all newcomb-like problems I encounter in future, and thus, right now, I can change my disposition.
I can't precommit to not finding something irresistable due to brain damage/lesions/whatever.
That's a pretty significant difference.