= 27d567bc2d48d9f40e9ccc20ea370b15)
Newcomb versus dust specks
You're given the option to torture everyone in the universe, or inflict a dust speck on everyone in the universe. Either you are the only one in the universe, or there are 3^^^3 perfect copies of you (far enough apart that you will never meet.) In the latter case, all copies of you are chosen, and all make the same choice. (Edit: if they choose specks, each person gets one dust speck. This was not meant to be ambiguous.)
As it happens, a perfect and truthful predictor has declared that you will choose torture iff you are alone.
What do you do?
How does your answer change if the predictor made the copies of you conditional on their prediction?
How does your answer change if, in addition to that, you're told you are the original?
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Comments (104)
Meta: I thought posting in Main was disabled?
[EDITED: so far as I know, it still is; this post is in fact in Discussion rather than Main; I was misled by a quirk of the LW user interface. I think following that link may also illustrate what confused me.]
I posted this to discussion.
Hmm, so you did. It turns out that
... which is how I came to think your post was in Main rather than Discussion. Sorry about that.
IMO since people are patterns (and not instances of patterns), there's still only one person in the universe regardless of how many perfect copies of me there are. So I choose dust specks. Looks like the predictor isn't so perfect. :P
This doesn't seem very coherent.
OK. Then that means if I choose torture, I am alone. If I choose the dust specks, I am not alone. I don't want to be tortured, and don't really care about 3 ^^^ 3 people getting dust specks in their eyes, even if they're all 'perfect copies of me'. I am not a perfect utilitarian.
A perfect utilitarian would choose torture though, because one person getting tortured is technically not as bad from a utilitarian point of view as 3 ^^^ 3 dust specks in eyes.
3^^^3 dust specks in everybody's eye?
So basically we're talking about turning all sentient life into black holes, or torturing everybody?
I mean, it depends on how good the torture we're talking about is, and how long it will last. If it's permanent and unchanging, eventually people will get used to it/evolve past it and move on. If it's short-term, eventually people will get past it. So in either of those cases, torture is the obvious choice.
If, on the other hand, it's permanent and adaptive such that all life is completely and totally miserable for perpetuity, and there is nothing remotely good about living, oblivion seems the obvious choice.
This seems like a weird mishmash of other hypotheticals on the site, I'm not really seeing the point of parts of your scenario.
I think the point may be: LW orthodoxy, in so far as there is such a thing, says to choose SPECKS over TORTURE [EDITED to add:] ... no, wait, I mean the exact opposite, TORTURE over SPECKS ... and ONE BOX over TWO BOXES, and that combining these in ike's rather odd scenario leads to the conclusion that we should prefer "torture everyone in the universe" over "dust-speck everyone in the universe" in that scenario, which might be a big enough bullet to bite to make some readers reconsider their adherence to LW orthodoxy.
My own view on this, for what it's worth, is that all my ethical intuitions -- including the one that says "torture is too awful to be outweighed by any number of dust specks" and the one that says "each of these vastly-many transitions from which we get from DUST SPECKS to TORTURE is a strict improvement" -- have been formed on the basis of experiences (my own, my ancestors', earlier people in the civilization I'm a part of) that come nowhere near to this sort of scenario, and I don't trust myself to extrapolate. If some incredibly weird sequence of events actually requires me to make such a choice for real then of course I'll have to make it (for what it's worth, I think I would choose TORTURE and ONE BOX in the separate problems and DUST SPECKS in this one, the apparent inconsistency notwithstanding, not least because I don't think I could ever actually have enough evidence to know something was a truly perfect truthful predictor) but I think its ability to tell me anything insightful about my values, or about the objective moral structure of the universe if it has one, is very very very doubtful.
No, Eliezer and Hanson are anti-specks.
Wow, did I really write that? It's the exact opposite of what I meant. Will fix.
I think your explanation may be correct, but I don't understand why torture would be the intuitive answer even so. First, if I select torture, everyone in the universe gets tortured, which means I get tortured. If instead I select dust speck, I get a dust speck, which is vastly preferable. Second, I would prefer a universe with a bunch of me to one with just me, because I'm pretty awesome so more me is pretty much just better. Basically I just fail to see a downside to the dust speck scenario.
The downside to the dust speck scenario is that lots and lots and lots of you get dust-specked. But yes, I think the thought experiment is seriously impaired by the fact that the existence of more copies of you is likely a bigger deal than whether they get dust-specked.
Perhaps we can fix it, as follows: Omega has actually set up two toy universes, one with 3^^^^3 of you who may or may not get dust-specked, one with just one of you who may or may not get tortured. Now Omega tells you the same as in ike's original scenario, except that it's "everyone sharing your toy universe" who will be either tortured or dust-specked.
The idea was that your choice doesn't change the number of people, so this shouldn't affect the answer.
That seems, if you don't mind my saying so, an odd thing to say when discussing a version of Newcomb's problem. ("Your choice doesn't change what's in the boxes, so ...")
In the first version, there's no causal relation between your choice and the number of people in the world. In the third, there is, and in the middle one, anthropics must also be considered.
I gave multiple scenarios to make this point.
If the predictor in newcomb doesn't touch the boxes but just tells you that they predict your choice is the same as what's in the box, it turns into the smoking lesions scenario.
Specks is supposed to be the intuitive answer.
That's why I gave scenarios where your choice doesn't cause the number of people, which is where Newcomblike scenarios come in.
Well I personally don't want to be tortured, so I choose the dust speck.
Even if I wasn't personally involved, and I was to decide on morality alone rather than personal interest, average utilitarianism tells me that I should choose the dust speck. (Better that 100% of all people suffer from a dust speck, than 100% of all people suffer from torture)
Do you generally endorse average utilitarianism? E.g., if you can press a button to create a new world, completely isolated from all others, containing 10^10 people 10x happier than typical present-day Americans, do you press it if what currently exists is a world with 10^10 people only 9x happier than typical present-day Americans and refrain from pressing it if it's 11x instead?
The answer is complex
First of all, the creation of people is a complex moral decision. Whether you espouse average utilitarianism or total utilitarianism or whatever other decision theory, if you ask someone "Would you press a button that would create a person", they'd normally be HESITANT, no matter whether you said it would be a very happy person or a moderately happy person. We tend to think of creating people as a big deal, that brings a big responsibility.
Secondly, my average utilitarianism is about the satisfaction of preferences, not happiness. This may seem a nitpick, though.
Thirdly, I can't help but notice that you're using the example of the creation of a world that in reality would increase average utility, even as you're using a hypothetical that states that in that particular case it would decrease average utility. This feels as a scenario designed to confuse the moral intuition into giving the wrong answer.
So using the current reality instead (rather than the one where people are 9x happier): Would I choose to create another universe happier than this one? In general, yes. Would I choose to create another universe, half as happy as this one? I general, no, not unless there's some additional value that the presence of that universe would provide to us, enough so that it would make up for the loss in average utility.
True enough. But it seems to me that hesitation in such cases is usually because of uncertainty either about whether the new people would really have good lives or about their effect on others around them. In the scenarios I described, everyone involved gets a good life when ask their interactions with others are taken into account. So yeah, creating livres is complex, but I don't see that that invalidates my question at all.
That happens to be my, er, preference too. I think I do think it's a nitpick; we can just take "10x happier" as a sort of shorthand for some corresponding statement about preferences.
I promise I had absolutely no such intention. I took the levels higher than typical ones in our world to avoid distracting digressions about whether the typical life in our world is in fact better than nothing. (Note that this isn't the same question as whether it's worth continuing such a life once it's already in progress.)
Your example of a world half as happy as this seems like it has a similar but opposite problem: depending on what "half as happy" actually means, you might be describing a change that would be rejected by total utilitarianism as well as average. That's the problem I was trying to avoid.
Okay, Now I reveal that just yesterday, we've discovered yet another universe which already exists and is a lot happier than the one you would choose to create. In fact it's so much happier that creating that universe would now drive the average down instead of up.
If you're using average utility, then whether this discovery has been made affects whether you want to create that other universe. Is that correct?
With the standard caveats, yes that seems reasonable. Given the existence of that ultrahappy universe an average human life will be more likely to exist in happier circumstances than the ones in the multiversal reality I'd create if I chose to add that less-than-averagely-happy universe.
Same way as I'd not take 20% of actual existing happy people and force them to live less happy lives.
Think about all sentient lives as if they were part of a single mind, called "Sentience". We design portions of Sentience's life. We want as much a proportion of Sentience's existence to be as happy as possible, satisfying Sentience's preferences.
The way the problem reads to me, choosing dust specks means I live in a universe where 3^^^3 of me exist, and choosing torture means 1 of me exist. I prefer that more of myself exist than not, so I should choose specks in this case.
In a choice between "torture for everyone in the universe" and "specks for everyone in the universe", the negative utility of the former obviously outweighs that of the latter, so I should choose specks.
I don't see any incongruity or reason to question my beliefs? I suppose it's meant to be implied that it's other selves that exist because of the size of the universe, so there's either one of "everyone in the universe" or 3^^^3 copies of everyone, but in that case my other selves are too far outside my light-cone for "iff you are alone" to be a prediction that makes sense.
For the case that dust specks aren't additive, assuming we treat copies of me as distinct entities with distinct moral weight, 3^^^3 copies of me is either a net negative - as a result of 3^^^3 lives not worth living - or a net positive - as a result of an additional 3^^^3 lives worth living. The point of the dust speck is that it has only a negligible effect; the weight of the dust speck moral issue is completely subsumed by the weight of the duplicate people issue.
If we don't treat them as distinct moral entities, well, the duplication and the dust speck doesn't enter into it.
I don't think your conceptual problem sufficiently isolates whatever moral quandary you're trying to express; there's just too much going on here.
If you smoke in the smoking lesions scenario, then you shouldn't choose your action here based on how many people would exist, because they would exist anyway. (At least in the first of three cases.)
Either you misunderstand the smoking lesions scenario and the importance between the difference between a correlation and a perfect predictor, or you're just trolling the board by throwing every decision theory edge case you can think of into a single convoluted mess.
I may be misunderstanding something, but isn't the standard LW position on smoking to smoke even if the gene's correlation to smoking and cancer is 1?
As long as the predictor doesn't cause anything but merely informs, they're equivalent to the gene. The reason why one-boxing is correct is because your choice causes the money, while the reason smoking is correct is because your choice doesn't cause cancer.
We've had this discussion before. When you one-box, your choice does not cause the money. The money is already there or it is not. Causality does not go backwards in time.
In other words, Newcomb and the smoking lesion are identical in logical form.
Your decision algorithm will cause the choice. The prediction of that choice, by someone knowing your decision algorithm, will have caused money.
If you want the money you should therefore be a decision algorithm that makes the choice whose prediction will cause the money.
You cannot make yourself into a certain decision algorithm, just as you cannot make yourself have or not have a lesion.
What, is this some sort of objection where you believe that determinism means we don't make 'real' choices'?
You could be convinced by my words and make yourself into a person who chooses to one-box. Or you could refuse to be convinced and remain a person who chooses to two-boxes.
Granted, by being "convinced" or "not convinced" it means that you're already the decision algorithm that would make that choice. So what? Whether you'll be convinced or not still affects your decision algorithm from then on.
No, I don't believe that determinism means we don't make real choices. But it is also true, as you note yourself, that if I am convinced by your words, then I was already the kind of person who would be convinced, and I did not make myself into that sort of person. And likewise for the opposite case.
But I am consistent: I believe we make real choices even if Omega predicts our actions, and I also believe we make real choices even if a lesion causes them. The people arguing against my position are saying we don't make real choices in the second case, so they are the ones raising the determinism objection.
Okay, can you just state clearly whether you one-box or two-box, and whether you smoke or not-smoke in the smoking lesion problem, so that I understand what your position is, before trying to understand why it is?
I'm referring to TDT, which disagrees.
Eliezer disagrees, but no formal decision theory disagrees, because the two situations are formally identical.
They're formally identical only if you consider the choice to not counterfactually affect the outcome. Asserting that counterfactuals don't go backwards in time makes the choice not affect it, but that's just question begging.
It hasn't been formalized because we don't know how to deal with logical uncertainty fully yet.
If I have the 100% version of the lesion, it is true to say, "If I had decided not to smoke, I would not have had the lesion," because that is the only way I could have decided not to smoke, in the same way that in Newcomb it is true to say, "If I had picked one-box, I would have been a one-boxer," because that is the only way I could have picked one box.
In one there's counterfactual dependence and in the other there isn't. If your model doesn't take into account counterfactuals then you can't even tell the difference between smoking lesions and the case where smoking really does cause cancer.
If the mutual correlation to both is 1, you will smoke if and only if you have the gene, and you will have the gene if and only if you smoke, and in which case you shouldn't smoke. At the point at which the gene is a perfect predictor, if you have a genetic test and you don't have the gene, and then smoke - then the genetic test produced a false negative. Perfect predictors necessarily make a mess of causality.
This implicitly assumes EDT.
But that's not what CDT counterfactuals do. You cut off previous nodes. As the choice to smoke doesn't causally affect the gene, smoking doesn't counterfactually contradict the prediction. If you would actually smoke, then yes, but counterfactuals don't imply there's any chance of it happening in reality.
No it doesn't. It assumes a "perfect predictor" is what it is. I don't give a damn about evidence - we're specifying properties of a universe here.
CDT assumes causality makes sense in the universe. Your hypotheticals don't take place in a universe with the kind of causality causal decision theory depends upon.
In the case of a perfect predictor, yes, smoking specifies which gene you have. You don't get to say "Everybody who smokes has this gene" as a property of the universe, and then pretend to be an exception to a property of the universe because you have a bizarre and magical agency that gets to bypass properties of the universe. You're a part of the universe; if the universe has a law (which it does, in our hypotheticals), the law applies to you, too.
We have a perfect predictor. We do something the perfect predictor predicted we wouldn't. There is a contradiction there, in case you didn't notice; either it's not, in fact, the perfect predictor we specified, or we didn't do the thing. One or the other. And our hypothetical universe is constructed such that the perfect predictor is a perfect predictor; therefore, we don't get to violate its predictions.
You said "you shouldn't smoke", which is a decision-theoretical claim, not a specification. It's consistent with EDT, but not CDT.
In other words, you're denying the exact thing that CDT asserts.
Which is what a counterfactual is.
Whatever your theory is, it is denying core claims that CDT makes, so you're denying CDT (and implicitly assuming EDT as the method for making decisions, your arguments literally map directly onto EDT arguments).
No it isn't, it's a statement about the universe: If you smoke, you'll get lesions. It's written into the specification of the universe; what decision theory you use doesn't change the characteristics of the universe.
No. You don't get to specify a universe without the kind of causality that the kind of CDT we use in our universe depends on, and then claim that this says something significant about decision theory. Causality in our hypothetical works differently.
No it isn't.
No it isn't. In terms of CDT, we can say that smoking causes the gene; this isn't wrong, because, according to the universe, anybody who smokes has the gene; if they didn't, they do now, because the correlation is guaranteed by the laws of the universe. No matter how much work you prepared to ensure you didn't have the gene in advance of smoking, the law of the universe says you have it now. No matter how many tests you ran, they were all wrong.
It may seem unintuitive and bizarre, because our own universe doesn't behave this way - but when you find yourself in an alien universe, stomping your foot and insisting that the laws of physics should behave the way you're used to them behaving is a fast way to die. Once you introduce a perfect predictor, the universe must bend to ensure the predictions work out.
What kind of causality is this, given that you assert that the correct thing to do in smoking lesions is refrain from smoking, and smoking lesions is one of the standard things where CDT says to smoke?
"A causes B, therefore B causes A" is a fallacy no matter what arguments you put forward.
CDT asserts the opposite, and so if you claim this then you disagree with CDT.
You don't understand what counterfactuals are.
"If you have a genetic test and you don't have the gene, and then smoke - then the genetic test produced a false negative."
If Omega makes the mistake of telling someone else that he predicted that you will one-box, and that person tells you, so you then take both boxes, knowing that the million is already there, then Omega's prediction was wrong.
Omega can be a perfect predictor, but he cannot tell you his prediction, at least not if you work the way normal humans do. Likewise, a gene could be a perfect predictor, but not if you know about it, at least not if you work the way normal humans do.
Trial problem:
Omega appears before you, and gives you a pencil. He tells you that, in universes in which you break this pencil in half in the next twenty seconds, the universe ends immediately. Not as a result of your breaking the pencil - it's pure coincidence that all universes in which you break the pencil, the universe ends, and in all universes in which you don't, it doesn't.
Do you break the pencil in half? It's not like you're changing anything by doing so, after all; some set of universes will end, some set won't, and you aren't going to change that.
You're just deciding which set of universes you happen to occupy. Which implies something.
I don't break the pencil. But I already pointed out in Newcomb and in the Smoking Lesion that I don't care if I can change anything or not. So I don't care here either.
It makes a huge difference whether the dust speck choices add up or not. If they do, OrphanWilde's objection applies and the only path to survival is to be tortured.
If they don't, so each one of me gets one dust speck total, then dust specks for sure. All of the copies of me (whether there are one or 3^^^3 of us) are experiencing what amounts to a choice between individually being dust-specked or individually being tortured. We get what we ask for either way, and no one else is actually impacted by the choice.
There's no need to drag average utilitarianism in.
Computational theory of identity so some large number of exact copies of the same individual experiencing the same thing don't sum, they only count as once instance?
That too. But my reasoning holds in the more general case, where instead of it being 3^^^3 copies of me, it was 3^^^3 entities from the pool of people who would choose specks.
I choose torture if and only if I'm alone. Otherwise the predictor would be wrong, contrary to the assumptions of the hypothetical. But I'd rather be in the world where dust specks gets chosen.
You don't know whether you're alone.
Doesn't matter - I'll still end up doing it, regardless of what algorithm I try to implement!
It is true in general that you will end up implementing the algorithm that you actually are. That doesn't mean you don't have to make a decision.