LESSWRONG
LW

Comment Permalink

You said "you shouldn't smoke", which is a decision-theoretical claim, not a specification. It's consistent with EDT, but not CDT.

In other words, you're denying the exact thing that CDT asserts.

No. You don't get to specify a universe without the kind of causality that the kind of CDT we use in our universe depends on, and then claim that this says something significant about decision theory. Causality in our hypothetical works differently.

Which is what a counterfactual is.

No it isn't.

Whatever your theory is, it is denying core claims that CDT makes, so you're denying CDT (and implicitly assuming EDT as the method for making decisions, your arguments literally map directly onto EDT arguments).

No it isn't. In terms of CDT, we can say that smoking causes the gene; this isn't wrong, because, according to the universe, anybody who smokes has the gene; if they didn't, they do now, because the correlation is guaranteed by the laws of the universe. No matter how much work you prepared to ensure you didn't have the gene in advance of smoking, the law of the universe says you have it now. No matter how many tests you ran, they were all wrong.

It may seem unintuitive and bizarre, because our own universe doesn't behave this way - but when you find yourself in an alien universe, stomping your foot and insisting that the laws of physics should behave the way you're used to them behaving is a fast way to die. Once you introduce a perfect predictor, the universe must bend to ensure the predictions work out.

ike9y00

You don't get to specify a universe without the kind of causality that the kind of CDT we use in our universe depends on, and then claim that this says something significant about decision theory.

"A causes B, therefore B causes A" is a fallacy no matter what arguments you put forward.

In terms of CDT, we can say that smoking causes the gene

C... (read more)

See in context

-2

Newcomb versus dust specks

by ike

12th May 2016

1 min read

104

-2

You're given the option to torture everyone in the universe, or inflict a dust speck on everyone in the universe. Either you are the only one in the universe, or there are 3^^^3 perfect copies of you (far enough apart that you will never meet.) In the latter case, all copies of you are chosen, and all make the same choice. (Edit: if they choose specks, each person gets one dust speck. This was not meant to be ambiguous.)

As it happens, a perfect and truthful predictor has declared that you will choose torture iff you are alone.

What do you do?

How does your answer change if the predictor made the copies of you conditional on their prediction?

How does your answer change if, in addition to that, you're told you are the original?

Personal Blog

-2

Newcomb versus dust specks

New Comment

104 comments, sorted by

top scoring

Click to highlight new comments since: Today at 11:01 PM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

[-]OrphanWilde9y80

3^^^3 dust specks in everybody's eye?

So basically we're talking about turning all sentient life into black holes, or torturing everybody?

I mean, it depends on how good the torture we're talking about is, and how long it will last. If it's permanent and unchanging, eventually people will get used to it/evolve past it and move on. If it's short-term, eventually people will get past it. So in either of those cases, torture is the obvious choice.

If, on the other hand, it's permanent and adaptive such that all life is completely and totally miserable for perpetuity, and there is nothing remotely good about living, oblivion seems the obvious choice.

[-]gjm9y40

Meta: I thought posting in Main was disabled?

[EDITED: so far as I know, it still is; this post is in fact in Discussion rather than Main; I was misled by a quirk of the LW user interface. I think following that link may also illustrate what confused me.]

0ike9y

I posted this to discussion.

8gjm9y

Hmm, so you did. It turns out that * for any Discussion-section post under /r/discussion/lw/[code]/[title], the exact same post appears at /lw/[code]/[title] (and, when viewed there, looks as if it is in Main because the "MAIN" at the top is boldfaced and the "DISCUSSION" is not); * when you get a reply to a comment on a post in the Discussion section, in your inbox the "Context" link attached to that reply goes to the "Main-ified" version. ... which is how I came to think your post was in Main rather than Discussion. Sorry about that.

[-]HungryHobo9y30

This seems like a weird mishmash of other hypotheticals on the site, I'm not really seeing the point of parts of your scenario.

9gjm9y

I think the point may be: LW orthodoxy, in so far as there is such a thing, says to choose SPECKS over TORTURE [EDITED to add:] ... no, wait, I mean the exact opposite, TORTURE over SPECKS ... and ONE BOX over TWO BOXES, and that combining these in ike's rather odd scenario leads to the conclusion that we should prefer "torture everyone in the universe" over "dust-speck everyone in the universe" in that scenario, which might be a big enough bullet to bite to make some readers reconsider their adherence to LW orthodoxy. My own view on this, for what it's worth, is that all my ethical intuitions -- including the one that says "torture is too awful to be outweighed by any number of dust specks" and the one that says "each of these vastly-many transitions from which we get from DUST SPECKS to TORTURE is a strict improvement" -- have been formed on the basis of experiences (my own, my ancestors', earlier people in the civilization I'm a part of) that come nowhere near to this sort of scenario, and I don't trust myself to extrapolate. If some incredibly weird sequence of events actually requires me to make such a choice for real then of course I'll have to make it (for what it's worth, I think I would choose TORTURE and ONE BOX in the separate problems and DUST SPECKS in this one, the apparent inconsistency notwithstanding, not least because I don't think I could ever actually have enough evidence to know something was a truly perfect truthful predictor) but I think its ability to tell me anything insightful about my values, or about the objective moral structure of the universe if it has one, is very very very doubtful.

1polymathwannabe9y

No, Eliezer and Hanson are anti-specks.

6gjm9y

Wow, did I really write that? It's the exact opposite of what I meant. Will fix.

19eB19y

I think your explanation may be correct, but I don't understand why torture would be the intuitive answer even so. First, if I select torture, everyone in the universe gets tortured, which means I get tortured. If instead I select dust speck, I get a dust speck, which is vastly preferable. Second, I would prefer a universe with a bunch of me to one with just me, because I'm pretty awesome so more me is pretty much just better. Basically I just fail to see a downside to the dust speck scenario.

4gjm9y

The downside to the dust speck scenario is that lots and lots and lots of you get dust-specked. But yes, I think the thought experiment is seriously impaired by the fact that the existence of more copies of you is likely a bigger deal than whether they get dust-specked. Perhaps we can fix it, as follows: Omega has actually set up two toy universes, one with 3^^^^3 of you who may or may not get dust-specked, one with just one of you who may or may not get tortured. Now Omega tells you the same as in ike's original scenario, except that it's "everyone sharing your toy universe" who will be either tortured or dust-specked.

0ike9y

The idea was that your choice doesn't change the number of people, so this shouldn't affect the answer.

3gjm9y

That seems, if you don't mind my saying so, an odd thing to say when discussing a version of Newcomb's problem. ("Your choice doesn't change what's in the boxes, so ...")

0ike9y

In the first version, there's no causal relation between your choice and the number of people in the world. In the third, there is, and in the middle one, anthropics must also be considered. I gave multiple scenarios to make this point. If the predictor in newcomb doesn't touch the boxes but just tells you that they predict your choice is the same as what's in the box, it turns into the smoking lesions scenario.

0ike9y

Specks is supposed to be the intuitive answer. That's why I gave scenarios where your choice doesn't cause the number of people, which is where Newcomblike scenarios come in.

[-]ArisKatsaris9y20

Well I personally don't want to be tortured, so I choose the dust speck.

Even if I wasn't personally involved, and I was to decide on morality alone rather than personal interest, average utilitarianism tells me that I should choose the dust speck. (Better that 100% of all people suffer from a dust speck, than 100% of all people suffer from torture)

6gjm9y

Do you generally endorse average utilitarianism? E.g., if you can press a button to create a new world, completely isolated from all others, containing 10^10 people 10x happier than typical present-day Americans, do you press it if what currently exists is a world with 10^10 people only 9x happier than typical present-day Americans and refrain from pressing it if it's 11x instead?

0ArisKatsaris9y

The answer is complex * First of all, the creation of people is a complex moral decision. Whether you espouse average utilitarianism or total utilitarianism or whatever other decision theory, if you ask someone "Would you press a button that would create a person", they'd normally be HESITANT, no matter whether you said it would be a very happy person or a moderately happy person. We tend to think of creating people as a big deal, that brings a big responsibility. * Secondly, my average utilitarianism is about the satisfaction of preferences, not happiness. This may seem a nitpick, though. * Thirdly, I can't help but notice that you're using the example of the creation of a world that in reality would increase average utility, even as you're using a hypothetical that states that in that particular case it would decrease average utility. This feels as a scenario designed to confuse the moral intuition into giving the wrong answer. So using the current reality instead (rather than the one where people are 9x happier): Would I choose to create another universe happier than this one? In general, yes. Would I choose to create another universe, half as happy as this one? I general, no, not unless there's some additional value that the presence of that universe would provide to us, enough so that it would make up for the loss in average utility.

4gjm9y

True enough. But it seems to me that hesitation in such cases is usually because of uncertainty either about whether the new people would really have good lives or about their effect on others around them. In the scenarios I described, everyone involved gets a good life when ask their interactions with others are taken into account. So yeah, creating livres is complex, but I don't see that that invalidates my question at all. That happens to be my, er, preference too. I think I do think it's a nitpick; we can just take "10x happier" as a sort of shorthand for some corresponding statement about preferences. I promise I had absolutely no such intention. I took the levels higher than typical ones in our world to avoid distracting digressions about whether the typical life in our world is in fact better than nothing. (Note that this isn't the same question as whether it's worth continuing such a life once it's already in progress.) Your example of a world half as happy as this seems like it has a similar but opposite problem: depending on what "half as happy" actually means, you might be describing a change that would be rejected by total utilitarianism as well as average. That's the problem I was trying to avoid.

2Jiro9y

Okay, Now I reveal that just yesterday, we've discovered yet another universe which already exists and is a lot happier than the one you would choose to create. In fact it's so much happier that creating that universe would now drive the average down instead of up. If you're using average utility, then whether this discovery has been made affects whether you want to create that other universe. Is that correct?

0ArisKatsaris9y

With the standard caveats, yes that seems reasonable. Given the existence of that ultrahappy universe an average human life will be more likely to exist in happier circumstances than the ones in the multiversal reality I'd create if I chose to add that less-than-averagely-happy universe. Same way as I'd not take 20% of actual existing happy people and force them to live less happy lives. Think about all sentient lives as if they were part of a single mind, called "Sentience". We design portions of Sentience's life. We want as much a proportion of Sentience's existence to be as happy as possible, satisfying Sentience's preferences.

[-]woodchopper9y10

This doesn't seem very coherent.

As it happens, a perfect and truthful predictor has declared that you will choose torture iff you are alone.

OK. Then that means if I choose torture, I am alone. If I choose the dust specks, I am not alone. I don't want to be tortured, and don't really care about 3 ^^^ 3 people getting dust specks in their eyes, even if they're all 'perfect copies of me'. I am not a perfect utilitarian.

A perfect utilitarian would choose torture though, because one person getting tortured is technically not as bad from a utilitarian point of view as 3 ^^^ 3 dust specks in eyes.

[-]RowanE9y00

The way the problem reads to me, choosing dust specks means I live in a universe where 3^^^3 of me exist, and choosing torture means 1 of me exist. I prefer that more of myself exist than not, so I should choose specks in this case.

In a choice between "torture for everyone in the universe" and "specks for everyone in the universe", the negative utility of the former obviously outweighs that of the latter, so I should choose specks.

I don't see any incongruity or reason to question my beliefs? I suppose it's meant to be implied that it's ... (read more)

[-]OrphanWilde9y00

For the case that dust specks aren't additive, assuming we treat copies of me as distinct entities with distinct moral weight, 3^^^3 copies of me is either a net negative - as a result of 3^^^3 lives not worth living - or a net positive - as a result of an additional 3^^^3 lives worth living. The point of the dust speck is that it has only a negligible effect; the weight of the dust speck moral issue is completely subsumed by the weight of the duplicate people issue.

If we don't treat them as distinct moral entities, well, the duplication and the dust spec... (read more)

-1ike9y

If you smoke in the smoking lesions scenario, then you shouldn't choose your action here based on how many people would exist, because they would exist anyway. (At least in the first of three cases.)

-1OrphanWilde9y

Either you misunderstand the smoking lesions scenario and the importance between the difference between a correlation and a perfect predictor, or you're just trolling the board by throwing every decision theory edge case you can think of into a single convoluted mess.

1ike9y

I may be misunderstanding something, but isn't the standard LW position on smoking to smoke even if the gene's correlation to smoking and cancer is 1? As long as the predictor doesn't cause anything but merely informs, they're equivalent to the gene. The reason why one-boxing is correct is because your choice causes the money, while the reason smoking is correct is because your choice doesn't cause cancer.

0OrphanWilde9y

If the mutual correlation to both is 1, you will smoke if and only if you have the gene, and you will have the gene if and only if you smoke, and in which case you shouldn't smoke. At the point at which the gene is a perfect predictor, if you have a genetic test and you don't have the gene, and then smoke - then the genetic test produced a false negative. Perfect predictors necessarily make a mess of causality.

0ike9y

This implicitly assumes EDT. But that's not what CDT counterfactuals do. You cut off previous nodes. As the choice to smoke doesn't causally affect the gene, smoking doesn't counterfactually contradict the prediction. If you would actually smoke, then yes, but counterfactuals don't imply there's any chance of it happening in reality.

2OrphanWilde9y

No it doesn't. It assumes a "perfect predictor" is what it is. I don't give a damn about evidence - we're specifying properties of a universe here. CDT assumes causality makes sense in the universe. Your hypotheticals don't take place in a universe with the kind of causality causal decision theory depends upon. In the case of a perfect predictor, yes, smoking specifies which gene you have. You don't get to say "Everybody who smokes has this gene" as a property of the universe, and then pretend to be an exception to a property of the universe because you have a bizarre and magical agency that gets to bypass properties of the universe. You're a part of the universe; if the universe has a law (which it does, in our hypotheticals), the law applies to you, too. We have a perfect predictor. We do something the perfect predictor predicted we wouldn't. There is a contradiction there, in case you didn't notice; either it's not, in fact, the perfect predictor we specified, or we didn't do the thing. One or the other. And our hypothetical universe is constructed such that the perfect predictor is a perfect predictor; therefore, we don't get to violate its predictions.

0ike9y

You said "you shouldn't smoke", which is a decision-theoretical claim, not a specification. It's consistent with EDT, but not CDT. In other words, you're denying the exact thing that CDT asserts. Which is what a counterfactual is. Whatever your theory is, it is denying core claims that CDT makes, so you're denying CDT (and implicitly assuming EDT as the method for making decisions, your arguments literally map directly onto EDT arguments).

2OrphanWilde9y

No it isn't, it's a statement about the universe: If you smoke, you'll get lesions. It's written into the specification of the universe; what decision theory you use doesn't change the characteristics of the universe. No. You don't get to specify a universe without the kind of causality that the kind of CDT we use in our universe depends on, and then claim that this says something significant about decision theory. Causality in our hypothetical works differently. No it isn't. No it isn't. In terms of CDT, we can say that smoking causes the gene; this isn't wrong, because, according to the universe, anybody who smokes has the gene; if they didn't, they do now, because the correlation is guaranteed by the laws of the universe. No matter how much work you prepared to ensure you didn't have the gene in advance of smoking, the law of the universe says you have it now. No matter how many tests you ran, they were all wrong. It may seem unintuitive and bizarre, because our own universe doesn't behave this way - but when you find yourself in an alien universe, stomping your foot and insisting that the laws of physics should behave the way you're used to them behaving is a fast way to die. Once you introduce a perfect predictor, the universe must bend to ensure the predictions work out.

0ike9y

What kind of causality is this, given that you assert that the correct thing to do in smoking lesions is refrain from smoking, and smoking lesions is one of the standard things where CDT says to smoke? "A causes B, therefore B causes A" is a fallacy no matter what arguments you put forward. CDT asserts the opposite, and so if you claim this then you disagree with CDT. You don't understand what counterfactuals are.

1OrphanWilde9y

Recursive causality. Perfect mutual correlation means both that A->B and that B->A. No it doesn't. A counterfactual is a state of existence which is not true of the universe. It is not a contradiction.

0entirelyuseless9y

"If you have a genetic test and you don't have the gene, and then smoke - then the genetic test produced a false negative." If Omega makes the mistake of telling someone else that he predicted that you will one-box, and that person tells you, so you then take both boxes, knowing that the million is already there, then Omega's prediction was wrong. Omega can be a perfect predictor, but he cannot tell you his prediction, at least not if you work the way normal humans do. Likewise, a gene could be a perfect predictor, but not if you know about it, at least not if you work the way normal humans do.

0OrphanWilde9y

Trial problem: Omega appears before you, and gives you a pencil. He tells you that, in universes in which you break this pencil in half in the next twenty seconds, the universe ends immediately. Not as a result of your breaking the pencil - it's pure coincidence that all universes in which you break the pencil, the universe ends, and in all universes in which you don't, it doesn't. Do you break the pencil in half? It's not like you're changing anything by doing so, after all; some set of universes will end, some set won't, and you aren't going to change that. You're just deciding which set of universes you happen to occupy. Which implies something.

0entirelyuseless9y

I don't break the pencil. But I already pointed out in Newcomb and in the Smoking Lesion that I don't care if I can change anything or not. So I don't care here either.

-4entirelyuseless9y

We've had this discussion before. When you one-box, your choice does not cause the money. The money is already there or it is not. Causality does not go backwards in time. In other words, Newcomb and the smoking lesion are identical in logical form.

1ArisKatsaris9y

Your decision algorithm will cause the choice. The prediction of that choice, by someone knowing your decision algorithm, will have caused money. If you want the money you should therefore be a decision algorithm that makes the choice whose prediction will cause the money.

-2entirelyuseless9y

You cannot make yourself into a certain decision algorithm, just as you cannot make yourself have or not have a lesion.

2ArisKatsaris9y

What, is this some sort of objection where you believe that determinism means we don't make 'real' choices'? You could be convinced by my words and make yourself into a person who chooses to one-box. Or you could refuse to be convinced and remain a person who chooses to two-boxes. Granted, by being "convinced" or "not convinced" it means that you're already the decision algorithm that would make that choice. So what? Whether you'll be convinced or not still affects your decision algorithm from then on.

-2entirelyuseless9y

No, I don't believe that determinism means we don't make real choices. But it is also true, as you note yourself, that if I am convinced by your words, then I was already the kind of person who would be convinced, and I did not make myself into that sort of person. And likewise for the opposite case. But I am consistent: I believe we make real choices even if Omega predicts our actions, and I also believe we make real choices even if a lesion causes them. The people arguing against my position are saying we don't make real choices in the second case, so they are the ones raising the determinism objection.

0ArisKatsaris9y

Okay, can you just state clearly whether you one-box or two-box, and whether you smoke or not-smoke in the smoking lesion problem, so that I understand what your position is, before trying to understand why it is?

0entirelyuseless9y

I take the one box in Newcomb, and I do not smoke in the smoking lesion. My position is that they are the same problem. The million is already there or it is not, and the lesion is there or it is not. I cannot change that in either case. But I still make a real decision, one that will be correlated with the outcome, and I choose the winning one.

0Pimgd9y

I can't even begin to model myself as "liking" smoking - it gives a disgusting smell that clings to everything and even being near second-hand smoke makes for uncomfortable breathing. If I try to model myself as someone who likes smoking, I don't see myself living, because I've been altered beyond recognition. Add to that that it seems to be a problem without a correct answer ("yes" seems to be the preferred option, given that there is no statement that you prefer smoking without cancer over smoking with cancer, thus "you prefer to smoke" + "some cancer related stuff that you may or may not have an opinion about" = "go smoke already". But this isn't the direct correct answer because if you take another worldview and look at the problem, "to smoke is to admit that you have this genetic flaw and thus you have cancer"), and I have massive problems when it comes to understanding this sort of thing. This question seems to have the same thing going on - pick one! A) "everyone is tortured" or B) "everyone gets a dust speck". But wait, there's some numbers going on in the background where there's either a lot of clones of you or only one of you. And if everyone gets tortured then there's only one of you. Here it is left unsaid that torture is far far far worse than the dust speck for a single individual, but the issue remains: I see "Do a really really really bad thing" or "Do a meh thing" and then some fancy attempts to trip up various logic systems - What about the logic that, hey, A is always worse than B? ... I guess you could fix this by there being OTHER people present, so that it's a "you get tortured" vs "you and everyone else (3^^^3) get a dust speck"... but then there'd be loopholes in the region of "yes, but my preferences prefer a world where there are people other than me, so I'll take torture if that means I get to exist in such a world". As for one-box/two-box, I'd open B up, and if it was empty I'd take the contents of A home. If it contained the cash, we

4gjm9y

Then for the "smoking lesion" problem to be any use to you, you need to perform a sort of mental translation in which it isn't about smoking but about some other (perhaps imaginary) activity that you do enjoy but is associated with harmful outcomes. Maybe it's eating chocolate and the harmful outcome is diabetes. Maybe it's having lots of sex and the harmful outcome is syphilis. Maybe it's spending all your time sitting alone and reading and the harmful outcome is heart disease. The important thing is to keep the structure of the thing the same: doing X is associated with bad outcome Y, it turns out (perhaps surprisingly) that this is not because X causes Y but because some other thing causes both X and Y, you find yourself very much wanting to do X, so now what do you do?

0Jiro9y

Having a smoking lesion make you choose smoking is vague. Does it make you choose smoking by increasing the utility you gain from smoking, but not affecting your ability to reason based on this utility? Or does it make you choose smoking by affecting your ability to do logical reasoning? In the former case, switching from nonsmoking to smoking because you made a logical conclusion should not affect your chances of dying, even though switching to smoking in general should affect your chance of dying. In the latter case, switching to smoking should affect your chance of dying, but you are then asking a question which presupposes under some circumstances that you can't answer it.

0Pimgd9y

I went looking around on wikipedia and found Kavka's toxin puzzle which seems to be about "you can get a billion dollars if you intend to drink this poison (which will hurt a lot for a whole day similar to the worst torture imaginable but otherwise leave no lasting effects) tomorrow evening, but I'll pay you tonight"... but there I don't get the paradox either - whats stopping you from creating a sub agent (informing a friend) with the task of convincing you not to drink AFTER you've gotten the money? ... Possibly by force. Possibly by relying on saying things in a manner that you don't know that he knows he has to do this. Possibly with a whole lot of actors. Like scheduling a text "I am perfectly fine, there is nothing wrong with me" to parents and friends to be sent tomorrow morning. Of course, this relies on my ability to raise the probability of intervention, but that seems like an easier challenge than engaging in willful doublethink... ... or you'd perhaps add various chemicals to your food the next day - I know I can be committed to an idea (I will do this task tonight), come home, eat dinner, and then I'd be totally uncommitted (that task can wait, I will play games first). ... A billion is a lot of money, perhaps I'd drink the poison and then have a hired person drug me to a coma, to be awoken the next day? You could hire a lot of medical staff with that kind of money. Yet I get the feeling that all these "creative" solutions are not really allowed. Why is that?

4gjm9y

Because the point of these questions isn't to challenge you to find a good answer, it's that the process of answering them may lead to insight into your actual value system, understanding of causation, etc. Finding clever ways around the problem is a bit like cheating in an optician's eye test[1]: sure, maybe you can do that, but the result will be that you get less effective eyesight correction and end up worse off. [1] e.g., maybe you have found a copy of whatever chart they use and memorized the letters on it. So, e.g., the point of the toxin puzzle is to ask: can you, really, form an intention to do something when you know that when the time comes you will be able to choose and will have no reason to choose to do it and much reason not to? That's an interesting psychological and/or philosophical question. You can avoid answering it by saying "well, I'd find a way to make taking the toxin not actually do me any harm", and that might be an excellent idea if you ever find yourself in that bizarre situation -- but the point of the question isn't to plan for an actual future where you encounter a quirkily sadistic but generous billionaire, it's to help clarify your thinking about what happens when you form an intention to do something. Of course you may repurpose the question, and then your "clever" answers may be entirely to the point. Suppose you decide that no, you cannot form an intention to do something that you will have good reason to choose not to do; well, situations might arise where it would be useful to do that (even though the precise situation Kavka describes is unlikely), so it's reasonable to think about how you might make it possible, and then some "clever" answers may become relevant. But others probably won't, and the "get drugged into a coma" solution is probably one of those. (Incidentally, in the original puzzle the amount of money was a million rather than a billion. That's probably still enough to hire someone to drug you into a coma.)

0Pimgd9y

It is indeed a million, woops. Thanks for explaining in detail about the purpose of such questions. I find that I get into "come up with a clever answer" mode faster if the question has losses - not getting money is "meh", a day worth of excruciating pain in exchange for money, well, that needs a workaround! As for the puzzle itself, I don't know if I can form such an intention... but I seem to be really good at it in real life. I call it procrastinating. I make a commitment that fails to account for time discounting and then I end up going to bed later than I wanted. After dinner I intended to go to bed early; at midnight I wanted to see another episode. So apparently it's possible.

1Lumifer9y

There are reasons.

1ArisKatsaris9y

Like Odysseus with the Sirens, you'd have to "create a subagent"/hire a friend to convince you not to drink, before you intend to drink it, then actually change your intentions and want to drink it. This doesn't seem possible for a human mind, though of course it's easier to imagine for artificial minds that can be edited at will.

0Pimgd9y

How is it not possible? When force is allowed, the hired people could simply physically restrain me - I'd fight them with tooth and nail, their vastly superior training would have me on the floor within a minute, after which I'd be kept separate from the vial of toxin for the remainder of the day. ... Although I guess "separation for a period of time"-based arguments rely on you both being obsessive AND pedantic enough to not care about it on the next day. Being really passionate about something and then dropping the issue the next day because the window of opportunity has been closed is ... unlikely to occur, so my solution might end up making me rich but leaving me in the looney-bin. I think a better argument against my ideas is logistics - how could I acquire everything I need in a span of (at most) 23 hours? (The wording is a such that at tonight as the day turns, you must intend to take the poison). A middle class worker generally doesn't have ties to any mercenaries, and payment isn't given until the morning after your intent has to be made. I get your point, though - convincing someone to later convince you already carries massive penalties ("Why are you acting so weird?"), the situation carries massive penalties ("And you believe this guy?", "For HOW MUCH?!")... My argument basically rests on turning the whole thing into a game: "Design a puzzle you cannot get out of. Then, a few minutes before midnight (to be safe), start doing your utmost best to break this puzzle."

0ArisKatsaris9y

Why would you hire people to stop you from drinking it, if you intend to drink it, since you know that hiring such people will increase the chances you will end up not drinking it? NO! That's not my point. My point isn't whether it's expensive or difficult to hire someone, My point is that you don't want to hire someone. Because you intend to drink the toxin, and hiring someone to stop you from doing that doesn't match your intention.

0Pimgd9y

If I intend to do my best at an exam tomorrow, but stay up late playing games, does this somehow lift my intention to do well on my exam? By the original problem statement, I have to have the intention of taking the poison AT midnight. Rephrased - when it is midnight, I must intend to take the poison that next day. BEFORE midnight, it is allowed to have OTHER intentions. I intend to use that time to set up hurdles for myself - and then to try my hardest. It would be especially helpful if these hurdles are also things like tricking myself that it won't actually hurt (via transquilizer to put me under straight afterward, for instance). I know it sounds like doublethink, but that's only if you think there is no difference between me before midnight and me after midnight.

0ArisKatsaris9y

If you can change your intentions like that, that's indeed a fine solution. I'm not sure that a human mind can, though. Most human minds anyway.

0Pimgd9y

I get the feeling maybe this ought to be two comments, one on the main thread and one here. But they're too entangled.

0Lumifer9y

Leaving Newcomb aside for the moment, in the smoking lesion case your decision is predetermined and you have no choice in the matter. I don't see how that counts as "a real decision".

2ArisKatsaris9y

"your decision is predetermined and you have no choice in the matter." Is LW now populated by the sort of people who haven't even heard of compatibilism and of the idea that determinism not only doesn't contradict having a choice, but is actually fundamental to the process of decision-making? You can only "choose", if your values and personality can determine the outcome.

0Lumifer9y

By "heard of", do you actually mean "agree with"?

0ArisKatsaris9y

No, I meant that you seemed confused by the fact that someone can think of 'predetermination' as compatible with choice, to the point that you seemingly felt that saying "your decision is predetermined and you have no choice in the matter" was an argument.. and you "don't see" how such predetermined choices are real choices. It's fine if you state your position, but you bring confusion when you present it in terms of ignorance and failure to understand the other one's position. Basically you spoke like I'd expect someone to speak who had indeed never heard of compatibilism, not merely disagreed with it.

0Lumifer9y

I was not trying to change entirelyuseless' mind. I was trying to figure out where exactly the disagreements between us are. No, I do not see that. Is there anything wrong with that? LOL. Are you quite sure I am allowed to disagree with compatibilism for reasons other than being a confused ignorant fool? Well, you speak like someone who does not understand why people could possibly disagree with compatibilism.

0ArisKatsaris9y

Look, say whatever you like, I was genuinely, truly, sincerely, honestly not able to distinguish you from someone who hadn't even heard of compatibilism. Feel free to mock and jeer and lol as you like, but take nonetheless this datapoint into consideration, about what you were communicating to me.

0Lumifer9y

If you think that's a problem, you probably should fix it :-) Or at least take this datapoint into consideration :-P Actually, I was having a reasonable conversation with entirelyuseless when you jumped in and sneered at that "sort of people", those uncough peasants whose presence pollutes the rarefied air of LW with crass ignorance...

0ArisKatsaris9y

Well, see, you understood I was sneering at you. I on the other hand, still don't understand whether you were pretending at ignorance of compatibilism as a weird debating tactic ("1. Pretend that I don't know there exist people who think determinism is compatible with free will, 2. ??? 3.Profit!") or I just misread you that way.

0Lumifer9y

Saying "But Bud Light is a bad beer" is not "pretending at ignorance" that there are people who like and drink Bud Light. It expresses my position which, absent other indicators, does NOT imply that all other positions are wrong and mistaken. Speaking in expressions like "Bud Light is a bad beer, however I acknowledge the existence of people who like Bud Light and accept that there is nothing inherently wrong with them liking Bud Light and, moreover, the expression of my position should not be taken as disparagement of those aforementioned people who like Bud Light" is a bit unwieldy.

0entirelyuseless9y

I agree that this is what most people think, but it is a mistake. I don't agree to leave Newcomb aside in considering this, because my position is that they are the same problem. If I have no choice in the smoking lesion, I have no choice in Newcomb. Consider the Newcomb case. I exist, and my brain and body are in a certain condition. I did not put them in that condition. I cannot make them not have been in that condition. Omega looks at me. Using the condition of my brain and body -- conditions over which I have no control whatsoever -- he determines whether I am going to choose one box or two boxes. He has 100% accuracy, and this implies that the situation is completely determined by the condition of my brain and body. In other words, "the condition of my brain and body" functions exactly like the lesion. It completely "predetermines" the outcome. If I have no choice in the lesion case, I have no choice in Newcomb. Nonetheless, I say I have a choice in Newcomb, because the condition of my brain and body imply that I will engage in a certain process of reasoning, considering the alternatives of one boxing and two boxing, and choose one of them. Likewise, I have a choice in the lesion case, because the lesion implies that I will engage in a certain process of reasoning, considering the alternatives of smoking and not smoking, and choose one of them. In both cases, the outcome is predetermined. In both cases, the outcome is the result of a choice that results from a process of thought.

0Lumifer9y

If they are the same problem, you shouldn't care about leaving one aside. The smoking lesion is a simpler and clearer problem because it doesn't need to postulate a supernatural entity. So you're a determinist. OK. That, to me, doesn't follow at all. You don't choose, you're just an automaton going through the motions. It is, as you say, similar to the lesion -- there might well be complicated intermediate steps but there is no choice involved. You literally do not have a choice. In which way your choice is different from the choice of a calculator which also goes through a bunch of processes before deciding to output 4 as a response to 2+2?

0entirelyuseless9y

This is all in the context of discussing Newcomb and the smoking lesion. It is possible that libertarian free will is true. If it is, neither Newcomb nor the smoking lesion is possible in the real world, at least in the 100% way. So I do not assert that determinism is necessarily true (although I do not know that it is not). But if it is true, it is equally true in Newcomb and in the smoking lesion, and if it is false, it is equally false in both cases. The situation is different from the calculator because the calculator does not consider various possible answers, but just calculates a single answer directly. However, the determinist choice would be similar to a chess computer, which considers various possible moves, but still computes a determinate outcome.

0Lumifer9y

Hold on. Are you saying that determinism is a precondition, an axiom built into the formulation of the Newcomb and the lesion problems? That they make no sense unless you accept determinism? Besides, I don't think Newcomb is possible in the real world anyway since, again, it requires a supernatural entity. This implies that the gap between a chess computer and a human is smaller than a gap between a calculator and a chess computer. I am not sure I'm willing to accept that :-/

0entirelyuseless9y

Yes, I am saying that 100% predictive accuracy does not make sense apart from determinism. I agree that lower degrees of accuracy could happen without complete determinism. Even lower degrees of accuracy would not necessarily change my decision in the scenarios (although it would change the decision once the degree of accuracy became too low.) I agree with you about the gap between humans, calculators, and chess computers. I am just saying that "making a choice" just implies considering several possibilities before selecting one of them. So it isn't true that "you don't have a choice" if you consider several possibilities, even if there are reasons why you will definitely choose a particular one. So for example, even if determinism is false, I am quite sure that I am not going to kill myself tomorrow. That doesn't change the fact that it is one possibility that I could consider. So I have a choice between killing myself and not killing myself, even though I know which one I am going to choose.

0Lumifer9y

That's a straightforward logical fallacy. You're saying "If A then B, therefore if B then A" where A="making a choice" and B="considering several possibilities". Besides, you just moved the heavy-lifting part to the word "considering". If I'm going to count to 3, whether I will consider 2 or 4 (five is right out) is quite irrelevant because I will count to 3 regardless. A considered alternative is one you could choose, but in the situation we're talking about you could not (since your choice is predetermined). And in this case, it's merely something your attention slides over before settling on the inevitable.

0entirelyuseless9y

I am saying "making a choice" is nothing more and nothing less than "considering two or more possibilities and selecting one of them." Each one implies the other. If you are planning to count to three, you do not consider stopping at two, so there is no choice. Your objection is that there are not really two or more possibilities, but only one. But that is not the way consideration works. When you consider two possible choices, they are both possible as far as you know, since you do not know which one you are going to choose. So from your point of view, you are making a choice, even if more fundamentally something is determining which choice you are making.

0Lumifer9y

Ah, so you're defining the expression "making a choice" as "considering and selecting". OK. Am I making a choice from an external point of view?

0entirelyuseless9y

I'm not sure what you mean by "a choice from an external point of view." Other people can see that you considered several possibilities and selected one of them. It may be (if this deterministic theory is true) that someone can figure out in advance which one you are going to select, and perhaps that person wouldn't describe it as a choice. That's just a question of how they are using the word.

0Lumifer9y

The usual -- e.g. in the "perfect predictor" version of the Newcomb problem you might think you're making a choice, but Omega knows what you are going to choose and so from its point of view ("external" to you) you don't actually have a choice and will do what you are predetermined to do. In any case, we've dug down to the more or less standard free-will debate...

1ike9y

I'm referring to TDT, which disagrees.

-2entirelyuseless9y

Eliezer disagrees, but no formal decision theory disagrees, because the two situations are formally identical.

0ike9y

They're formally identical only if you consider the choice to not counterfactually affect the outcome. Asserting that counterfactuals don't go backwards in time makes the choice not affect it, but that's just question begging. It hasn't been formalized because we don't know how to deal with logical uncertainty fully yet.

0entirelyuseless9y

If I have the 100% version of the lesion, it is true to say, "If I had decided not to smoke, I would not have had the lesion," because that is the only way I could have decided not to smoke, in the same way that in Newcomb it is true to say, "If I had picked one-box, I would have been a one-boxer," because that is the only way I could have picked one box.

0ike9y

In one there's counterfactual dependence and in the other there isn't. If your model doesn't take into account counterfactuals then you can't even tell the difference between smoking lesions and the case where smoking really does cause cancer.

0entirelyuseless9y

Exactly. There is no difference; either way you should not smoke. Also, what do you mean by saying that there is "counterfactual dependence" in one case and not in the other? Do you disagree with my previous comment? Do you think that I would have had the lesion no matter what I decided, in a situation where having the lesion has a 100% chance of causing smoking?

-2ike9y

So you're not just arguing with Eliezer, you're arguing with the entirety of causal decision theory. I strongly suspect you don't understand causal decision theory at this point, or counterfactuals as used by it. If this is the case, see https://en.wikipedia.org/wiki/Causal_decision_theory, or http://lesswrong.com/lw/164/timeless_decision_theory_and_metacircular/, or https://wiki.lesswrong.com/wiki/Causal_Decision_Theory Those links explain it better than I can quickly, but I'll try anyway: counterfactuals ask "if you reached into the universe from outside and changed A, what would happen?" Only things caused by A change, not things merely correlated with A.

0entirelyuseless9y

I understand causal decision theory, and yes, I disagree with it. That should be obvious since I am in favor of both one-boxing and not smoking. (Also, if you reach inside and change your decision in Newcomb, that will not change what it is in the box anymore than changing your decision will change whether you have a lesion.)

0ike9y

So why did you ask me what I meant about counterfactuals? If you take the TDT assumption that identical copies of you counterfactually effect each other, then Newcomb has counterfactual dependence and Lesions doesn't. I'm not sure of your point here.

0entirelyuseless9y

I don't think there is any difference even with that assumption. Newcomb and the Lesion are entirely equivalent. Modify it to the situation discussed in the previous discussion of this topic. The Lesion case works like this: the lesion causes people to take two boxes, and the absence of the lesion causes people to take one box. The other parts are the same, except that Omega just checks whether you have the lesion in order to make his prediction. Then we have the two cases: 1. Regular Newcomb. I am a certain kind of algorithm, either one that is going to one-box, or one that is going to two-box. 2. Lesion Newcomb. I either have the lesion and am going to take both boxes, or I don't and am going to take only one. 3. Regular Newcomb. Omega checks my algorithm and decides whether to put the million. 4. Lesion Newcomb. Omega checks the lesion and decides whether to put the million. 5. Regular Newcomb. I decide whether to take one or two boxes. 6. Lesion Newcomb. I decide whether to take one or two boxes. 7. Regular Newcomb. If I decided to take one box, it turns out that I had the one-boxing algorithm, that Omega predicted it, and I get the million. If I decided to take both boxes, the opposite occurs. 8. Lesion Newcomb. If I decided to take one box, it turns out that I did not have the lesion, Omega saw I did not, and I get the million. If I decided to take both boxes, it turns out that I had the lesion etc. This is a simple case of substituting terms. The cases are identical.

1ike9y

Well it depends on what procedure omega uses: you can't change the procedure and assert the same result obtains! If they predict you by simulating you, that creates a causal dependence, but not if they predict you by your genes or similar. You're not accounting for the causal relationship in your comparison.

0entirelyuseless9y

In the lesion case, I am assuming that the lesion has 100% chance of causing you to make a certain decision. If that is not assumed, we are not discussing the situation I am talking about. So the causal process is like this: 1. Lesion exists. 2. Lesion causes certain thought process (e.g. "I really, really, want to smoke. And according to TDT, I should smoke, because smoking doesn't cause cancer. So I think I will.") 3. Thought process causes smoking and lesion causes cancer. I just simulated the lesion process by thinking about it. Omega does the same thing; the details of 2 are irrelevant, as long as we know that the lesion will cause a thought process that will cause smoking.

0ike9y

Sure. The details of 2 is irrelevant, but the details of how Omage works are relevant. If Omega checks for the lesion, then your choice has no counterfactual causal effect on Omega. If Omega simulates your mind, then your choice does have a counterfactual causal effect. Lesion -> thought process -> choice. TDT says choose as if you're determining the outcome of your thought process. If Omega predicts from there, your optimal choice differs from when Omega predicts from Lesion.

0entirelyuseless9y

So you're saying that if Omega predicts from your thought process, you choose one-boxing or not smoking, but if Omega predicts directly from the lesion, you choose two-boxing or smoking? I don't see how that is relevant. The description I gave above still applies. If you choose one-boxing / not smoking, it turns out that you get the million and didn't have the lesion. If you choose two-boxing / smoking, it turns out that you don't get the million, and you had the lesion. This is true whether you followed the rule you suggest or any other. So if TDT recommends smoking when Omega predicts from the lesion, then TDT gives the wrong answer in that case.

0ike9y

Well as I said above, this ignores causality. Of course if you ignore causality, you'll get the EDT answers. And if you define the right answer as the EDT answer, then whenever it differs from another decision theory you'll think the other theory gets the wrong answer. None of this is particularly interesting, and I already made these points above.

0entirelyuseless9y

When you say, "this ignores causality," do you intend to make the opposite statements? Do you think that if a lesion has a 100% chance to cause you to decide to smoke, and you do not decide to smoke, you might have the lesion anyway?

0ike9y

No. But the counterfactual probability of having the lesion given that you smoke is identical to the counterfactual probability given that you don't smoke. This follows directly from the meaning of counterfactual, and you claimed to know what they are. Are you just arguing against the idea of counterfactual probability playing a role in decisions?

0entirelyuseless9y

"Counterfactual probability", in the way you mean it here, should not play a role in decisions where your decision is an effect of something else without taking that thing into account. In other words, the counterfactual you are talking about is this: "If I could change the decision without the lesion changing, the probability of having the lesion is the same." That's true, but entirely irrelevant to any reasonable decision, because the decision cannot be different without the lesion being different.

0ike9y

So all you're doing is denying CDT and asserting EDT is the only reasonable theory, like I thought.

0entirelyuseless9y

I'm denying CDT, but it is a mistake to equate CDT with Eliezer's opinion anyway. CDT says you should two-box in Newcomb; Eliezer says you should one-box (and he is right about that.) More specifically: you assert that in Newcomb, you cause Omega's prediction. That's wrong. Omega's prediction is over and done with, a historical fact. Nothing you can do will change that prediction. Instead, it is true that "Thinking AS THOUGH I could change Omega's prediction will get good results, because I will choose to take one-box, and it will turn out that Omega predicted that." It is equally true that "Thinking AS THOUGH I could change the lesion will get good results, because I will choose not to smoke, and it will turn out that I did not have the lesion." In both cases your real causality is zero. In both cases thinking as though you can cause something has good results.

0ike9y

I'm not equating them. TDT is CDT with some additional claims about causality for logical uncertainties. You deny those claims, but causality doesn't matter to you anyway, because you deny CDT.

[-]Luke_A_Somers9y00

It makes a huge difference whether the dust speck choices add up or not. If they do, OrphanWilde's objection applies and the only path to survival is to be tortured.

If they don't, so each one of me gets one dust speck total, then dust specks for sure. All of the copies of me (whether there are one or 3^^^3 of us) are experiencing what amounts to a choice between individually being dust-specked or individually being tortured. We get what we ask for either way, and no one else is actually impacted by the choice.

There's no need to drag average utilitarianism in.

0HungryHobo9y

Computational theory of identity so some large number of exact copies of the same individual experiencing the same thing don't sum, they only count as once instance?

0Luke_A_Somers9y

That too. But my reasoning holds in the more general case, where instead of it being 3^^^3 copies of me, it was 3^^^3 entities from the pool of people who would choose specks.

[-]CronoDAS9y00

I choose torture if and only if I'm alone. Otherwise the predictor would be wrong, contrary to the assumptions of the hypothetical. But I'd rather be in the world where dust specks gets chosen.

2ike9y

You don't know whether you're alone.

0CronoDAS9y

Doesn't matter - I'll still end up doing it, regardless of what algorithm I try to implement!

0entirelyuseless9y

It is true in general that you will end up implementing the algorithm that you actually are. That doesn't mean you don't have to make a decision.

[-]Furcas9y-10

IMO since people are patterns (and not instances of patterns), there's still only one person in the universe regardless of how many perfect copies of me there are. So I choose dust specks. Looks like the predictor isn't so perfect. :P

Moderation Log