dxu comments on Conceptual Analysis and Moral Theory - Less Wrong

60 Post author: lukeprog 16 May 2011 06:28AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (456)

Comments (456)

Sort By: Best

Show:

You are viewing a single comment's thread. Show more comments above.

Comment author: dxu 19 November 2014 02:52:18AM * 2 points [-]

You get $1000 with 99% probability and $1001000 with 1% probability, for a final expected value of $101090. A one-boxer gets $1000000 with 99% probability and $0 with 1% probability, with a final expected value of $990000. Even with probabilistic uncertainties, you would still have been comparatively better off one-boxing. And this isn't just limited to high probabilities; theoretically any predictive power better than chance causes Newcomb-like situations.

In practice, this tends to go away with lower predictive accuracies because the relative rewards aren't high enough to justify one-boxing. Nevertheless, I have little to no trouble believing that a skilled human predictor can reach accuracies of >80%, in which case these Newcomb-like tendencies are indeed present.

Comment author: Lumifer comment score below threshold [+] (123 children)

Comment author: dxu 19 November 2014 03:02:55AM * 2 points [-]

Assuming you are a two-boxer, there is a 99% chance that there is nothing in Box B (and $1000 in Box A, as always), along with a 1% chance that Box B contains $1000000. If we're going with the most likely scenario, there is nothing in Box B.

Comment author: Lumifer 19 November 2014 03:12:03AM 0 points [-]

In the classic Newcomb's Problem Omega moves first before I can do anything. Step 1 happens before I made any choices.

If Omega is a good predictor, he'll predict my decision, but there is nothing I can do about it. I don't make a choice to be a "two-boxer" or a "one-boxer".

I can make a choice only after step 1, once the boxes are set up and unchangeable. And after step 1 everything is fixed so you should two-box.

Comment author: MarkusRamikin 20 November 2014 09:53:48AM * 5 points [-]

In the classic Newcomb's Problem Omega moves first before I can do anything. Step 1 happens before I made any choices.

This is true both for the 99% and 100% accurate predictor, isn't it? Yet you say you one-box with the 100% one.

I can make a choice only after step 1, once the boxes are set up and unchangeable. And after step 1 everything is fixed so you should two-box.

Please answer me this:

What does 99% accuracy mean to you exactly, in this scenario? If you know that Omega can predict you with 99% accuracy, what reality does this correspond to for you? What do you expect to happen different, compared to if he could predict you with, say, 50% accuracy (purely chance guesses)?

Actually, let's make it more specific: suppose you do this same problem 1000 times, with a 99% Omega, what amount of money do you expect to end up with if you two-box? And what if you one-box?

The reason I am asking is that it appears to me like, the moment Omega stops being perfectly 100% accurate, you really stop believing he can predict you at all. It's like, if you're given a Newcomblike problem that involves "Omega can predict you with 99% accuracy", you don't actually accept this information (and are therefore solving a different problem).

It's unsafe to guess at another's thoughts, and I could be wrong. But I simply fail to see, based on the things you've said, how the "99% accuracy" information informs your model of the situation at all.

Comment author: Lumifer 20 November 2014 03:59:57PM -1 points [-]

This is true both for the 99% and 100% accurate predictor, isn't it? Yet you say you one-box with the 100% one.

Yes, because 100% is achievable only through magic. Omniscience makes Omega a god and you can't trick an omniscient god.

That's why there is a discontinuity between P=1 and P=1-e -- we leave the normal world and enter the realm of magic.

What does 99% accuracy mean to you exactly, in this scenario? If you know that Omega can predict you with 99% accuracy, what reality does this correspond to for you?

In the frequentist framework this means that if you were to fork the universe and make 100 exact copies of it, in 99 copies Omega would be correct and in one of them he would be wrong.

In the Bayesian framework probabilities are degrees of belief and the local convention is to think of them as betting odds, so this means I should be indifferent which side to take of a 1 to 99 bet on the correctness of Omega's decision.

suppose you do this same problem 1000 times, with a 99% Omega, what amount of money do you expect to end up with if you two-box? And what if you one-box?

The question is badly phrased because it ignores the temporal order and so causality.

If you become omniscient for a moment and pick 1000 people who are guaranteed to two-box and 1000 people who are guaranteed to one-box, the one-box people will, of course, get more money from a 99% Omega. But it's not a matter of their choice, you picked them this way.

the moment Omega stops being perfectly 100% accurate, you really stop believing he can predict you at all

Not at all. I mentioned this before and I'll repeat it again: there is no link between Omega's prediction and the choice of a standard participant in the Newcomb's Problem. The standard participant does not have any advance information about Omega with his boxes and so cannot pre-commit to anything. He only gets to do something after the boxes become immutable.

At the core, I think, the issue is of causality and I'm not comfortable with the acausal manoeuvres that LW is so fond of.

Comment author: MarkusRamikin 20 November 2014 05:07:17PM * 2 points [-]

I asked what it means to you. Not sure why I got an explanation of bayesian vs frequentist probability.

there is no link between Omega's prediction and the choice of a standard participant in the Newcomb's Problem. The standard participant does not have any advance information about Omega with his boxes and so cannot pre-commit to anything.

You seem to believe precommitment is the only thing that makes your choice knowable to Omega in advance. But Omega got his track record of 99% accurate predictions somehow. Whatever algorithms are ultimately responsible for your choice, they - or rather their causal ancestors - exist in the world observed by Omega at the time he's filling his boxes. Unless you believe in some kind of acausal choicemaking, you are just as "committed" if you'd never heard of Newcomb's problem. However, from within the algorithms, you may not know what choice you're bound to make until you're done computing. Just as a deterministic chess playing program is still choosing a move, even if the choice in a given position is bound to be, say, Nf4-e6.

Indeed, your willingness (or lack thereof) to believe that, whatever the output of your thinking, Omega is 99% likely to have predicted it, is probably going to be a factor in Omega's original decision.

Comment author: Lumifer 20 November 2014 05:14:43PM -1 points [-]

I asked what it means to you.

To me personally? Pretty much nothing, an abstract exercise with numbers. As I said before (though the post was heavily downvoted and probably invisible by now), I don't expect to meet Omega and his boxes in my future, so I don't care much, certainly not enough to pre-commit.

Or are you asking what 1% probability means to me? I suspect I have a pretty conventional perception of it.

You seem to believe precommitment is the only thing that makes your choice knowable to Omega in advance.

No, that's not the issue. We are repeating the whole line of argument I went through with dxu and TheOtherDave during the last couple of days -- see e.g. this and browse up and down this subthread. Keep in mind that some of my posts there were downvoted into invisibility so you may need to click on buttons to open parts of the subthread.

Comment author: MarkusRamikin 20 November 2014 07:12:24PM * 4 points [-]

Sigh. I wasn't asking if you care. I meant more something like this:

But NASA told Mr. Ullian that the probability of failure was more like 1 in 10^5. [...] “That means you could fly the shuttle every day for an average of 300 years between accidents — every day, one flight, for 300 years — which is obviously crazy!”

Feynman doesn't believe the number, but this is what it means to him: if he were to take the number seriously, this is the reality he thinks it would correspond to. That's what I meant when I asked "what does this number mean to you". What reality the "99% accuracy" (hypothetically) translates to for you when you consider the problem. What work it's doing in your model of it, never mind if it's a toy model.

Suppose you - or if you prefer, any not-precommitted participant - faces Omega, who presents his already-filled boxes, and the participant chooses to either one-box or two-box. Does the 99% accuracy mean you expect to afterwards find that Omega predicted that choice in 99 out of 100 cases on average? If so, can you draw up expected values for either choice? If not, how else do you understand that number?

the whole line of argument I went through with dxu and TheOtherDave during the last couple of days

OK, I re-read it and I think I see it.

the optimal choice for me after Stage 1 is to two-box

I think the issue lies in this "after" word. If determinism, then you don't get to first have a knowable-to-Omega disposition to either one-box or two-box, and then magically make an independent choice after Omega fills the boxes. The choice was already unavoidably part of the Universe before Stage 1, in the form of its causal ancestors, which are evidence for Omega to pick up to make his 99% accurate prediction. (So the choice affected Omega just fine, which is why I am not very fond of the word "acausal"). The standard intuition that places decisionmaking in some sort of a causal void removed from the rest of the Universe doesn't work too well when predictability is involved.

Comment author: Lumifer 20 November 2014 07:49:49PM -1 points [-]

If determinism

Yep, that's another way to look at causality issue. I asked upthread if the correctness of the one-boxing "solution" implies lack of free will and, in fact, depends on the lack of free will. I did not get a satisfying answer (instead I got condescending nonsense about my corrupt variation of an ideal CDT agent).

If "the choice was already unavoidably part of the Universe before Stage" then it is not a "choice" as I understand the word. In this case the whole problem disappears since if the choice to one-box or two-box is predetermined, what are we talking about, anyway?

As is often the case, Christianity already had to deal with this philosophical issue of a predetermined choice -- see e.g. Calvinism.

continue this thread »

Comment author: nshepperd 19 November 2014 03:55:05AM 3 points [-]

I can make a choice only after step 1, once the boxes are set up and unchangeable.

It's common to believe that we have the power to "change" the future but not the past. Popular conceptions of time travel such as Back To The Future show future events wavering in and out of existence as people deliberate about important decisions, to the extent of having a polaroid from the future literally change before our eyes.

All of this, of course, is a nonsense in deterministic physics. If any part of the universe is "already" determined, it all is (and by the way quantum "uncertainty" doesn't change this picture in any interesting way). So there is not much difference between controlling the past and controlling the future, except that we don't normally get an opportunity to control the past, due to the usual causal structure of the universe.

In other words, the boxes are "already set up and unchangeable" even if you decide before being scanned by Omega. But you still get to decide whether they are unchangeable in a favourable or unfavourable way.

Comment author: Lumifer 19 November 2014 03:57:53AM 1 point [-]

It's common to believe that we have the power to "change" the future but not the past.

That's the free-will debate. Does the "solution" to one-box depend on rejection of free will?

Comment author: nshepperd 19 November 2014 04:07:48AM * 2 points [-]

Do you believe that objects in the future waver in and out of existence as you deliberate?

(On the free will debate: The common conception of free will is confused. But that doesn't mean our will isn't free, or imply fatalism.)

Comment author: Lumifer 19 November 2014 04:22:58AM -1 points [-]

I am aware of the LW (well, EY's, I guess) position on free will. But here we are discussing the Newcomb's Problem. We can leave free will to another time. Still, what about my question?

Comment author: TheOtherDave 19 November 2014 04:32:39AM * 2 points [-]

If Omega is a good predictor, he'll predict my decision, but there is nothing I can do about it. I don't make a choice to be a "two-boxer" or a "one-boxer".

Well, if that's true -- that is, if whether you are the sort of person who one-boxes or two-boxes in Newcomblike problems is a fixed property of Lumifer that you can't influence in any way -- then you're right that there's no point to thinking about which choice is best with various different predictors. After all, you can't make a choice about it, so what difference does it make which choice would be better if you could?

Similarly, in most cases, given a choice between accelerating to the ground at 1G and doing so at .01 G once I've fallen off a cliff, I would do better to choose the latter... but once I fall off the cliff, I don't actually have a choice, so that doesn't matter at all.

Many people who consider it useful to think about Newcomblike problems, by contrast, believe that there is something they can do about it... that they do indeed make a choice to be a "two-boxer" or a "one-boxer."

Comment author: Lumifer 19 November 2014 04:35:31AM -2 points [-]

whether you are the sort of person who one-boxes or two-boxes in Newcomblike problems is a fixed property of Lumifer that you can't influence in any way

It's not a fixed property, it's undetermined. Go ask a random person whether he one-boxes or two-boxes :-)

Comment author: TheOtherDave 19 November 2014 04:42:55AM 2 points [-]

Correction accepted. Consider me to have repeated the comment with the word "fixed" removed, if you wish. Or not, if you prefer.

Comment author: Lumifer comment score below threshold [+] (25 children)

Comment author: dxu 19 November 2014 03:20:07AM 1 point [-]

I don't make a choice to be a "two-boxer" or a "one-boxer".

If you said earlier in this thread that you would two-box, you are a two-boxer. If you said earlier in this thread that you would one-box, you are a one-boxer. If Omega correctly predicts your status as a one-boxer/two-boxer, he will fill Box B with the appropriate amount. Assuming that Omega is a good predictor, his prediction is contingent on your disposition as a one-boxer or a two-boxer. This means you can influence Omega's prediction (and thus the contents of the boxes) simply by choosing to be a one-boxer. If Omega is a good-enough predictor, he will even be able to predict future changes in your state of mind. Therefore, the decision to one-box can and will affect Omega's prediction, even if said decision is made AFTER Omega's prediction.

This is the essence of being a reflectively consistent agent, as opposed to a reflectively inconsistent agent. For an example of an agent that is reflectively inconsistent, see causal decision theory. Let me know if you still have any qualms with this explanation.

Comment author: Lumifer 19 November 2014 03:26:52AM -2 points [-]

If you said earlier in this thread that you would two-box, you are a two-boxer. If you said earlier in this thread that you would one-box, you are a one-boxer.

Oh, I can't change my mind? I do that on regular basis, you know...

This means you can influence Omega's prediction (and thus the contents of the boxes) simply by choosing to be a one-boxer.

This implies that I am aware that I'll face the Newcomb's problem.

Let's do the Newcomb's Problem with a random passer-by picked from the street -- he has no idea what's going to happen to him and has never heard of Omega or the Newcomb's problem before. Omega has to make a prediction and fill the boxes before that passer-by gets any hint that something is going to happen.

So, Step 1 happens, the boxes are set up, and our passer-by is explained the whole game. What should he do? He never chose to be a one-boxer or a two-boxer because he had no idea such things existed. He can only make a choice now and the boxes are done and immutable. Why should he one-box?

Comment author: dxu 19 November 2014 03:34:17AM * 1 point [-]

Oh, I can't change my mind? I do that on regular basis, you know...

It seems unlikely to me that you would change your mind about being a one-boxer/two-boxer over the course of a single thread. Nevertheless, if you did so, I apologize for making presuppositions.

So, Step 1 happens, the boxes are set up, and our passer-by is explained the whole game. What should he do? He never chose to be a one-boxer or a two-boxer because he had no idea such things existed. He can only make a choice now and the boxes are done and immutable. Why should he one-box?

As I wrote in my earlier comment:

If Omega is a good-enough predictor, he will even be able to predict future changes in your state of mind. Therefore, the decision to one-box can and will affect Omega's prediction, even if said decision is made AFTER Omega's prediction.

If our hypothetical passerby chooses to one-box, then to Omega, he is a one-boxer. If he chooses to two-box, then to Omega, he is a two-boxer. There's no "not choosing", because if you make a choice about what to do, you are choosing.

Comment author: Lumifer 19 November 2014 03:43:03AM -1 points [-]

The only problem is that you have causality going back in time. At the time of Omega's decision the passer-by's state with respect to one- or two-boxing is null, undetermined, does not exist. Omega can scan his brain or whatever and make his prediction, but the passer-by is not bound by that prediction and has not (yet) made any decisions.

The first chance our passer-by gets to make a decision is after the boxes are fixed. His decision (as opposed to his personality, preferences, goals, etc.) cannot affect Omega's prediction because causality can't go backwards in time. So at this point, after step 2, the only time he can make a decision, he should two-box.

Comment author: dxu 19 November 2014 03:50:15AM * 1 point [-]

As far as I'm aware, what you're saying is basically the same thing as what causal decision theory says. I hate to pass the buck, but So8res has written a very good post on this already; anything I could say here has already been said by him, and better. If you've read it already, then I apologize; if not, I'd say give it a skim and see what you think of it.

Comment author: Lumifer 19 November 2014 04:00:35AM * -1 points [-]

As far as I'm aware, what you're saying is basically the same thing as what causal decision theory says.

So8res' post points out that

CDT is the academic standard decision theory. Economics, statistics, and philosophy all assume (or, indeed, define) that rational reasoners use causal decision theory to choose between available actions.

It seems I'm in good company :-)

dxu comments on Conceptual Analysis and Moral Theory - Less Wrong

Article Navigation

Comments (456)

Nearest Meetups

Virtual Study Room

Recent Comments

Recent Posts

Recent Rationality Quotes

Recent Wiki Edits

Top Contributors, 30 Days

Recent Karma Awards

Comments (456)

Top Contributors, 30 Days

You'll need to login or register to do that

(Don't worry, it only takes a few seconds)

Create

Login