Comment author: brianm 24 September 2009 09:47:49AM *  1 point [-]

The doomsday assumption makes the assumptions that:

  1. We are randomly selected from all the observers who will ever exist.
  2. The observers increase expoentially, such that there are 2/3 of those who have ever lived at any particular generation
  3. They are wiped out by a catastrophic event, rather than slowly dwindling or other

(Now those assumptions are a bit dubious - things change if for instance, we develop life extension tech or otherwise increase rate of growth, and a higher than 2/3 proportion will live in future generations (eg if the next generation is immortal, they're guaranteed to be the last, and we're much less likely depending on how long people are likely to survive after that. Alternatively growth could plateau or fluctuate around the carrying capacity of a planet if most potential observers never expand beyond this) However, assuming they hold, I think the argument is valid.

I don't think your situation alters the argument, it just changes some of the assumptions. At point D, it reverts back to the original doomsday scenario, and the odds switch back.

At D, the point you're made aware, you know that you're in the proportion of people who live. Only 50% of the people who ever existed in this scenario learn this, and 99% of them are blue-doors. Only looking at the people at this point is changing the selection criteria - you're only picking from survivors, never from those who are now dead despite the fact that they are real people we could have been. If those could be included in the selection (as they are if you give them the information and ask them before they would have died), the situation would remain as in A-C.

Making creating the losing potential people makes this more explicit. If we're randomly selecting from people who ever exist, we'll only ever pick those who get created, who will be predominantly blue-doors if we run the experiment multiple times.

Comment author: CannibalSmith 21 July 2009 08:53:32AM 0 points [-]

I get what you're saying.

What principle is being tested here right now?

Comment author: brianm 21 July 2009 12:15:28PM 2 points [-]

The various Newcombe situations have fairly direct analogues in everyday things like ultimatum situations, or promise keeping. They alter it to reduce the number of variables, so the "certainty of trusting other party" dial gets turned up to 100% of Omega, "expectation of repeat" to 0 etc, in order to evaluate how to think of such problems when we cut out certain factors.

That said, I'm not actually sure what this question has to do with Newcombe's paradox / counterfactual mugging, or what exactly is interesting about it. If it's just asking "what information do you use to calculate the probability you plug into the EU calculation?" and Newcombe's paradox is just being used as one particular example of it, I'd say that the obvious answer is "the probability you believe it is now." After all, that's going to already be informed by your past estimates, and any information you have available (such as that community of rationalists and their estimates). If the question is something specific to Newcombe's paradox, I'm not getting it.

In response to Shut Up And Guess
Comment author: brianm 21 July 2009 11:36:54AM 18 points [-]

I think the problem is that people tend to conflate intention with effect, often with dire effect, (eg. "Banning drugs == reducing harm from drug use"). Thus when they see a mechanism in place that seems intended to penalise guessing, they assume that its the same as actually penalising guessing, and that anything that shows otherwise must be a mistake.

This may explian the "moral" objection of the one student: The test attempts to penalise guessing, so working against this intention is "cheating" by exploiting a flaw in the test. With the no-penalty multiple choice, theres no such intent so the assumption is that the benefits of guessing are already factored in.

This may not in fact be as silly as it sounds. Suppose that the test is unrelated to mathematics, and that there is no external motive to doing well. Eg. you are taking a test on Elizabethan history with no effect on your final grade, and want to calibrate yourself against the rest of the class. Here, this kind of test is a flaw, because the test isn't measuring solely what it intends to, but will be biased towards those who spot this advantage. If you are interested solely in an accurate result, and you think the rest of the class won't realise the advantage of guessing, taking the extra marks will just introducing noise, so it is not to your advantage to take them.

For a mathematics or logic based test, the extra benefit could be considered an extra, hidden question. For something else, it could be considered as immoral as taking advantage of any other unintentional effect (a printing error that adds a detectable artifact on the right answer for instance). Taking advantage of it means you are getting extra marks for something the test is not supposed to be counting. I don't think I'd consider it immoral (certainly not enough to forgo the extra marks in something important), but Larry's position may not be as inconsistent as you think.

Comment author: brianm 07 May 2009 04:53:26PM 3 points [-]

I don't see the purpose of such thought experiments as being to model reality (we've already got a perfectly good actual reality for that), but to simplify it. Hypothesizing omnipotent beings and superpowers may not seem like simplification, but it is in one key aspect: it reduces the number of variables.

Reality is messy, and while we have to deal with it eventually, it's useful to consider simpler, more comprehensible models, and then gradually introduce complexity once we understand how the simpler system works. So the thought experiments arbitrarily set certain variables (such as predictive ability) to 100% or 0% simply to remove that aspect from consideration.

This does give a fundamentally unrealistic situation, but that's really the point - they are our equivalent of spherical cows. Dealing with all those variables at once is too hard. In the situations where it isn't and we have "real" situations we can fruitfully consider, there's no need for the thought experiment in the first place. Once we can understand the simpler system, we have somewhere to start from once we start adding back in the complexity.

In response to comment by brianm on Re-formalizing PD
Comment author: cousin_it 30 April 2009 09:48:53AM *  0 points [-]

No, reread the post - in the second scenario programs can't read or compare each other's source code. You're given two ObjectCode instances that are totally opaque except you can pass them to the simulator. If you still succeed in constructing a counterexample for my hypothesis, do let me know.

Comment author: brianm 30 April 2009 11:12:08AM *  1 point [-]

Ah sorry, I'd thought this was in relation to the source available situation. I think this may still be wrong however. Consider the pair of programs below:

A: return Strategy.Defect.
B: if(random(0, 1.0) <0.5) {return Strategy.Cooperate; }
while(true)
{
if(simulate(other, self) == Strategy.Cooperate) { return Strategy.Cooperate; }
}

simulate(A,A) terminates immediately. simulate(B,B) eventually terminates. simulate(B,A) will not terminate 50% of the time.

Comment author: cousin_it 29 April 2009 07:16:24AM *  0 points [-]

I have an unproven hypothesis: if simulate(A, A) terminates and simulate(B, B) terminates, then simulate(A, B) terminates. This would give us a natural rule: each participant must make sure that it terminates against itself. Of course I might be wrong...

Comment author: brianm 30 April 2009 09:44:04AM *  0 points [-]

I don't think this holds. Its clearly possible to construct code like:

if(other_src == my_sourcecode) { return Strategy.COOPERATE; }
if(simulate(other_src, my_sourcecode) == Strategy.COOPERATE)
{
return Strategy.COOPERATE;
}
else
{
return Strategy.DEFECT;
}

B is similar, with slightly different logic in the second part (even a comment difference would suffice).

simulate(A,A) and simulate(B,B) clearly terminate, but simulate(A,B) still calls simulate(B,A) which calls simulate(A,B) ...

Comment author: cousin_it 06 April 2009 12:05:24PM *  1 point [-]

3b) Our ignorance doesn't change the fact that, if the scanner is in principle repeatable, reality contains a contradiction. Type 3 is just impossible.

5) If I were in this situation, I'd assume a prior over possible Omegas that gave large weight to types 1 and 2, which means I would one-box. My prior is justified because a workable Omega of type 3 or 4 is harder for me to imagine than 1 or 2. Disagree? What would you do as a good Bayesian?

Comment author: brianm 06 April 2009 12:19:42PM 5 points [-]

Type 3 is just impossible.

No - it just means it can't be perfect. A scanner that works 99.9999999% of the time is effectively indistinguishable from a 100% for the purpose of the problem. One that is 100% except in the presence of recursion is completely identical if we can't construct such a scanner.

My prior is justified because a workable Omega of type 3 or 4 is harder for me to imagine than 1 or 2. Disagree? What would you do as a good Bayesian?

I would one-box, but I'd do so regardless of the method being used, unless I was confident I could bluff Omega (which would generally require Omega-level resources on my part). It's just that I don't think the exact implementation Omega uses (or even whether we know the method) actually matter.

Comment author: brianm 06 April 2009 11:55:25AM *  6 points [-]

Aren't these rather ducking the point? The situations all seem to be assuming that we ourselves have Omega-level information and resources, in which case why do we care about the money anyway? I'd say the relevant cases are:

3b) Omega uses a scanner, but we don't know how the scanner works (or we'd be Omega-level entities ourselves).

5) Omega is using one of the above methods, or one we haven't thought of, but we don't know which. For all we know he could be reading the answers we gave on this blog post, and is just really good at guessing who will stick by what they say, and who won't. Unless we actually know the method with sufficient confidence to risk losing the million, we should one-box. ([Edit]: Originally wrote two-box here - I meant to say one-box)

Comment author: topynate 21 March 2009 04:01:49PM *  2 points [-]

(This only scales to the point where you value integrity however: you may be able to live with yourself better after finding you're self deluding than after murdering 15 people to prove a point)

This is precisely my reasoning too. It doesn't seem at all sensible to me that the principle of "acting as one would formerly have liked to have precommitted to acting" should have unbounded utility.

ETA: When you say:

Making credible threats requires us to back up what we say, even to someone who we will never encounter again afterwards, so similar situations (without the absolute predictive ability) are quite common in life. I know in the past I have acted perversely against my own self-interest to satisfy a past decision / issued threat.

Now this seems a very good point to me indeed. If we have evolved machinery present in our brains that predictably and unavoidably makes us feel good about following through on a threat and bad about not doing so - and I think that we do have that machinery - then this comes close to resolving the problem. But the point about such a mechanism is that it is tuned to have a limited effect - an effect that I am pretty sure would be insufficient to cause me to murder 15 people in the vast majority of circumstances.

Comment author: brianm 22 March 2009 10:14:11AM *  2 points [-]

It doesn't seem at all sensible to me that the principle of "acting as one would formerly have liked to have precommitted to acting" should have unbounded utility.

Mostly agreed, though I'd quibble that it does have unbounded utility, but that I probably don't have unbounded capability to enact the strategy. If I were capable of (cheaply) compelling my future self to murder in situations where it would be a general advantage to precommit, I would.

Comment author: Vladimir_Nesov 21 March 2009 08:50:48PM *  4 points [-]

What if there is no "on average", if the choice to give away the $100 is the only choice you are given in your life? There is no value in being the kind of person who globally optimizes because of the expectation to win on average. You only make this choice because it's what you are, not because you expect the reality on average to be the way you want it to be.

Comment author: brianm 22 March 2009 09:53:52AM 4 points [-]

From my perspective now, I expect the reality to be the winning case 50% of the time because we are told this as part of the question: Omega is trustworthy and said it tossed a fair coin. In the possible futures where such an event could happen, 50% of the time my strategy would have paid off to a greater degree than it would lose the other 50% of the time. If omega did not toss a fair coin, then the situation is different, and my choice would be too.

There is no value in being the kind of person who globally optimizes because of the expectation to win on average.

There is no value in being such a person if they happen to lose, but that's like saying there's no value in being a person who avoids bets that lose on average by only posing the 1 in several million time they would have won the lottery. On average they'll come out ahead, just not in the specific situation that was described.

View more: Prev | Next