# Solve Psy-Kosh's non-anthropic problem

28 20 December 2010 09:24PM

The source is here. I'll restate the problem in simpler terms:

You are one of a group of 10 people who care about saving African kids. You will all be put in separate rooms, then I will flip a coin. If the coin comes up heads, a random one of you will be designated as the "decider". If it comes up tails, nine of you will be designated as "deciders". Next, I will tell everyone their status, without telling the status of others. Each decider will be asked to say "yea" or "nay". If the coin came up tails and all nine deciders say "yea", I donate \$1000 to VillageReach. If the coin came up heads and the sole decider says "yea", I donate only \$100. If all deciders say "nay", I donate \$700 regardless of the result of the coin toss. If the deciders disagree, I don't donate anything.

First let's work out what joint strategy you should coordinate on beforehand. If everyone pledges to answer "yea" in case they end up as deciders, you get 0.5*1000 + 0.5*100 = 550 expected donation. Pledging to say "nay" gives 700 for sure, so it's the better strategy.

But consider what happens when you're already in your room, and I tell you that you're a decider, and you don't know how many other deciders there are. This gives you new information you didn't know before - no anthropic funny business, just your regular kind of information - so you should do a Bayesian update: the coin is 90% likely to have come up tails. So saying "yea" gives 0.9*1000 + 0.1*100 = 910 expected donation. This looks more attractive than the 700 for "nay", so you decide to go with "yea" after all.

Only one answer can be correct. Which is it and why?

(No points for saying that UDT or reflective consistency forces the first solution. If that's your answer, you must also find the error in the second one.)

Sort By: Old
Comment author: 20 December 2010 10:20:41PM *  4 points [-]

Your second option still implicitly assumes that you're the only decider. In fact each of the possible deciders in each branch of the simulation would be making an evaluation of expected payoff -- and there are nine times as many in the "tails" branches.

There are twenty branches of the simulation, ten with nine deciders and ten with one decider. In the one-decider branches, the result of saying "yea" is a guaranteed \$100; in the nine-decider branches, it's \$1000 in the single case where everyone agrees, \$700 in the single case where everyone disagrees, and \$0 otherwise. The chance of defection depends on the strength of everyone's precommitments, and the problem quickly goes recursive if you start taking expected behavior into account, but in the simplified case where everyone has a well-defined probability of defection the expected payoffs are much higher if your precommitments are strong.

Comment author: 20 December 2010 10:26:45PM *  5 points [-]

I claim that the first is correct.

Reasoning: the Bayesian update is correct, but the computation of expected benefit is incomplete. Among all universes, deciders are "group" deciders nine times as often as they are "individual" deciders. Thus, while being a decider indicates you are more likely to be in a tails-universe, the decision of a group decider is 1/9th as important as the decision of an individual decider.

That is to say, your update should shift probability weight toward you being a group decider, but you should recognize that changing your mind is a mildly good idea 9/10 of the time and a very bad idea 1/10 of the time, and that these should balance to you NOT changing your mind. Since we know that half the time the decision is made by an individual, their decision to not change their mind must be as important as all the decisions of the collective the other half the time.

Comment author: 20 December 2010 10:48:09PM *  1 point [-]

If the decision of a group decider is "1/9th as important", then what's the correct way to calculate the expected benefit of saying "yea" in the second case? Do you have in mind something like 0.9*1000/9 + 0.1*100/1 = 110? This doesn't look right :-(

Comment author: 20 December 2010 11:17:45PM 1 point [-]

I think my answer is actually equivalent to Nornagest's.

The obvious answer is that the factors you divide by are (0.9 / 0.5) and (0.1 / 0.5), which results in the same expected value as the pre-arranged calculation.

Comment author: 21 December 2010 09:39:32AM *  7 points [-]

Do you have in mind something like 0.9 * 1000/9 + 0.1 * 100/1 = 110? This doesn't look right

This can be justified by change of rules: deciders get their part of total sum (to donate it of course). Then expected personal gain before:

``````for "yea": 0.5*(0.9*1000/9+0.1*0)+0.5*(0.9*0+0.1*100/1)=55 for "nay": 0.5*(0.9*700/9+0.1*0)+0.5*(0.9*0+0.1*700/1)=70
``````

Expected personal gain for decider:

``````for "yea": 0.9*1000/9+0.1*100/1=110
for "nay": 0.9*700/9+0.1*700/1=140
``````

Edit: corrected error in value of first expected benefit.

Edit: Hm, it is possible to reformulate Newcomb's problem in similar fashion. One of subjects (A) is asked whether ze chooses one box or two boxes, another subject (B) is presented with two boxes with content per A's choice. If they make identical decision, then they have what they choose, otherwise they get nothing.

Comment author: 22 December 2010 07:07:50AM 1 point [-]

This is awesome! Especially the edit. Thanks.

Comment author: 22 December 2010 03:50:42PM *  7 points [-]

And here's a reformulation of Counterfactual Mugging in the same vein. Find two subjects who don't care about each other's welfare at all. Flip a coin to choose one of them who will be asked to give up \$100. If ze agrees, the other one receives \$10000.

This is very similar to a rephrasing of the Prisoner's Dilemma known as the Chocolate Dilemma. Jimmy has the option of taking one piece of chocolate for himself, or taking three pieces and giving them to Jenny. Jenny faces the same choice: take one piece for herself or three pieces for Jimmy. This formulation makes it very clear that two myopically-rational people will do worse than two irrational people, and that mutual precommitment at the start is a good idea.

This stuff is still unclear to me, but there may be a post in here once we work it out. Would you like to cooperate on a joint one, or something?

Comment author: 22 December 2010 06:16:45PM 0 points [-]

I'm still unsure if it is something more than intuition pump. Anyway, I'll share any interesting thoughts.

Comment author: 21 December 2010 10:25:16PM 2 points [-]

This kind of answer seems like on the right track, but I do not know of a good decision theory when you are not 100% "important". I have an intuitive sense of what this means, but I don't have a technical understanding of what it means to be merely part of a decision and not the full decision maker.

Comment author: 21 December 2010 11:34:27PM 0 points [-]

Can the Shapely value and its generalizations help us here? They deal with "how important was this part of the coalition to the final result?".

Comment author: 20 December 2010 10:45:32PM *  1 point [-]

you should do a Bayesian update: the coin is 90% likely to have come up tails. So saying "yea" gives 0.9*1000 + 0.1*100 = 910 expected donation.

I'm not sure if this is relevant to the overall nature of the problem, but in this instance, the term 0.9*1000 is incorrect because you don't know if every other decider is going to be reasoning the same way. If you decide on "yea" on that basis, and the coin came up tails, and one of the other deciders says "nay", then the donation is \$0.

Is it possible to insert the assumption that the deciders will always reason identically (and, thus, that their decisions will be perfectly correlated) without essentially turning it back into an anthropic problem?

Comment author: 20 December 2010 11:12:41PM 2 points [-]

Because of the context of the original idea (an anthropic question), I think the idea is that all ten of you are equivalent for decision making purposes, and you can be confident that whatever you do is what all the others will do in the same situation.

Comment author: 20 December 2010 11:47:03PM 7 points [-]

Okay. If that is indeed the intention, then I declare this an anthropic problem, even if it describes itself as non-anthropic. It seems to me that anthropic reasoning was never fundamentally about fuzzy concepts like "updating on consciousness" or "updating on the fact that you exist" in the first place; indeed, I've always suspected that whatever it is that makes anthropic problems interesting and confusing has nothing to do with consciousness. Currently, I think that in essence it's about a decision algorithm locating other decision algorithms correlated with it within the space of possibilities implied by its state of knowledge. In this problem, if we assume that all deciders are perfectly correlated, then (I predict) the solution won't be any easier than just answering it for the case where all the deciders are copies of the same person.

(Though I'm still going to try to solve it.)

Comment author: 21 December 2010 12:10:38AM *  2 points [-]

I've always suspected that whatever it is that makes anthropic problems interesting and confusing has nothing to do with consciousness. Currently, I think that in essence it's about a decision algorithm locating other decision algorithms correlated with it within the space of possibilities implied by its state of knowledge.

Sounds right, if you unpack "implied by its state of knowledge" to not mean "only consider possible worlds consistent with observations". Basically, anthropic reasoning is about logical (agent-provable even) uncertainty, and for the same reason very sensitive to the problem statement and hard to get right, given that we have no theory that is anywhere adequate for understanding decision-making given logical uncertainty.

(This is also a way of explaining away the whole anthropic reasoning question, by pointing out that nothing will be left to understand once you can make the logically correlated decisions correctly.)

Comment author: 21 December 2010 12:12:45AM *  1 point [-]

I'm not sure if this is relevant either, but I'm also not sure that such an assumption is needed. Note that failing to coordinate is the worst possible outcome - worse than successfully coordinating on any answer. Imagine that you inhabit case 2: you see a good argument for "yea", but no equally good argument for "nay", and there's no possible benefit to saying "nay" unless everyone else sees something that you're not seeing. Framed like this, choosing "yea" sounds reasonable, no?

Comment author: 21 December 2010 12:20:57AM *  1 point [-]

There's no particular way I see to coordinate on a "yea" answer. You don't have any ability to coordinate with others while you're answering questions, and "nay" appears to be the better bet before the problem starts.

It's not uncommon to assume that everyone in a problem like this thinks in the same way you do, but I think making that assumption in this case would reduce it to an entirely different and less interesting problem -- mainly because it renders the zero in the payoff matrix irrelevant if you choose a deterministic solution.

Comment author: 20 December 2010 11:23:18PM 2 points [-]

(No points for saying that UDT or reflective consistency forces the first solution. If that's your answer, you must also find the error in the second one.)

Under the same rules, does it make sense to ask what is the error in refusing to pay in a Counterfactual Mugging? It seems like you are asking for an error in applying a decision theory, when really the decision theory fails on the problem.

Comment author: 20 December 2010 11:41:49PM *  2 points [-]

Ah - I was waiting for the first commenter to draw the analogy with Counterfactual Mugging. The problem is, Psy-Kosh's scenario does not contain any predictors, amnesia, copying, simulations or other weird stuff that we usually use to break decision theories. So it's unclear why standard decision theory fails here.

Comment author: 21 December 2010 12:07:48AM 3 points [-]

This problem contains correlated decision making, which is what makes copies anthropically confusing.

Comment author: 21 December 2010 03:12:50AM *  1 point [-]

Would it be the same problem if we said that there were nine people told they were potential deciders in the first branch, one person told ey was a potential decider in the second branch, and then we chose the decision of one potential decider at random (so that your decision had a 1/9 chance of being chosen in the first branch, but a 100% chance of being chosen in the second)? That goes some of the way to eliminating correlated decision making weirdness.

Comment author: 21 December 2010 03:30:06AM 0 points [-]

If you change it so that in the tails case, rather than taking the consensus decision, and giving nothing if there is not consensus, the experimenter randomly selects one of the nine decision makers as the true decision maker (restating to make sure I understand), then this analysis is obviously correct. It is not clear to me which decision theories other than UDT recognize that this modified problem should have the same answer as the original.

Comment author: 21 December 2010 08:11:33AM 0 points [-]

Meanwhile, that forumulation is equivalent to just picking one decider at random and then flipping heads or tails to determine what a "yea" is worth! So in that case of course you choose "nay".

Comment author: 21 December 2010 12:35:00AM 1 point [-]

It seems like you are asking for an error in applying a decision theory, when really the decision theory fails on the problem.

Why does it fail on this problem, and less so on others? What feature of this problem makes it fail?

Comment author: 20 December 2010 11:27:23PM *  1 point [-]

This gives you new information you didn't know before - no anthropic funny business, just your regular kind of information - so you should do a Bayesian update: the coin is 90% likely to have come up tails.

Why 90% here? The coin is still fair, and anthropic reasoning should still remain, since you have to take into account the probability of receiving the observation when updating on it. Otherwise you become vulnerable to filtered evidence.

Edit: I take back the sentence on filtered evidence.

Edit 2: So it looks like the 90% probability estimate is actually correct, and the error is in estimating (acausal) consequences of possible decisions.

Comment author: 20 December 2010 11:37:44PM *  1 point [-]

I don't understand your comment. There is no anthropic reasoning and no filtered evidence involved. Everyone gets told their status, deciders and non-deciders alike.

Imagine I have two sacks of marbles, one containing 9 black and 1 white, the other containing 1 black and 9 white. I flip a fair coin to choose one of the sacks, and offer you to draw a marble from it. Now, if you draw a black marble, you must update to 90% credence that I picked the first sack. This is a very standard problem of probability theory that is completely analogous to the situation in the post, or am I missing something?

Comment author: 20 December 2010 11:45:26PM *  0 points [-]

Sorry, I'm just being stupid, always need more time to remember the obvious (and should learn to more reliably take it).

Edit: On further reflection, the intuition holds, although the lesson is still true, since I should try to remember what the intuitions stand for before relying on them.

Edit 2: Nope, still wrong.

Comment author: 21 December 2010 12:53:03AM 0 points [-]

Imagine I have two sacks of marbles, one containing 9 black and 1 white, the other containing 1 black and 9 white. I flip a fair coin to choose one of the sacks, and offer you to draw a marble from it. Now, if you draw a black marble, you must update to 90% credence that I picked the first sack.

The marbles problem has a clear structure where you have 20 possible worlds of equal probability. There is no corresponding structure with 20 possible worlds in our problem.

Comment author: 21 December 2010 12:59:42AM *  2 points [-]

There is. First there's a fair coinflip, then either 1 or 9 deciders are chosen randomly. This means you receive either a black or a white marble, with exactly the same probabilities as in my analogy :-)

Comment author: 21 December 2010 02:02:44AM *  0 points [-]

Very interesting. So probability estimate is correct, and acausal control exerted by the possible decisions somehow manages to exactly compensate the difference in the probability estimates. Still need to figure out what exactly is being controlled and not counted by CDT.

Comment author: 20 December 2010 11:30:20PM 3 points [-]

I can't formalize my response, so here's an intuition dump:

It seemed to me that a crucial aspect of the 1/3 solution to the sleeping beauty problem was that for a given credence, any payoffs based on hypothetical decisions involving said credence scaled linearly with the number of instances making the decision. In terms of utility, the "correct" probability for sleeping beauty would be 1/3 if her decisions were rewarded independently, 1/2 if her (presumably deterministic) decisions were rewarded in aggregate.

The 1/2 situation is mirrored here: There are only two potential decisions (ruling out inconsistent responses), each applicable with probability 0.5, each resulting in a single payoff. Since "your" decision is constrained to match the others', "you" are effectively the entire group. Therefore, the fact that "you" are a decider is not informative.

Or from another angle: Updating on the fact that you (the individual) are a decider makes the implicit assumption that your decision process is meaningfully distinct from the others', but this assumption violates the constraints of the problem.

I remain thoroughly confused by updating. It seems to assume some kind of absolute independence of subsequent decisions based on the update in question.

Comment author: 23 December 2010 05:58:49AM 0 points [-]

I think "implicit assumption that your decision process is meaningfully distinct from the others', but this assumption violates the constraints of the problem." is a good insight.

Comment author: 20 December 2010 11:44:42PM *  2 points [-]

The first is correct. If you expect all 10 participants to act the same you should not distinguish between the cases when you yourself are the sole decider and when one of the others is the sole decider. Your being you should have no special relevance. Since you are a pre-existing human with a defined identity this is highly counterintuitive, but this problem really is no different from this one: An urn with 10 balls in different colors, someone tosses a coin and draws 1 ball if it comes up head and 9 balls if it comes up tails, and in either case calls out the color of one ball. Suppose that color is red, what is the probability the coin came up tails?

If the coin came up heads there is a 1/10 chance of you being drawn, if it came up tails there is a 9/10 chance of you being drawn, and a 1/9 chance of you being you. Since in your intuition you are always you this probably seems nonsensical. But if you distinguish between yourself and the others a priori you can't really treat the payoff like that.

Comment author: 21 December 2010 12:17:44AM *  0 points [-]

Stream of conciousness style answer. Not looking at other comments so I can see afterwards if my thinking is the same as anyone else's.

The argument for saying yea once one is in the room seems to assume that everyone else will make the same decision as me, whatever my decision is. I'm still unsure whether this kind of thinking is allowed in general, but in this case it seems to be the source of the problem.

If we take the opposite assumption, that the other decisions are fixed, then the problem depends on those decisions. If we assume that all the others (if there are any) will say yea then U(yea) = 0.9\$1000 + 0.1\$100 = \$910 = 0.91 lives while U(nay) = 0.90 + 0.1\$700 = \$70 = 0.07 lives, so clearly I say yea. If we assume that all the others (if there are any) will say nay then U(yea) = 0.90 + 0.1\$100 = 0.01 lives while U(nay) = 1*\$700 = 0.7 lives.

In other words, if we expect others to say yea, then we say yea, and if we expect others t say nay, then we say nay (if people say yea with some probability p then our response is either yea or nay depending on whether p is above some threshold). It appears we have a simple game theory problem with two nash equilibria. I'm not sure of a rule for deciding which equilibrium to pick, so I'll try some toy problems.

If we have 10 people, each in a room with two buttons, one red and one blue, and they are told that if they all press the same colour then \$1000 will be donated to Village Reach, but if there is any disagreement no money will be donated, then they have quite a difficult dilemma, two nash equilibria, but no way to single out one of them, which makes them unlikely to end up in either.

If we change the problem, so that now only \$500 is donated when they all press blue, but \$1000 is still given on red, then the decision becomes easy. I'm sure everyone will agree with me that you should definitely press red in this dilemma. It seems like in general a good rule for game-theory problems is "if faced with multiple nash equilibria, pick the one you would have agreed to in advance". This might not give an answer in games where the various utility functions are opposed to each other, but it works fine in this problem.

So I say nay. Both before and after hearing that I am a decider. Problem solved.

Thinking about this a bit more, it seems like the problem came from Timeless Decision Theory, and was solved by Causal Decision Theory. A rather disturbing state of events.

Comment author: 21 December 2010 12:22:50AM *  0 points [-]

I like how you apply game theory to the problem, but I don't understand why it supports the answer "nay". The calculations at the beginning of your comment seem to indicate that the "yea" equilibrium gives a higher expected payoff than the "nay" equilibrium, no?

Comment author: 21 December 2010 12:33:03AM *  1 point [-]

If all ten individuals were discussing the problem in advance the would conclude that nay was better, so, by the rule I set up, when faced with the problem you should say nay.

The problem comes from mixing individual thinking, where you ask what is the best thing for you to do, with group thinking (no relation to groupthink), where you ask what is the best thing for the group to do. The rule I suggested can be expressed as "when individual thinking leaves you with more than one possible solution, use group thinking to decide between them". Updating on the fact that you are a decider is compulsory in individual thinking but forbidden in group thinking, and problems arise when you get confused about this distinction.

Comment author: 21 December 2010 12:24:41AM 0 points [-]

We could say that the existence of pre-agreed joint strategies invalidates standard decision theory.

It's easy to come up with scenarios where coordination is so valuable that you have to choose not to act on privileged information. For example, you're meeting a friend at a pizzeria, and you spot a better-looking pizzeria two blocks away, but you go to the worse one because you'd rather eat together than apart.

Psy-Kosh's problem may not seem like a coordination game, but possibilities for coordination can be subtle and counter-intuitive. See, for example, these two beautiful logic puzzles.

Comment author: 21 December 2010 01:02:25AM *  2 points [-]

If we're assuming that all of the deciders are perfectly correlated, or (equivalently?) that for any good argument for whatever decision you end up making, all the other deciders will think of the same argument, then I'm just going to pretend we're talking about copies of the same person, which, as I've argued, seems to require the same kind of reasoning anyway, and makes it a little bit simpler to talk about than if we have to speak as though that everyone is a different person but will reliably make the same decision.

Anyway:

Something is being double-counted here. Or there's some kind of sleight-of-hand that vaguely reminds me of this problem, where it appears that something is being misplaced but you're actually just being misdirected by the phrasing of the problem. (Not that I accuse anyone of intentionally doing that in any of the versions of this problem.) I can't quite pin it down, but it seems like whatever it is that would (under any circumstance) lead you to assign a .9 decision-theoretic weighting to the tails-world is already accounted for by the fact that there are 9 of you (i.e. 9 who've been told that they're deciders) in that world. I'm not sure how to formally express what I'm getting at, but I think this is moving in the right direction. Imagine a tree of agents existing as a result of the coin flip; the heads branch contains one decider and nine non-deciders; the tails branch contains nine deciders and one non-decider. And each decider needs to have its own judgment of decision-theoretic weighting... but that varies depending on what kind of decision it is. If each one assigns .9 weight to the possibility that it is in the tails branch, then that would be relevant if every agent's decision were to be counted individually (say, if each one had to guess either heads or tails, and would get \$1 if correct; they'd do better guessing tails than by flipping a coin to decide), but in this case the decision is collective and only counted once — so there's no reason to count the multiple copies of you as being relevant to the decision in the first place. It's like if in tails-world you run a (constant) program nine times and do something based on the output, and in heads-world you run it once and do something else based on the output. The structure of the problem doesn't actually imply that the algorithm needs to know how many times it's being executed. I think that's the misdirection.

(Edit: Sorry, that was sort of a ramble/stream-of-consciousness. The part from "If each one assigns..." onward is the part I currently consider relevant and correct.)

Comment author: 22 December 2010 12:47:04AM 1 point [-]

It looks like the double-count is that you treat yourself as an autonomous agent when you update on the evidence of being a decider, but as an agent of a perfectly coordinated movement when measuring the payoffs. The fact that you get the right answer when dividing the payoffs in the 9-decider case by 9 points in this direction.

Comment author: 21 December 2010 02:00:49AM 0 points [-]

I'm assuming all deciders are coming to the same best decision so no worries about deciders disagreeing if you change your mind.

I'm going to be the odd-one-out here and say that both answers are correct at the time they are made… if you care far more (which I don't think you should) about African kids in your own Everett branch (or live in a hypothetical crazy universe where many worlds is false).

(Chapter 1 of Permutation City spoiler, please click here first if not read it yet, you'll be glad you did...): Jura lbh punatr lbhe zvaq nsgre orvat gbyq, lbh jvyy or yvxr Cnhy Qheunz, qvfnoyvat gur cnenpuhgr nsgre gur pbcl jnf znqr.

If you care about African kids in other branches equally, then the first decision is always correct, because although the second choice would make it more likely that kids in your branch will be better off, it will cost the kids in other branches more.

Comment author: 21 December 2010 03:53:00AM *  3 points [-]

Why drag quantum mechanics into this? Taking the expected value gives you exactly the same thing as it does classically, and the answer is still the same. Nay is right and yea is wrong. You seem to be invoking "everett branch" as a mysterious, not-so-useful answer.

Comment author: 21 December 2010 03:11:20PM 1 point [-]

I'm not trying to be mysterious. As far as I can see, there is a distinction. The expected value of switching to Yea from your point of view is affected by whether or not you care about the kids in the branches you are not yourself in.

After being told your status, you're split:

• 9/20 are U=Decider, U=Tails. Yea is good here.

• 1/20 are U=Passenger, U=Tails. Yea is good here.

After being told your status, the new information changes the expected values across the set of branches you could now be in, because that set has changed. It is now only the first 2 lines, above, and is heavily weighted towards Yea = good, so for the kids in your own branches, Yea wins.

But the other branches still exist. If all deciders must come to the same decision (see above), then the expected value of Yea is lower than Nay as long as you care about the kids in branches you're not in yourself - Nay wins. If fact, this expected value is exactly what it was before you had the new information about which branches you can now be in yourself.

Comment author: 21 December 2010 04:13:18PM *  0 points [-]

Okay. You're bringing up quantum mechanics needlessly, though. This is exactly the same reasoning as cousin it went through in the post, and leads to exactly the same problem, since everyone can be expected to reason like you. If yea is only said because it generates better results, and you always switch to yea, then QED always saying yea should have better results. But it doesn't!

Comment author: 21 December 2010 10:02:17PM 0 points [-]

But my whole point has been that yea can yield better results, iff you don't care about kids in other branches, which would make branches relevant.

To show that branches are not relevant, tell me why that argument (that Yeah wins in this case) is wrong, don't just assert that it's wrong.

Comment author: 21 December 2010 11:39:19PM 0 points [-]

Since, as I've been saying, it's identical to the original problem, if I knew how to resolve it I'd already have posted the resolution. :)

What can be shown is that it's contradictory. If yea is better for your "branch" when you vote yea, and everyone always follows this reasoning and votes yea, and the whole point of "branches" is that they're no longer causally linked, all the branches should do better. More simply, if yea is the right choice for every decider, it's because "always yea" actually does better than "always nay."

But always yea is not better than always nay.

If you would like to argue that there is no contradiction, you could try and find a way to resolve it by showing how a vote can be better every single time without being better all the time.

Comment author: 22 December 2010 12:47:08AM 0 points [-]

if yea is the right choice for every decider, it's because "always yea" actually does better than "always nay.

It is the best choice for every decider who only cares about the kids in their Everett branches.

It's not the best choice for deciders (or non-deciders, though they don't get a say) who care equally about kids across all the branches. Their preferences are as before.

It's a really lousy choice for any non-deciders who only care about the kids in their Everett branches. Their expected outcome for "yea" just got worse by the same amount that first lot of deciders who only care about their kids got better. Unfortunately for them, their sole decider thinks he's probably in the Tails group, and that his kids will gain by saying "yea", as he is perfectly rational to think given the information he has at that time.

Comment author: 22 December 2010 01:44:53AM 2 points [-]

What does an entity that only cares about the kids in its Everett branches even look like? I am confused. Usually things have preferences about lotteries over outcomes, and an outcome is an entire multiverse, and these things are physically realized and their preferences change when the coinflip happens? How does that even work? I guess if you want you can implement an entity that works like that, but I'm not certain why we'd even call it the same entity at any two times. This sort of entity would do very well to cut out its eyes and ears so it never learns it's a decider and begin chanting "nay, nay, nay!" wouldn't it?

Comment author: 22 December 2010 02:36:34AM 0 points [-]

What does an entity that only cares about the kids in its Everett branches even look like?

Example 1: Someone that doesn't know about or believe in many worlds. The don't care about kids in alternate Everett branches, because to their mind they don't exist, so have zero value. In his mind, all value is in this single universe, with a coin that he is 90% sure landed Tails. By his beliefs, "yea" wins. Most people just don't think about entire multiverses.

Example 2: Someone who gets many worlds, but tends inclined to be overwhelmingly more charitable to those that feel Near rather than Far, and to those that feel like Their Responsibility rather than Someone Else's Problem. I hear this isn't too uncommon :-)

This sort of entity would do very well to cut out its eyes and ears so it never learns it's a decider and begin chanting "nay, nay, nay!" wouldn't it?

Actual cutting aside, this is an excellent strategy. Upvoted :)

Comment author: 22 December 2010 02:44:56AM *  -1 points [-]

I suppose I'll avoid repeating myself and try to say new things.

You seem to be saying that when you vote yea, it's right, but when other people vote yea, it's wrong. Hmm, I guess you could resolve it by allowing the validity of logic to vary depending on who used it. But that would be bad.

(Edited for clarity)

Comment author: 22 December 2010 02:59:53AM 2 points [-]

I think we may be misunderstanding each-other, and possibly even arguing about different things. I'm finding it increasingly hard to think how your comments could possibly be a logical response to those you're responding to, and I suspect you're feeling the same.

So when you do it, it's right, but when other people do it, it's wrong?

When I do what? What are you even talking about?

Comment author: 22 December 2010 04:42:32AM *  0 points [-]

So when you do it, it's right, but when other people do it, it's wrong?

When I do what? What are you even talking about?

Ah, sorry, that does look odd. I meant "when you vote 'yea,' it's okay, but when they vote 'yea' for exactly the same reasons, it's bad."

Comment author: 23 December 2010 01:37:57AM -2 points [-]

I think your reasoning here is correct and that it is as good an argument against the many worlds interpretation as any that I have seen.

Comment author: 23 December 2010 01:46:29AM 2 points [-]

I think your reasoning here is correct and that it is as good an argument against the many worlds interpretation as any that I have seen.

The best argument against the many worlds interpretation that you have seen is somewhat muddled thinking about ethical considerations with respect to normal coin tosses?

Comment author: 23 December 2010 03:33:16PM 0 points [-]

Yup, that's the best. I'd be happy to hear about the best you've seen, especially if you've seen better.

Comment author: 23 December 2010 04:47:21PM 0 points [-]

Yup, that's the best. I'd be happy to hear about the best you've seen, especially if you've seen better.

Why do you assume I would be inclined to one up the argument? The more natural interpretation of my implied inference is in approximately the reverse direction.

If the best argument against MWI that a self professed physicist and MWI critic has ever seen has absolutely zero persuasive power then that is rather strong evidence in favor.

Comment author: 23 December 2010 07:45:08PM 3 points [-]

I am new to this board and come in with a "prior" of rejecting MWI beyond the tiniest amount on the basis of, among other things, conservation of energy and mass. (Where do these constantly forming new worlds come from?) MWI seems more like a mapmakers mistake than a description of the territory, which manifestly has only one universe in it every time I look.

I was inviting you to show me with links or description whatever you find most compelling, if you could be bothered to. I am reading main sequence stuff and this is one of the more interesting puzzles among Less Wrong's idiosyncratic consensi.

Comment author: 23 December 2010 07:58:07PM 0 points [-]

Wikipedia points to a site that says conservation of energy is not violated. Do you know if it's factually wrong or what's going on here? (if so can you update wikipedia? :D)

Q22 Does many-worlds violate conservation of energy?

First, the law conservation of energy is based on observations within each world. All observations within each world are consistent with conservation of energy, therefore energy is conserved. Second, and more precisely, conservation of energy, in QM, is formulated in terms of weighted averages or expectation values. Conservation of energy is expressed by saying that the time derivative of the expected energy of a closed system vanishes. This statement can be scaled up to include the whole universe. Each world has an approximate energy, but the energy of the total wavefunction, or any subset of, involves summing over each world, weighted with its probability measure. This weighted sum is a constant. So energy is conserved within each world and also across the totality of worlds.

One way of viewing this result - that observed conserved quantities are conserved across the totality of worlds - is to note that new worlds are not created by the action of the wave equation, rather existing worlds are split into successively "thinner" and "thinner" slices, if we view the probability densities as "thickness".

Comment author: 23 December 2010 08:23:22PM *  1 point [-]

Here a subsequent discussion about some experimental test(s) of MWI. Also here a video dicussion between Scott Aaronson and Yudkowsky (starting at 38:11). More links on topic can be found here.

Comment author: 23 December 2010 12:46:19PM 0 points [-]

I don't understand. How is my argument an argument against the many worlds interpretation? (Without falling into the logical fallacy of Appeal to Consequences).

Comment author: 23 December 2010 03:32:32PM 0 points [-]

It would seem to suggest that if I want to be rich I should buy a bunch of lottery tickets and then kill myself when I don't win.

I have not seen the local discussion of MWI and everett branches, but my "conclusion" in the past has been that MWI is a defect of the map maker and not a feature of the territory. I'd be happy to be pointed to something that would change my mind or at least rock it a bit, but for now it looks like angels dancing on the heads of pins. Has somebody provided an experiment that would rule MWI in or out? If so, what was the result? If not, then how is a consideration of MWI anything other than confusing the map with the territory?

If I have fallen in to Appeal to Consequences with my original post, than my bad.

Comment author: 23 December 2010 04:11:53PM 0 points [-]

It would seem to suggest that if I want to be rich I should buy a bunch of lottery tickets and then kill myself when I don't win.

I don't think that's the case, but even if it were, using that to argue against the likelihood of MWI would be Appeal to Consequences.

I have not seen the local discussion of MWI and everett branches, but my "conclusion" in the past has been that MWI is a defect of the map maker and not a feature of the territory.

That's what I used to think :)

I'd be happy to be pointed to something that would change my mind or at least rock it a bit

If you're prepared for a long but rewarding read, Eliezer's Quantum Physics Sequence is a non-mysterious introduction to quantum mechanics, intended to be accessible to anyone who can grok algebra and complex numbers. Cleaning up the old confusion about QM is used to introduce basic issues in rationality (such as the technical version of Occam's Razor), epistemology, reductionism, naturalism, and philosophy of science.

For a shorter sequence that concentrates on why MWI wins, see And the Winner is... Many-Worlds!

Has somebody provided an experiment that would rule MWI in or out? If so, what was the result? If not, then how is a consideration of MWI anything other than confusing the map with the territory?

The idea is that MWI is the simplest explanation that fits the data, by the definition of simplest that has proven to be most useful when predicting which of different theories that match the same data is actually correct.

Comment author: 21 December 2010 03:36:45AM 6 points [-]

It is an anthropic problem. Agents who don't get to make decisions by definition don't really exist in the ontology of decision theory. As a decision theoretic agent being told you are not the decider is equivalent to dying.

Comment author: 21 December 2010 05:02:04AM *  4 points [-]

As a decision theoretic agent being told you are not the decider is equivalent to dying.

Or more precisely it is equivalent to falling into a crack in spacetime without Amelia Pond having a crush on you. ;)

Comment author: 21 December 2010 11:06:19AM 1 point [-]

It is an anthropic problem.

Depends of what you mean by "Anthropic problem". The first google result for that term right now is this post, so the term doesn't seem to have a widely-agreed upon meaning, though there is some interesting discussion on Wikipedia.

Maybe we could distinguish

• "Anthropic reasoning", where your reasoning needs to take into account not only the facts you observed (i.e. standard bayesian reasoning) but also the fact that you are there to take the decision period.

• "Anthropic scenarios" (ugh), where the existence of agents comes into account (ike the sleeping beauty problem, our universe, etc.)

Anthropic scenarios feature outlandish situations (teleporters, the sleeping beauty) or are somewhat hard to reproduce (the existence of our universe). So making scenarios that aren't outlandish anthropic scenarios but still require anthropic reasoning is nice for intuition (especially in an area like this where everybody's intuition starts breaking down), even if it doesn't change anything from a pure decision theory point of view.

I'm not very happy with this decomposition; seems to me "is this an anthropic problem?" can be answered by "Well it does require anthropic reasoning but doesn't require outlandish scenarios like most similar problems", but there may be a better way of putting it.

Comment author: 21 December 2010 07:19:54PM 7 points [-]

It is a nice feature that Psy-kosh's problem that it pumps the confusing intuitions we see in scenarios like the Sleep Beauty problem without recourse to memory erasing drugs or teleporters-- I think it tells us something important about this class of problem. But mathematically the problem is equivalent to one where the coin-flip doesn't make nine people deciders but copies you nine times- I don't think there is a good justification for labeling these problems differently.

The interesting question is what this example tells us about the nature of this class of problem- and I'm having trouble putting my finger on just what that is.

Comment author: 22 December 2010 06:58:53AM 4 points [-]

RIght. That's the question I wanted people to answer, not just solve the object-level problem (UDT solves it just fine).

Comment author: 21 December 2010 05:00:39AM *  1 point [-]

I would decide "nay". Very crudely speaking a fundamental change in viewpoints is involved. If I update on the new information regarding heads vs tales I must also adjust my view of what I care about. It is hard to describe in detail without writing an essay describing what probability means (which is something Wei has done and I would have to extend to allow for the way to describe the decision correctly if updating is, in fact, allowed.).

Comment author: 21 December 2010 05:47:21AM *  0 points [-]

A note on the problem statement: You should probably make it more explicit that mixed strategies are a bad idea here (due to the 0-if-disagreement clause). I spent a bit wondering why you restricted to pure strategies and didn't notice it until I actually attempted to calculate an optimal strategy under both assumptions. (FWIW, graphing things, it seems that indeed, if you assume both scenarios are equally likely, pure "nay" is best, and if you assume 9 deciders is 90% likely, pure "yea" is best.)

Comment author: 21 December 2010 09:10:39AM *  6 points [-]

After being told whether they are deciders or not, 9 people will correctly infer the outcome of the coin flip, and 1 person will have been misled and will guess incorrectly. So far so good. The problem is that there is a 50% chance that the one person who is wrong is going to be put in charge of the decision. So even though I have a 90% chance of guessing the state of the coin, the structure of the game prevents me from ever having more than a 50% chance of the better payoff.

eta: Since I know my attempt to choose the better payoff will be thwarted 50% of the time, the statement "saying 'yea' gives 0.9*1000 + 0.1*100 = 910 expected donation" isn't true.

Comment author: 21 December 2010 09:51:16AM *  4 points [-]

I tell you that you're a decider [... so] the coin is 90% likely to have come up tails.

Yes, but

So saying "yea" gives 0.9 * 1000 + 0.1 * 100 = 910 expected donation.

... is wrong: you only get 1000 if everybody else chose "yea". The calculation of expected utility when tails come up has to be more complex than that.

Let's take a detour through a simpler coordination problem: There are 10 deciders with no means of communication. I announce I will give \$1000 to a Good and Worth Charity for each decider that chose "yea", except if they all chose yea, in which case I will give nothing. Here the optimal strategy is to choose "yea" with a certain probability p, which I don't have time to calculate right now.

But anyway, in coordination problems in general, the strategy is to find what the group strategy should be, and execute that. Calculation of expected utility should not be done on the individual level (because there's no reliable way to account for the decisions of others if they depend on yours), but on the group level.

Comment author: 21 December 2010 10:15:20AM *  8 points [-]

Here the optimal strategy is to choose "yea" with a certain probability p, which I don't have time to calculate right now

The expected value is \$1000 (10 * p - 10 p^ 10). Maximums and minimums of functions may occur when the derivative is zero, or at boundaries.

The derivative is \$1000(10 - 100 p^ 9). This is zero when p = 0.1^(1/9) ~= 0.774. The boundaries of 0 and 1 are minima, and this is a maximum.

EDIT: Huh. This simple calculation that mildly adds to the parent is worth more karma than the parent? I thought the parent really got to the heart of things with: "(because there's no reliable way to account for the decisions of others if they depend on yours)" Of course, TDT and UDT are attempts to do just that in some circumstances.

Comment author: 21 December 2010 10:40:16AM *  0 points [-]

EDIT: Nevermind, I massively overlooked a link.

Comment author: 21 December 2010 12:54:46PM *  1 point [-]

Un-thought-out-idea: We've seen in the Dutch-book thread that probability and decision theory are inter-reliant so maybe

Classical Bayes is to Naive Decision Theory

as

Bayes-we-need-to-do-anthropics is to UDT

Actually now that I've said that it doesn't seem to ground-breaking. Meh.

Comment deleted 21 December 2010 05:53:57PM *  [-]
Comment author: 22 December 2010 06:56:59AM 1 point [-]

Are you claiming that it's rational to precommit to saying "nay", but upon observing D1 (assuming no precommitment happened) it becomes rational to say "yea"?

Comment author: 21 December 2010 07:42:38PM -1 points [-]

This struck an emotional nerve with me, so I'm going to answer as if this were an actual real-life situation, rather than an interesting hypothetical math problem about maximizing expected utility.

IMHO, if this was a situation that occurred in real life, neither of the solutions is correct. This is basically another version of Sophie's Choice. The correct solution would be to punch you in the face for using the lives of children as pawns in your sick game and trying to shift the feelings of guilt onto me, and staying silent. Give the money or not as you see fit, but don't try to saddle me with guilt over lives that YOU chose not to save.

The correct option is refusal to play combined with retaliation for blackmail. If - as with the Nazi officer - this is impossible and one is stuck with choosing from the two options you gave THEN calculation can enter into it. However, future vengeance opportunities should be explored.

It may be better to phrase such problems in terms of game-shows or other less emotionally charged situations. Given a twisted enough game, people will look for ways to pull the rope sideways.

Comment author: 21 December 2010 08:12:08PM 1 point [-]

Each decider will be asked to say "yea" or "nay". If the coin came up tails and all nine deciders say "yea", I donate \$1000 to VillageReach. If the coin came up heads and the sole decider says "yea", I donate only \$100. If all deciders say "nay", I donate \$700 regardless of the result of the coin toss. If the deciders disagree, I don't donate anything.

Suppose that instead of donating directly (presumably for tax reasons), you instead divide the contribution up among the deciders, and then let them pass it on. As I figure it, that simple change in the rules eliminates the paradox.

I'm not sure what this means, but I'm going to blame the paradox on the tax-code. :)

Comment author: 21 December 2010 09:34:26PM 0 points [-]

Once I have been told I am a decider, the expected payouts are:

For saying Yea: \$10 + P\$900 For saying Nay: \$70 + Q\$700

P is the probability that the other 8 deciders if they exist all say Yea conditioned on my saying Yay, Q is the probability that the other 8 deciders if they exist all say Nay conditioned on my saying Nay.

For Yay to be the right answer, to maximize money for African Kids, we manipulate the inequality to find P > 89% - 78%*Q The lowest value for P consistent with 0<=Q<=1 is P > 11% which occurs when Q = 100%.

What are P & Q? All we are told about the other deciders if they exist is that they are "people who care about saving African kids." With only that information, I can't see how to rationally use anything other than ~0.5% = P = Q, that is, our prior probability that any other decider says yay or nay is 50%, so getting all 8 of them to give the same answer of Yay or Nay has prior probability of (~(1/2)^8). Any other assumption of a different prior given the statement of the problem seems irrational to me.

So with P & Q on the small side, the correct decision is to pick "Nay." The error in the original post analysis of the case for picking Yay was assuming that the value of P is 1, that if I picked Yay that all people in the class of "people who care about African Kids" would also pick Yay. As stated above, from the information we have the rational prior estimate of that probability P is about 0.5%.

I would go further and say what about updating probabilities based on what we see in the answers here? If anything, they decrease the probability of P (everybody picks Yay), as more people seem to favor Nay than Yay. But this little bit of peeking below the fold is not needed for the estimate.

Comment author: 22 December 2010 03:25:01AM *  0 points [-]

First let's work out what joint strategy you should coordinate on beforehand. If everyone pledges to answer "yea" in case they end up as deciders, you get 0.5*1000 + 0.5*100 = 550 expected donation. Pledging to say "nay" gives 700 for sure, so it's the better strategy.

This suggests to me that you actually can coordinate with everyone else, but the problem isn't clear on whether or not that's the case.

Comment author: 23 December 2010 01:35:17AM 6 points [-]

Thank you for that comment. I think I understand the question now. Let me restate it somewhat differently to make it, I think, clearer.

All 10 of us are sitting around trying to pre-decide a strategy for optimizing contributions.

The first situation we consider is labeled "First" and quoted in your comment. If deciders always say yay, we get \$1000 for heads and \$100 for tails which gives us an expected \$550 payout. If we all say nay, we get \$700 for either heads or tails. So we should predecide to say "nay."

But then Charlie says "I think we should predecide to say "yay." 90% of the time I am informed I am a decider, there are 8 other deciders besides me, and if we all say "yay" we get \$1000. 10% of the time I am informed I am a decider, we will only get \$100. But when I am informed I am a decider, the expected payout is \$910 if we all say yay, and only \$700 if we all say nay."

Now Charlie is wrong, but its no good just asserting it. I have to explain why.

It is because Charlie has failed to consider the cases when he is NOT chosen as a decider. He is mistakenly thinking that since in those cases he is not chosen as a decider his decision doesn't matter, but that is not true because we are talking about a predetermined group decision. So Charlie must recognize that in the 50% of cases where he is NOT chosen as a decider, 90% of THOSE cases are TAILS and his participation in directing the group to vote "yay" in those cases nets an expected \$190 each time he is not chosen, while his participation in directing the group to vote "nay" nets \$700 in each of those times when he is not chosen.

So the nub of it is that since this is a group decision, Charlie is actually a decider whether he is chosen as a decider or not, his decision, the group decision, affects the outcome of EVERY branch, not just the branches in which Charlie is an "official" decider. The nub of it is that we have equivocation on the word "decider," when it is a group decision, when we count on every individual reaching the same decision, than every individual analyzing the situation must evaluate payout in ALL branches, and not just the ones in which the setter of the problem has chosen to label him a "decider." When Charlie is NOT chosen as a decider, he can conclude that tails was thrown with 90% probability and heads with only 10 probability.

Comment author: 23 December 2010 03:14:28AM 1 point [-]

Very good explanation; voted up. I would even go so far as to call it a solution.

Comment author: 23 December 2010 12:16:02PM *  0 points [-]

Below is very unpolished chain of thoughts, which is based on vague analogy with symmetrical state of two indistinguishable quantum particles.

When participant is said ze is decider, ze can reason: let's suppose that before coin was flipped I changed places with someone else, will it make difference? If coin came up heads, than I'm sole decider and there are 9 swaps which make difference in my observations. If coin came up tails then there's one swap that makes difference. But if it doesn't make difference it is effectively one world, so there's 20 worlds I can distinguish, 10 correspond to my observations, 9 have probability (measure?) 0.5 * 0.1 (heads, I'm decider), 1 have probability 0.5 * 0.9 (tails, I'm decider). Consider following sentence as edited out. What I designated as P(heads) is actually total measure (?) of worlds participant is in. All this worlds are mutually exclusive, thus P(heads)=9 * 0.5 * 0.1+1 * 0.5 * 0.9=0.9.

What is average benefit of "yea"? (9 * 0.5 * 0.1 * \$100 + 1 * 0.5 * 0.9 * \$1000)=\$495

Same for "nay": (9 * 0.5 * 0.1 * \$700+1 * 0.5 * 0.9 * \$700)=\$630

Comment author: 23 December 2010 12:28:12PM *  -1 points [-]

Um, the probability-updating part is correct, don't spend your time attacking it.

Comment author: 25 December 2010 03:48:39PM -1 points [-]

Precommitting to "Yea" is the correct decision.

The error: the expected donation for an individual agent deciding to precommit to "nay" is not 700 dollars. It's pr(selected as decider) * 700 dollars. Which is 350 dollars.

Why is this the case? Right here:

Next, I will tell everyone their status, without telling the status of others ... Each decider will be asked to say "yea" or "nay".

In all the worlds where you get told you are not a decider (50% of them - equal probability of 9:1 chance or a 1:9 chance) your precommitment is irrelevant. Therefore, the case where everyone precommits to yea is logically equivalent to the case where everyone precommits to nay and then change to yea upon being told they are a decider.

So we can talk about two kinds of precommitments: "I precommit to answering yea/nay" and "I precommit to answer yea/nay given that I am informed I am a decider". The expected donation in the first precomittment is yea: 455 dollars and nay: 350 dollars; the expected donation in the second precomittment is yea: 910 dollars and nay: 700 dollars.¹

Yes, the first precommittment is half that of the second: because the first goes through a 50% filter, the second starts on the other side of that filter. Of course if you start yea before the filter and nay afterwards, you're going to get the wrong result.

¹: Summing the expected values of yea. pr(heads) * pr(selected as decider) * value of 'yea' for heads = 0.5 * 0.1 * 100 = 5 dollars, pr(tails) * pr(selected as decider) * value of 'yea' for tails = 0.5 * 0.9 * 1000 = 450 dollars. Sum = 455 dollars.

Comment author: 25 December 2010 04:20:39PM *  1 point [-]

In all the worlds where you get told you are not a decider (50% of them - equal probability of 9:1 chance or a 1:9 chance) your precommitment is irrelevant.

How can that be, when other people don't know whether or not you're a decider?

Imagine the ten sitting in a room, and two people stand up and say "If I am selected as a decider, I will respond with 'yea'." This now forces everyone else to vote 'yea' always, since in only 5% of all outcomes (and thus 10% of the outcomes they directly control) does voting 'nay' increase the total donation (by 600*.1=60) whereas in the other 45% / 90% of cases it decreases the donation (by 1000*.9=900).

The two people who stood up should then suddenly realize that the expected donation is now \$550 instead of \$700, and they have made everyone worse off by their declaration.

(One person making the declaration also lowers the expected donation by a lower amount, but the mechanism is clearer with two people.)

Comment author: 26 December 2010 05:54:51AM *  2 points [-]

How can that be, when other people don't know whether or not you're a decider?

It doesn't make much sense to say "I precommit to answering 'nay' iff I am not selected as a decider."

But then ... hmm, yeah. Maybe I have this the wrong way around. Give me half an hour or so to work on it again.

edit: So far I can only reproduce the conundrum. Damn.

Comment author: 26 December 2010 09:55:44AM 2 points [-]

I think the basic error with the "vote Y" approach is that it throws away half of the outcome space. If you make trustworthy precommitments the other people are aware of, it should be clear that once two people have committed to Y the best move for everyone else is to vote Y. Likewise, once two people have committed to N the best move for everyone else is to vote N.

But, since the idea of updating on evidence is so seductive, let's take it another step. We see that before you know whether or not you're a decider, E(N)>E(Y). One you know you're a decider, you naively calculate E(Y)>E(N). But now you can ask another question- what are P(N->Y|1) and P(N->Y|9)? That is, the probability you change your answer from N to Y given that you are the only decider and given that you are one of the nine deciders.

It should be clear there is no asymmetry there- both P(N->Y|1) and P(N->Y|9)=1. But without an asymmetry, we have obtained no actionable information. This test's false positive and false negative rates are aligned exactly as to do nothing for us. Even though it looks like we're preferentially changing our answer in the favorable circumstance, it's clear from the probabilities that there's no preference, and we're behaving exactly as if we precommitted to vote Y, which we know has a lower EV.

Comment author: 30 July 2012 10:56:02PM 1 point [-]

The devil is in the assumption that everyone else will do the same thing as you for presumably the same reasons. "Nay" is basically a better strategy in general, though it's not always right. The 90% odds of tails are correctly calculated.

• If you are highly confident that everyone else will say "yea", it is indeed correct to say "yea"
• If you're highly confident that everyone will say the same thing but you're not sure what it is (an unlikely but interesting case), then make a guess (if it's 50-50 then "yea" is better - the cutoff in fact is just a bit above 45% chance of all-yea over all-nay)
• If you don't really know what the others will say, and they're all acting independently, then odds are they've blown it by giving different answers whatever you do. In that case, 1-decider case dominates your decision, for essentially different reasons (so "nay" is better).
• The most difficult case is the more "natural" assumption that the other people are basically idential copies of you. In this case, what you have is a stochastic version of Newcomb's Paradox. Actually it's more like Newcomb's Mugging. Once you know you are a decider, saying "yea" will genuinely give you a better average payout. But, when you're not a decider, you'll see more money donated if you are the sort of person who would say "nay" when you are a decider. It's obvious why that's true if you make the "identical copies" assumption explicitly, because your identical copy will say "nay" and cause a larger donation. So, you expect to make more money saying "yea" once you know you're a decider, but you make more money overall if you're a "nay"-sayer. The trick is that most of the extra money is made in the cases where you're not a decider.

So, nay is definately the better precommitment, and TDT would presumably tell you to say nay. But the argument that, as an individual, your expected returns are better saying "yea" once you know you are a decider is still perfectly valid.

Comment author: [deleted] 17 January 2013 08:32:21PM *  -1 points [-]

Initially either 9 or 1 of the 10 people will have been chosen with equal likelihood, meaning I had a 50% chance of being chosen. If being chosen means I should find 90% likelihood that the coin came up tails, then not being chosen should mean I find 90% likelihood that the coin came up heads (which it does). If that were the case, I'd want nay to be what the others choose (0.9 * 100 + 0.1 * 1000 = 190 < 700). Since both branches are equally likely (initially), and my decision of what to choose in the branch in which I choose corresponds (presumably) to the others' decision of what to choose in the branch in which I don't choose, I should therefore say nay. You can't win with mere straightforward optimization when your action in 1 branch affects other (no longer reachable) branches.