How should Eliezer and Nick's extra $20 be split

Scott Garrabrant

In "Principles of Disagreement," Eliezer Yudkowsky shared the following anecdote:

Nick Bostrom and I once took a taxi and split the fare. When we counted the money we'd assembled to pay the driver, we found an extra twenty there.

"I'm pretty sure this twenty isn't mine," said Nick.

"I'd have been sure that it wasn't mine either," I said.

"You just take it," said Nick.

"No, you just take it," I said.

We looked at each other, and we knew what we had to do.

"To the best of your ability to say at this point, what would have been your initial probability that the bill was yours?" I said.

"Fifteen percent," said Nick.

"I would have said twenty percent," I said.

I have left off the ending to give everyone a chance to think about this problem for themselves. How would you have split the twenty?

In general, EY and NB disagree about who deserves the twenty. EY believes that EY deserves it with probability p, while NB believes that EY deserves it with probability q. They decide to give EY a fraction of the twenty equal to f(p,q). What should the function f be?

In our example, p=1/5 and q=17/20

Please think about this problem a little before reading on, so that we do not miss out on any original solutions that you might have come up with.

I can think of 4 ways to solve this problem. I am attributing answers to the person who first proposed that dollar amount, but my reasoning might not reflect their reasoning.

f=p/(1+p-q) or $11.43 (Eliezer Yodkowsky/Nick Bostrom) -- EY believes he deserves p of the money, while NB believes he deserves 1-q. They should therefore be given money in a ratio of p:1-q.
f=(p+q)/2 or $10.50 (Marcello) -- It seems reasonable to assume that there is a 50% chance that EY reasoned properly and a 50% chance that NB reasoned properly, so we should take the average of the amounts of money that EY would get under these two assumptions.
f=sqrt(pq)/(sqrt(pq)+sqrt((1-p)(1-q))) or $10.87 (GreedyAlgorithm) -- We want to chose an f so that log(f/(1-f)) is the average of log(p/(1-p)) and log(q/(1-q)).
f=pq/(pq+(1-p)(1-q)) or $11.72 -- We have two observations that EY deserves the money with probability p and probability q respectively. If we assume that these are two independent pieces of evidence as to whether or not EY should get the money, then starting with equal likelihood of each person deserving the money, we should do a Bayesian update for each piece of information.

I am very curious about this question, so if you have any opinions, please comment. I have some opinions on this problem, but to avoid biasing anyone, I will save them for the comments. I am actually more interested in the following question. I believe that the two will have the same answer, but if anyone disagrees, let me know.

I have two hypotheses, A and B. I assign probability p to A and probability q to B. I later find out that A and B are equivalent. I then update to assign the probability g(p,q) to both hypotheses. What should the function g be?

In "Principles of Disagreement," Eliezer Yudkowsky shared the following anecdote:

Nick Bostrom and I once took a taxi and split the fare. When we counted the money we'd assembled to pay the driver, we found an extra twenty there.

"I'm pretty sure this twenty isn't mine," said Nick.

"I'd have been sure that it wasn't mine either," I said.

"You just take it," said Nick.

"No, you just take it," I said.

We looked at each other, and we knew what we had to do.

"To the best of your ability to say at this point, what would have been your initial probability that the bill was yours?" I said.

"Fifteen percent," said Nick.

"I would have said twenty percent," I said.

I have left off the ending to give everyone a chance to think about this problem for themselves. How would you have split the twenty?

In our example, p=1/5 and q=17/20

Please think about this problem a little before reading on, so that we do not miss out on any original solutions that you might have come up with.

I can think of 4 ways to solve this problem. I am attributing answers to the person who first proposed that dollar amount, but my reasoning might not reflect their reasoning.

f=p/(1+p-q) or $11.43 (Eliezer Yodkowsky/Nick Bostrom) -- EY believes he deserves p of the money, while NB believes he deserves 1-q. They should therefore be given money in a ratio of p:1-q.
f=(p+q)/2 or $10.50 (Marcello) -- It seems reasonable to assume that there is a 50% chance that EY reasoned properly and a 50% chance that NB reasoned properly, so we should take the average of the amounts of money that EY would get under these two assumptions.
f=sqrt(pq)/(sqrt(pq)+sqrt((1-p)(1-q))) or $10.87 (GreedyAlgorithm) -- We want to chose an f so that log(f/(1-f)) is the average of log(p/(1-p)) and log(q/(1-q)).
f=pq/(pq+(1-p)(1-q)) or $11.72 -- We have two observations that EY deserves the money with probability p and probability q respectively. If we assume that these are two independent pieces of evidence as to whether or not EY should get the money, then starting with equal likelihood of each person deserving the money, we should do a Bayesian update for each piece of information.

"This bill clearly belongs to exactly one of the two us; I think it is 80% likely to be yours and 20% likely to be mine, and you think it is 85% likely to be mine and 15% likely to be yours. In order to fairly divide it, we weigh our beliefs equally (by symmetry), and divide it according to the ratio 80+15:20+85 you:me; that's 95:105, or 47.5:52.5. That's a 52.5% chance that I get the bill, and a 47.5% chance that you get it." (Same expected values as 2, from equivalent math)

I don't think there's a general case for the function g. Consider the case where hypotheses B, C, and D are mutually exclusive. Proposition A is equivalent to "not C" but B is only known to be mutually exclusive with C. On discovering that D is false, A becomes equivalent to B, but there is also potentially new information about C.

Variant 1: In a departure from Monty Hall; I put a bean under one of the cups B,C,D after rolling a d10. (Time a) Then I show you what is under cup D. There is not a bean. (Time b)

If the odds ratio at time a is 2:4:4, then the odds at time b are 2:4:0. Proposition A went from 6/10 likely to 1/3 likely, while B went from 2/10 to 1/3.

Variant 2: I roll the die; if the result is prime, I put the bean under cup d. Otherwise, if the result is greater than five, I put the bean under cup c. Otherwise I put the bean under cup b. At time a, the odds are 2:4:4. Then I tell you that the number I rolled is a perfect square. You should update to 2:1:0

In both cases, you updated that B and not-C were equivalent, but in the latter case you gained new information about B and C. In the general case, if there is no new information about B and C, then the ratio B:C should remain constant; I intuit that that will probably mean a fairly simple transformation.

22

How should Eliezer and Nick's extra $20 be split

22

22

22

How should Eliezer and Nick's extra $20 be split

22

22