This has been in my drafts folder for ages, but in light of Eliezer's post yesterday, I thought I'd see if I could get some comment on it:
A couple weeks ago, Vladimir Nesov stirred up the biggest hornet's nest I've ever seen on LW by introducing us to the Counterfactual Mugging scenario.
If you didn't read it the first time, please do -- I don't plan to attempt to summarize. Further, if you don't think you would give Omega the $100 in that situation, I'm afraid this article will mean next to nothing to you.
So, those still reading, you would give Omega the $100. You would do so because if someone told you about the problem now, you could do the expected utility calculation 0.5*U(-$100)+0.5*U(+$10000)>0. Ah, but where did the 0.5s come from in your calculation? Well, Omega told you he flipped a fair coin. Until he did, there existed a 0.5 probability of either outcome. Thus, for you, hearing about the problem, there is a 0.5 probability of your encountering the problem as stated, and a 0.5 probability of your encountering the corresponding situation, in which Omega either hands you $10000 or doesn't, based on his prediction. This is all very fine and rational.
So, new problem. Let's leave money out of it, and assume Omega hands you 1000 utilons in one case, and asks for them in the other -- exactly equal utility. What if there is an urn, and it contains either a red or a blue marble, and Omega looks, maybe gives you the utility if the marble is red, and asks for it if the marble is blue? What if you have devoted considerable time to determining whether the marble is red or blue, and your subjective probability has fluctuated over the course of you life? What if, unbeknownst to you, a rationalist community has been tracking evidence of the marble's color (including your own probability estimates), and running a prediction market, and Omega now shows you a plot of the prices over the past few years?
In short, what information do you use to calculate the probability you plug into the EU calculation?
Let me try restating the scenario more explicitly, see if I understand that part.
Omega comes to you and says, "There is an urn with a red or blue ball in it. I decided that if the ball were blue, I would come to you and ask you to give me 1000 utilons. Of course, you don't have to agree. I also decided that if the ball were red, I would come to you and give you 1000 utilons - but only if I predicted that if I asked you to give me the utilons in the blue-ball case, you would have agreed. If I predicted that you would not have agreed to pay in the blue-ball case, then I would not pay you in the red-ball case. Now, as it happens, I looked at the ball and found it blue. Will you pay me 1000 utilons?"
The difference from the usual case is that instead of a coin flip determining which question Omega asks, we have the ball in the urn. I am still confused about the significance of this change.
Is it that the coin flip is a random process, but that the ball may have gotten into the urn by some deterministic method?
Is it that the coin flip is done just before Omega asks the question, while the ball has been sitting in the urn, unchanged, for a long time?
Is it that we have partial information about the urn state, therefore the odds will not be 50-50, but potentially something else?
Is it the presence of a prediction market that gives us more information about what the state of the urn is?
Is it that our previous estimates, and those of the prediction market, have varied over time, rather than being relatively constant? (Are we supposed to give some credence to old views which have been superseded by newer information?)
Another difference is that in the original problem, the positive payoff was much larger than the negative one, while in this case, they are equal. Is that significant?
And once again, if this were not an Omega question, but just some random person offering a deal whose outcome depended on a coin flip vs a coin in an urn, why don't the same considerations arise?
Randomness is uncertainty, and determinism doesn't absolve you of uncertainty. If you find yourself wondering what exactly was that deterministic process that fits your incomplete knowledge, it is a thought about randomness. A coin flip is as random as a pre-placed ball in an urn, both in deterministic and stochastic worlds, so long as you don't know what the outcome is, based on the given state of knowledge.
... (read more)