shokwave comments on Counterfactual Calculation and Observational Knowledge - Less Wrong

11 Post author: Vladimir_Nesov 31 January 2011 04:28PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (183)

You are viewing a single comment's thread. Show more comments above.

Comment author: shokwave 31 January 2011 06:11:40PM *  2 points [-]

Q is most likely even,

Derived from the likelihood of the calculator being in error

so in the counterfactual the calculator is most likely in error,

You can't conclude this - think about what evidence you have that the calculator is in error!

Comment author: MC_Escherichia 31 January 2011 06:45:08PM 1 point [-]

You can't conclude this

Yes you can. The real calculator in the real world had a 99% chance of being right. The counterfactual case is (in all probability) the 1% chance where it was wrong.

Comment author: Nisan 31 January 2011 06:54:44PM *  -1 points [-]

Nah. See, given that the real calculator says "even", there's a 0.99% chance that it's correct and that, in a repetition of the experiment, it would say incorrectly say "odd". There's also a 0.99% chance that the real calculator is incorrect and that, in a repetition of the experiment, it would correctly say "odd". The counterfactual case is just as likely to be the calculator being correct as the calculator being incorrect.

ETA: The above is wrong. I was confused about the problem because I wasn't thinking updatelessly. It's like Newcomb's problem.

Comment author: MC_Escherichia 31 January 2011 07:03:19PM *  3 points [-]

I'm not following you.

Imagine this scenario happens 10000 times, with different formulae.

In 9900 of those cases, the calculator says <correct-answer>, and Omega asks what the answer is if the calculator says <incorrect-answer>.

In 100 of those cases, the calculator says <incorrect-answer>, and Omega asks what the answer is if the calculator says <correct-answer>.

So you are more likely to be in the first scenario.

Comment author: AlexMennen 31 January 2011 07:27:09PM *  4 points [-]

This is a dispute over the premises of the problem (whether Omega's counterfactual is always different than yours, or is correct 99% of the time and independent of yours), not a dispute about how to solve the problem. The actual premise needs to be made clear before the question can be properly answered.

Comment author: Manfred 01 February 2011 10:02:24AM *  2 points [-]

"I have no probability assignment, you haven't told me your motives" is not an allowed answer. Pretend Omega holds a gun to your head and will fire unless you answer in ten seconds. There is always some information, I promise. You can avoid getting shot.

EDIT: Upon reflection, this post was too simplistic. If we have some prior information about Omega (e.g. can we ascribe human-like motives to it?), then we would have to use it in making our decision, which would add an element of apparent subjectivity. But I think it's safe to make the simplifying assumption that we can't say anything about Omega, to preserve the intent of the question.

Comment author: AlexMennen 01 February 2011 10:25:13PM 0 points [-]

If Omega doesn't tell you what premises he's using, then you will have some probability distribution over possible premises. However, that distribution, or the information that led to it, needs to be made explicit for the thought experiment to be useful.

If you assume that your prior is 50% that Omega's counterfactual is always different than yours and 50% that it is independent, then updating on the fact that the counterfactual is different in this case gives you a posterior of 99% always different and 1% independent. This means that there is a (99%)^2 + 0.5% chance that your answer is right, and a 1.49% chance that your answer is wrong.

Interestingly, since the question is about math rather than a feature of the world, your answer should be the same for real life and the counterfactual, meaning that if you know that the counterfactual calculator is right mod 2 99% of the time and independent of yours, you should be indifferent to righting "even" or "odd" on your real-life paper.

Comment author: Manfred 02 February 2011 02:30:35AM *  0 points [-]

Good point, but I think the following is wrong: "Interestingly, since the question is about math rather than a feature of the world, your answer should be the same for real life and the counterfactual." This does not follow. The correct answer is the same, yes, but the best answer you can give depends on your state of knowledge, not on the unknown true answer. I would argue that you should give the best answer you can, since that's the only way to give an answer at all.

Comment author: AlexMennen 02 February 2011 03:42:28AM 1 point [-]

The question wasn't "What would you write if the calculator said odd?". It was "Given that you already know your calculator says even, what answer would you like written down in the counterfactual in which the calculator said odd?". This means that you are not obligated to ignore any evidence in either real life or the counterfactual, and the answers are the same in each. Therefor your probability distribution should be the same in regards to the answers in each.

Comment author: Manfred 02 February 2011 04:04:27AM *  -1 points [-]

Omega asks you "what is the true answer?" Vladimir asks you "what does Omega say in the counterfactual that your calculator returned odd?" Since Omega always writes the true answer at the end, the question is equivalent to "what is the true answer if your calculator returned odd?" Since the true answer is not affected by the calculator, this is further equivalent to "What is the true answer?"

So it's possible we were just answering different questions.

Comment author: shokwave 01 February 2011 06:11:08AM *  0 points [-]

Imagine this scenario happens 10000 times, with different formulae.

Given our prior, 5000 of the times the actual answer is even, and 5000 times the answer is odd.

In 4950 of the 5000 Q-is-even cases, the calculator says <correct answer>. And in the other 50 cases of Q-is-even, the calculator says <incorrect answer>. Then, in 4950 of the Q-is-odd cases, the calculator says <correct answer> and in 50 cases it says <incorrect answer>. Note that we still have 9900 cases of <correct answer> and 100 cases of <incorrect answer>.

Omega presents you with a counterfactual world that might be one of the 50 cases of Q-is-even, <incorrect-answer> or one of the 4950 cases of Q-is-odd, <correct answer>. So you're equally likely (5000:5000) to be in either scenario (Q-is-odd, Q-is-even) for actually writing down the right answer (as opposed to writing down the answer the calculator gave you).

Comment author: MC_Escherichia 01 February 2011 09:46:06AM *  0 points [-]

I'm still not following. Either the answer is even in every possible world, or it is odd in every possible world. It can't be legitimate to consider worlds where it is even and worlds where it is odd, as if they both actually existed.

Comment author: Vladimir_Nesov 01 February 2011 03:01:03PM *  2 points [-]

Either the answer is even in every possible world, or it is odd in every possible world. It can't be legitimate to consider worlds where it is even and worlds where it is odd, as if they both actually existed.

If you don't know which is the case, considering such possibly impossible possible worlds is a standard tool. When you're making a decision, all possible decisions except the actual one are actually impossible, but you still have to consider those possibilities, and infer their morally relevant high-level properties, in the course of coming to a decision. See, for example, Controlling Constant Programs.

Comment author: shokwave 01 February 2011 09:54:21AM 1 point [-]

Either the answer is even in every possible world, or it is odd in every possible world.

Which is the case? What do you do if you're uncertain about which is the case?

Comment author: MC_Escherichia 01 February 2011 10:33:01AM *  0 points [-]

Which is the case?

Your initial read off your calculator tells you with 99% certainty.

Now Omega comes in and asks you to consider the opposite case. It matters how Omega decided what to say to you. If Omega was always going to contradict your calculator, then what Omega says offers no new information. But if Omega essentially had its own calculator, and was always going to tell you the result even if it didn't contradict yours, then the probabilities become 50%.

Comment author: Manfred 01 February 2011 12:45:36PM 0 points [-]

True, but I'd like to jump in and say that you can still make a probability estimate with limited information - that's the whole point of having probabilities, after all. If you had unlimited information it wouldn't be much of a probability.

Comment author: lukstafi 31 January 2011 07:53:18PM *  0 points [-]

You can't conclude this - think about what evidence you have that the calculator is in error!

You can't conclude this, but for a different reason: changing the value on the display means changing Omega. You cannot have the same Omega and a different value of the same process. (ETA: and calling the name of Everett does not affect my reasoning here ETA2 meaning that I don't think Q is most likely even in the counterfactual universe.)

Comment author: Nisan 31 January 2011 06:49:02PM 0 points [-]

Oh, you're right. I see.