Lapsed_Lurker comments on Why do theists, undergrads, and Less Wrongers favor one-boxing on Newcomb? - Less Wrong

15 Post author: CarlShulman 19 June 2013 01:55AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (299)
Tags: philosophy newcomb

Comments (299)

You are viewing a single comment's thread. Show more comments above.

Comment author: Lapsed_Lurker 19 June 2013 10:52:25PM 0 points [-]

Omega gives you a choice of either $1 or $X, where X is either 2 or 100?

It seems like you must have meant something else, but I can't figure it out.

Comment author: solipsist 20 June 2013 12:40:54AM *  0 points [-]

Omega gives you a choice of either $1 or $X, where X is either 2 or 100

Yes, that's what I mean. I'd like to know what, if anything, is wrong with this argument that no decision theory can be optimal.

Suppose that there were a computable decision theory T that was at least as good as all other theories. In any fair problem, no other decision theory could recommend actions with better expected outcomes than the expected outcomes of T's recommended actions.

  1. We can construct a computable agent, BestDecisionAgent, using theory T.
  2. For any fair problem, no computable agent can perform better (on average) than BestDecisionAgent.
  3. Call the problem presented in the grandfather post the Prejudiced Omega Problem. In the Prejudiced Omega Problem, BestDecisionAgent will almost assuredly collect $2.
  4. In the Prejudiced Omega Problem, another agent can almost assuredly collect $100.
  5. The Prejudiced Omega Problem does not involve an Omega inspecting the source code of the agent.
  6. The Prejudiced Omega Problem, like Newcomb's problem, is fair.
  7. Contradiction

I'm not asserting this argument is correct -- I just want to know where people disagree with it.

Qiaochu_Yuan's post is related.

Comment author: klkblake 20 June 2013 11:16:16AM *  0 points [-]

Let BestDecisionAgent choose the $1 with probability p. Then the various outcomes are:

Simulation's choice | Our Choice | Payoff
$1 | $1 = $1
$1 | $2 or $100 = $100
$2 or $100 | $1 = $1
$2 or $100 | $2 or $100 = $2

And so p should be chosen to maximise p^2 + 100p(1-p) + p(1-p) + 2(1-p)^2. This is equal to the quadratic -98p^2 + 97p + 2, which Wolfram Alpha says is maximised by p = 97/196, for a expected payoff of ~$26.

If we are not BestDecisionAgent, and so are allowed to choose separately, we aim to maximise pq + 100p(1-q) + q(1-p) + 2(1-p)(1-q), which simplifies to -98pq+98p-q+2, which is maximized by q = 0, for a payoff of ~$50.5. This surprises me, I was expecting to get p = q.

So (3) and (4) are not quite right, but the result is similar. I suspect BestDecisionAgent should be able to pick p such that p = q is the best option for any agent, at the cost of reducing the value it gets.

ETA: Of course you can do this just by setting p = 0, which is what you assume. Which, actually, means that (3) and (4) contradict each other: if BestDecisionAgent always picks the $2 over the $1, then the best any agent can do is $2.

(Incidentally, how do you format tables properly in comments?)

Comment author: solipsist 20 June 2013 01:30:36PM *  0 points [-]

$100 if the Omega thinks the agent acts differently than BestDecisionAgent in a simulated rationality test, otherwise $2 if the agent acts like BestDecisionAgent in the rationality test.

The Omega chooses payoff of $2 vs. $100 based off of a separate test that can differentiate between BestDecisionAgent and some other agent. If we are BestDecisionAgent, the Omega will know this and will be offered at most a $2 payoff. But some other agent will be different from BestDecisionAgent in a way that the Omega detects and cares about. That agent can decide between $1 and $100. Since another agent can perform better than BestDecisionAgent, BestDecisionAgent cannot be optimal.

Comment author: klkblake 20 June 2013 01:46:57PM 1 point [-]

Ah, ok. In that case though, the other agent wins at this game at the expense of failing at some other game. Depending on what types of games the agent is likely to encounter, this agents effectiveness may or may not actually be better than BestDecisionAgent. So we could possibly have an optimal decision agent in the sense that no change to its algorithm could increase its expected lifetime utility, but not to the extent of not failing in any game.

Powered by Reddit Powered by Reddit