You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

kokotajlod comments on New Pascal's Mugging idea for potential solution - Less Wrong Discussion

2 Post author: kokotajlod 04 August 2016 08:38PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (18)

You are viewing a single comment's thread. Show more comments above.

Comment author: kokotajlod 10 August 2016 05:37:50PM 0 points [-]

This was helpful, thanks!

As I understand it, you are proposing modifying the example so that on some H1 through HN, choosing A gives you less utility than choosing B, but then thereafter choosing B is better, because there is some cost you pay which is the same in each world.

It seems like the math tells us that any price would be worth it, that we should give up an unbounded amount of utility to choose A over B. I agree that this seems like the wrong answer. So I don't think whatever I'm proposing solves this problem.

But that's a different problem than the one I'm considering. (In the problem I'm considering, choosing A is better in every possible world.) Can you think of a way they might be parallel--any way that the "I give up" which I just said above applies to the problem I'm considering too?

Comment author: ike 10 August 2016 07:14:11PM *  0 points [-]

The problem there, and the problem with Pascal's Mugging in general, is that outcomes with a tiny amount of probability dominate the decisions. A could be massively worse than B 99.99999% of the time, and still naive utility maximization says to pick B.

One way to fix it is to bound utility. But that has its own problems.

The problem with your solution is that it's not complete in the formal sense: you can only say some things are better than other things if they strictly dominate them, but if neither strictly dominates the other you can't say anything.

I would also claim that your solution doesn't satisfy framing invariants that all decision theories should arguably follow. For example, what about changing the order of the terms? Let us reframe utility as after probabilities, so we can move stuff around without changing numbers. E.g. if I say utility 5, p:.01, that really means you're getting utility 500 in that scenario, so it adds 5 total in expectation. Now, consider the following utilities:

1<2 p:.5

2<3 p:.5^2

3<4 p:.5^3

n<n+1 p:.5^n

...

etc. So if you're faced with choosing between something that gives you the left side or the right side, choose the right side.

But clearly re-arranging terms doesn't change the expected utility, since that's just the sum of all terms. So the above is equivalent to:

1>0 p:.5

2>0 p:.5^2

3>2 p:.5^3

4>3 p:.5^4

n>n-1 p:.5^n

So your solution is inconsistent if it satisfies the invariant of "moving around expected utility between outcomes doesn't change the best choice".

Comment author: kokotajlod 18 August 2016 03:24:22PM 0 points [-]

Again, thanks for this.

"The problem with your solution is that it's not complete in the formal sense: you can only say some things are better than other things if they strictly dominate them, but if neither strictly dominates the other you can't say anything."

As I said earlier, my solution is an argument that in every case there will be an action that strictly dominates all the others. (Or, weaker: that within the set of all hypotheses of probability less than some finite N, one action will strictly dominate all the others, and that this action will be the same action that is optimal in the most probable hypothesis.) I don't know if my argument is sound yet, but if it is, it avoids your objection, no?

I'd love to understand what you said about re-arranging terms, but I don't. Can you explain in more detail how you get from the first set of hypotheses/choices (which I understand) to the second?

Comment author: ike 19 August 2016 10:47:14AM 0 points [-]

I'd love to understand what you said about re-arranging terms, but I don't. Can you explain in more detail how you get from the first set of hypotheses/choices (which I understand) to the second?

I just moved the right hand side down by two spaces. The sum still stays the same, but the relative inequality flips.

As I said earlier, my solution is an argument that in every case there will be an action that strictly dominates all the others.

Why would you think that? I don't really see where you argued for that, could you point me at the part of your comments that said that?