You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

jimrandomh comments on Pascal's Mugging as an epistemic problem - Less Wrong Discussion

3 [deleted] 04 October 2010 05:52PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (37)

You are viewing a single comment's thread. Show more comments above.

Comment author: jimrandomh 07 October 2010 11:49:04AM 0 points [-]

He doesn't get to make you believe anything he chooses; making you believe statements of the form "The mugger said X(n)" is entirely sufficient.

Comment author: RichardKennaway 07 October 2010 12:20:13PM 0 points [-]

There would have to be statements X(n) such that the maximum over E of P(E|The mugger said X(n)) U(E|The mugger said X(n)) is unbounded in n. I don't see why there should be, even if the maximum over E of P(E|X) U(E|X) is unbounded in X.

Comment author: jimrandomh 07 October 2010 12:54:21PM 0 points [-]

There would have to be statements X(n) such that the maximum over E of P(E|The mugger said X(n)) U(E|The mugger said X(n)) is unbounded in n.

Yes, and that is precisely what I said causes vulnerability to Pascal's Mugging and should therefore be forbidden. Does your version of the anti-mugging axiom ensure that no such X exists, and can you prove it mathematically?

Comment author: RichardKennaway 07 October 2010 09:03:00PM *  0 points [-]

It does not ensure that no such X exists, but I think this scenario is outside the scope of your suggestion, which is expressed in terms of P(X) and U(X), rather than conditional probabilities and utilities.

What do you think of the other potential defect in a decision theory resulting from too strong an anti-mugging axiom: the inability to believe in the possibility of a sufficiently large amount of utility, regardless of any evidence?

Comment author: jimrandomh 07 October 2010 09:57:10PM 0 points [-]

Oh, so that's where the confusion is coming from; the probabilities and utilities in my formulation are conditional, I just chose the notation poorly. Since X is a function of type number=>evidence-set, P(X(n)) means the probability of something (which I never assigned a variable name) given X(n), and U(X(n)) is the utility of that same thing given X. Giving that something a name, as in your notation, these would be P(E|X) and U(E|X).

Being unable to believe in sufficiently large amounts of utility regardless of any evidence would be very bad; we need to be careful not to phrase our anti-mugging defenses in ways that would do that. This is a problem with globally bounded utility functions, for example. I'm pretty sure that requiring all parameterized statements to produce expected utility that does not diverge to infinity as the parameter increases, does not cause any such problems.