Hmm, apparently that wasn't as clearly expressed as I thought. Let's try that again. I said that a predictor P and utility function U are vulnerable to Pascal's mugging if
exists function X of type number => evidence-set
such that X(a) differs from X(b) only in that one number appearing literally, and
forall u exists n such that P(X(n))U(X(n)) > u
The last line is the delta-epsilon definition for limits diverging to infinity. It could be equivalently written as
lim[n->inf] P(X(n))U(X(n)) = inf
If that limit diverges to infinity, then you could scale the probability down arbitrarily far and the mugger will just give you a bigger n. But if it doesn't diverge that way, then there's a maximum amount of expected utility the mugger can offer you just by increasing n, and the only way to get around it would be to offer more evidence that wasn't in X(n).
(Note that while the limit can't diverge to infinity, it is not required to converge. For example, the Pebblesorter utility function, U(n pebbles) = if(isprime(n)) 1 else 0, does not converge when combined with the null predictor P(X)=0.5.)
(The reductio you gave in the other reply does not apply, because the high-utility statement you gave is not parameterized, so it can't diverge.)
Related to: Some of the discussion going on here
In the LW version of Pascal's Mugging, a mugger threatens to simulate and torture people unless you hand over your wallet. Here, the problem is decision-theoretic: as long as you precommit to ignore all threats of blackmail and only accept positive-sum trades, the problem disappears.
However, in Nick Bostrom's version of the problem, the mugger claims to have magic powers and will give Pascal an enormous reward the following day if Pascal gives his money to the mugger. Because the utility promised by the mugger so large, it outweighs Pascal's probability that he is telling the truth. From Bostrom's essay:
As a result, says Bostrom, there is nothing from rationally preventing Pascal from taking the mugger's offer even though it seems intuitively unwise. Unlike the LW version, in this version the problem is epistemic and cannot be solved as easily.
Peter Baumann suggests that this isn't really a problem because Pascal's probability that the mugger is honest should scale with the amount of utility he is being promised. However, as we see in the excerpt above, this isn't always the case because the mugger is using the same mechanism to procure the utility, and our so our belief will be based on the probability that the mugger has access to this mechanism (in this case, magic), not the amount of utility he promises to give. As a result, I believe Baumann's solution to be false.
So, my question is this: is it possible to defuse Bostrom's formulation of Pascal's Mugging? That is, can we solve Pascal's Mugging as an epistemic problem?