Sorry for not responding earlier; I had to think about this a bit. Whether the presence of astronomically large numbers can make you vulnerable to Pascal's Mugging seems to be a property of the interaction between the method you use to assign probabilities from evidence, and your utility function. Call the probability-assignment method P(X), which takes a statement X and returns a probability; and the utility function U(X), which assigns a utility to something (such as the decision to pay the mugger) based on the assumption that X is true.
P and U are vulnerable to Pascal's Mugging if and only if you can construct sets of evidence X(n), which differ only by a single number n, such that for any utility value u, there exists n such that P(X(n))U(X(n)) > u.
Now, I really don't know of any reason apart from Pascal's Mugging why utility function-predictor pairs should have this property. But being vulnerable to Pascal's Mugging is such a serious flaw, I'm tempted to say that it's just a necessary requirement for mental stability, so any utility function and predictor which don't guarantee this when they're combined should be considered incompatible.
But being vulnerable to Pascal's Mugging is such a serious flaw, I'm tempted to say that it's just a necessary requirement for mental stability, so any utility function and predictor which don't guarantee this when they're combined should be considered incompatible.
Is the wording of this correct? Did you mean to say that vulnerability to Pascal's mugging is a necessary requirement for mental stability or the opposite?
Related to: Some of the discussion going on here
In the LW version of Pascal's Mugging, a mugger threatens to simulate and torture people unless you hand over your wallet. Here, the problem is decision-theoretic: as long as you precommit to ignore all threats of blackmail and only accept positive-sum trades, the problem disappears.
However, in Nick Bostrom's version of the problem, the mugger claims to have magic powers and will give Pascal an enormous reward the following day if Pascal gives his money to the mugger. Because the utility promised by the mugger so large, it outweighs Pascal's probability that he is telling the truth. From Bostrom's essay:
As a result, says Bostrom, there is nothing from rationally preventing Pascal from taking the mugger's offer even though it seems intuitively unwise. Unlike the LW version, in this version the problem is epistemic and cannot be solved as easily.
Peter Baumann suggests that this isn't really a problem because Pascal's probability that the mugger is honest should scale with the amount of utility he is being promised. However, as we see in the excerpt above, this isn't always the case because the mugger is using the same mechanism to procure the utility, and our so our belief will be based on the probability that the mugger has access to this mechanism (in this case, magic), not the amount of utility he promises to give. As a result, I believe Baumann's solution to be false.
So, my question is this: is it possible to defuse Bostrom's formulation of Pascal's Mugging? That is, can we solve Pascal's Mugging as an epistemic problem?