In the dialog you give, Pascal assigns a probability that the mugger will fulfill his promise without hearing what that promise is, then fails to update it when the promise is revealed. But after hearing the number "1000 quadrillion", Pascal would then be justified in updating his probability to something less than 1 in 1000 quadrillion.
I think this might be it, but I'm not sure. Here is the key piece of the puzzle:
Mugger: Wow, you are pretty confident in your own ability to tell a liar from an honest man! But no matter. Let me also ask you, what’s your probability that I not only have magic powers but that I will also use them to deliver on any promise – however extravagantly generous it may seem – that I might make to you tonight?
Pascal: Well, if you really were an Operator from the Seventh Dimension as you assert, then I suppose it’s not such a stretch to suppose that you might also be right in this additional claim. So, I’d say one in 10 quadrillion.
This is why Pascal doesn't update based on the number 1000 quadrillion, because he has already stated his probability for "mugger has magic powers" x "given that mugger has magic, he will deliver on any promise", and this number is less then the utility the mugger claims he will deliver. So I suppose we could say that he isn't justified in doing so, but I don't know how well-supported that claim would be.
Other known defenses against Pascal's mugging are bounded utility functions, and rounding probabilities below some noise floor to zero. Another strategy that might be less likely to carry adverse side effects would be to combine a sub-linear utility function with a prior that assigns statements involving a number N probability at most 1/N (and the Occamian prior does indeed do this).
Yes, those would definitely work, since Bostrom is careful to exclude them in his essay. What I was looking for was more of a fully general solution.
Suppose you say that the probability that the mugger has magical powers, and will deliver on any promise he makes, is 1 in 10^30. But then, instead of promising you quadrillions of days of extra life, the mugger promises to do an easy card trick. What's your estimate of the probability that he'll deliver? (It should be much closer to 0.8 than to 10^-30).
That's because the statement "the mugger will deliver on any promise he makes" carries with it an implied probability distribution over possible promises. If he promises to do a card trick, the pr...
Related to: Some of the discussion going on here
In the LW version of Pascal's Mugging, a mugger threatens to simulate and torture people unless you hand over your wallet. Here, the problem is decision-theoretic: as long as you precommit to ignore all threats of blackmail and only accept positive-sum trades, the problem disappears.
However, in Nick Bostrom's version of the problem, the mugger claims to have magic powers and will give Pascal an enormous reward the following day if Pascal gives his money to the mugger. Because the utility promised by the mugger so large, it outweighs Pascal's probability that he is telling the truth. From Bostrom's essay:
As a result, says Bostrom, there is nothing from rationally preventing Pascal from taking the mugger's offer even though it seems intuitively unwise. Unlike the LW version, in this version the problem is epistemic and cannot be solved as easily.
Peter Baumann suggests that this isn't really a problem because Pascal's probability that the mugger is honest should scale with the amount of utility he is being promised. However, as we see in the excerpt above, this isn't always the case because the mugger is using the same mechanism to procure the utility, and our so our belief will be based on the probability that the mugger has access to this mechanism (in this case, magic), not the amount of utility he promises to give. As a result, I believe Baumann's solution to be false.
So, my question is this: is it possible to defuse Bostrom's formulation of Pascal's Mugging? That is, can we solve Pascal's Mugging as an epistemic problem?