Tetronian comments on Pascal's Mugging as an epistemic problem - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (37)
Suppose you say that the probability that the mugger has magical powers, and will deliver on any promise he makes, is 1 in 10^30. But then, instead of promising you quadrillions of days of extra life, the mugger promises to do an easy card trick. What's your estimate of the probability that he'll deliver? (It should be much closer to 0.8 than to 10^-30).
That's because the statement "the mugger will deliver on any promise he makes" carries with it an implied probability distribution over possible promises. If he promises to do a card trick, the probability that he delivers on it is very high; if he promises to deliver quadrillions of years of life, it's very low. When you made your initial probability estimate, you didn't know which promise he was going to make. After he reveals the details, you have new information, so you have to update your probability. And if that new information includes an astronomically large number, then your new probability estimate ought to be infinitesimally small in a way that cancels out that astronomically large number.
Agreed, but that's not the whole picture. Let's break this down a slightly different way: we know that p(mugger has magic) is very small number, and as you point out p(mugger will deliver on any promise) is a distribution, not a number. But we aren't just dealing with p(mugger will deliver on any promise), we are dealing with the conditional probability of p(mugger will deliver on any promise|mugger has magic) times p(mugger has magic). Though this might be a distribution based on what exactly the mugger is promising, it is still different from p(mugger will deliver on any promise), and it might still allow for a Pascal's Mugging.
This is why the card trick example doesn't work: p(mugger performs card trick) is indeed very high, but what we are really dealing with is p(mugger performs card trick|mugger has magic) times p(mugger has magic), so our probability that he does a card trick using actual magic would be extremely low.