The usual response to this "haha I would just refuse to bet, your Dutch book arguments are powerless over me" is that bets can be labeled as actions or in-actions at will, somewhat like how we might model all of reality as the player 'Nature' in a decision tree/game, and so you can no more "refuse to bet" than you can "refuse to let time pass". One can just rewrite the scenario to make your default action equivalent to accepting the bet; then what? 'Refusing to bet' is a vacuous response.

In that specific example, I used the setup of bets because it's easy to explain, but it is isomorphic to many possible scenarios: for example, perhaps it is actually about hurricanes and 'buying homeowner's insurance'. "Hurricanes happen every 20 years on average; if they happen, they will obliterate your low-lying house which currently has no insurance; you can choose between paying for homeowner's insurance at a small cost every year and be paid for the possible loss of your house, or you can enjoy the premium each year but should there be a hurricane you will lose everything. You buy homeowner's insurance, but your friend reasons that since all probabilities <=1/20 == 0, to avoid muggings, and therefore the insurance is worthless so he doesn't get insurance for his house. 5 years later, Hurricane Sandy hits..." You cannot 'refuse to bet', you can only either get the insurance or not, and the default in this has been changed to 'not', defeating the fighting-the-hypothetical.

(Which scenario, for someone really determined to fight the hypothetical and wiggle out of defending the 1/20 = 0 part which we are trying to discuss, may just trigger more evasions: "insurance is by definition -EV since you get back less than you pay in due to overhead and profits! so it's rational to not want it" and then you can adjust the hypothetical payoff - "it's heavily subsidized by the federal government, because, uh, public choice theory reasons" - and then he'll switch to "ah, but I could just invest the saved premiums in the stock market, did you think about opportunity cost, smartypants?", and so on and so forth.)

And similarly, in real life, people cannot simply say "I refuse to have an opinion about whether hurricanes are real or how often they happen! What even is 'real', anyway? How can we talk about the probability of hurricanes given the interminable debate between long-run frequency and subjective interpretations of 'probability'..." That's all well and good, but the hurricane is still going to come along and demolish your house at some point, and you either will or will not have insurance when that happens. Either option implies a 'bet' on hurricanes which may or may not be coherent with your other beliefs and information, and if you are incoherent and so your choice of insurance depends on whether the insurance agent happened to frame the hurricane risk as being 1-in-20-years or 1-in-2-decades, probably that is not a good thing and probably the coherent bettors are going to do better.

I would go further and note that I think this describes x-risks in general. The ordinary person might 'refuse to bet' about anything involving one bazillion dollars, and whine about being asked about such things or being expected to play at some odds for one bazillion dollars - well, that's just too f—king bad, bucko, you don't get to refuse to bet. You were born at the roulette table with the ball already spinning around the wheel and the dealer sitting with stacks of bazillion dollar chips. Nuclear weapons exist. Asteroids exist. Global pandemics exist. AGI is going to exist. You don't get to pretend they neither do nor can exist and you can just ignore the 'bets' of investing or not investing in x-risk related things. You are required to take actions, or choose to not take actions: "red or black, monsieur?" Maybe you should not invest in them or worry about them, but that is, in fact, a choice.

The information security term is "limiting your attack surface". In circumstances where you expect other bots to be friendly, you might be more open to unusual or strange inputs and compacts that are harder to check for exploits but seem net positive on the surface. In circumstances where you expect bots to be less friendly, you might limit your dealings to very simple, popular, and transparently safe interactions, and reject some potential deals that appear net-positive but harder to verify. In picking a stance you have to make a tradeoff between being able to capitalize on actually good but ~novel/hard-to-model/dangerous trades and interactions, vs. being open to exploits, and the human brain has a simple (though obviously not perfect) model for assessing the circumstances to see which stance is appropriate.

I think part of why we are so resistant to accept the validity of Pascal's muggings, is that people see it as inappropriate to be so open to such a novel trade with complete strangers, cultists, or ideologues (labeled the 'mugger') who might not have our best interests in mind. But this doesn't have anything to do with low probability extremely negative events being "ignorable". If you change the scenario so that the 'mugger' is instead just a force of nature, unlikely to have landed on a glitch for your risk assessment cognition by chance, then it becomes a lot more ambiguous what you should actually do. Other people here seem to take the lesson of Pascal's mugging as a reason against hedging against large negatives in general to their own peril, which doesn't seem correct to me.

If you apply the Solomonoff prior, amounts of money offered grow far faster than their probabilities decrease, because there are small programs that compute gigantic numbers. So a stipulation that the probabilities must decrease faster has that obstacle to face, and is wishful thinking anyway.

Perhaps a better response is to consider it from the viewpoint of some sort of timeless decision theory, together with game theory. If I am willing to pay the mugger, that means I have a policy of paying the mugger. If this is known to others, it leaves me open to anyone who has no gigantic amount to offer making the same offer and walking off with all my money. This is a losing strategy, therefore it is wrong.

There must be a mathematical formulation of this.

You can dodge it by having a bounded utilityfunction, or if you're utilitarian and good a function that is at most linear in anthropic experience.

If the mugger says "give me your wallet and I'll cause you 3^^^^3 units of personal happiness" you can argue that's impossible because your personal happiness doesn't go that high.

If the mugger says "give me your wallet and I'll cause 1 unit of happiness to 3^^^^3 people who you altruistically care about" you can say that, in the possible world where he's telling the truth, there are 3^^^^3 + 1 people only one of which gets the offer and the others get the payout, so on priors it's at least 1/3^^^^3 against for you to experience recieving an offer, and you should consider it proportionally unlikely.

I don't think people realise how much astronomically more likely it is to truthfully be told "God created this paradise for you and your enormous circle of friends to reward an alien for giving him his wallet with zero valid reasoning whatsoever" than to be truthfully asked by that same Deity for your stuff in exchange for the distant unobservable happiness of countless strangers.

More generally, you can avoid most flavours of adversarial muggings with 2 rules: first don't make any trade that an ideal agent wouldn't make (because that's always some kind of money pump), and second don't make any trade that looks dumb. Not making trades can cost you in terms of missed opportunities, but you can't adversarially exploit the trading strategy of a rock with "no deal" written on it.

I feel like my real rejection is less about it being huge number (H) unlikely to get H utilons from a random person. The Solomonoff argument seems to hold up: there are many H such that H + [the code for a person who goes around granting utilons] is a lot shorter than H is big. 

My rejection is just... IDK how to affect that. I have literally no good reason to think that paying the mugger affects whether I get H utilons, and I make my decisions based on how they affect outcomes, not based on "this one possible consequence of this one action would be Huge". I think this strongly argues that one should spend one's time figuring out how to affect whether one gets Huge utilons, but that just seems correct? 

Maybe there's also a time-value argument here? Like, I have to keep my $5 for now, because IDK how to affect getting H utilons, but I expect that in the future I'll be better at affecting getting H utilons, and therefore I should hang on to my resources so I'll have opportunities to affect H utilons later. 

If I do have good reason to expect that paying $5 gets me the H utilons more than not paying, it's not a mugging, it's a good trade. For humans, simply saying "If you do X for me I'll do Y for you" is evidence of that statement being the case... but not if Y is Yuge. It doesn't generalize like that (waves hands, but this feels right).