As I said, at this point I am most interested in determining why we disagree,
The meaning of a phrase, primarily. And slightly about the proper use of an abstract concept.
A utility function should be a representation of my values. If my values are such that paying a mugger is the best option then I am glad to pay a mugger.
Suppose that you have somehow calculated that, with probability 10^(-100), the mugger will cause X to happen if you don't pay him $5. Would you pay him? If you would pay him, then why?
If I were to pay him it would be because I happen to value not having a 10^(-100) chance of X happening more than I value $5.
As an aside, if you wouldn't pay him then the definition of utility implies that u($5) > 10^(-100) u(X), which implies that u(X), and therefore the entire utility function, is bounded.
My utility function quite likely is bounded. Not because that is a way around pascal's mugging. Simply because that happens to be what the arbitrary value system represented by this particular bunch of atoms happens to be.
Hm...it sounds like we agree on far more than I thought, then.
What I am saying is that my utility function is bounded because it would be ridiculous to be Pascal's mugged, even in the hypothetical universe I created that disobeys komponisto's priors. Put another way, I am simply not willing to seriously consider events at probabilities of, say, 10^(-10^(100)), because such events don't happen. For this same reason, I have a hard time taking anyone seriously who claims to have an unbounded utility function, because they would then care about events that can...
For background, see here.
In a comment on the original Pascal's mugging post, Nick Tarleton writes:
Coming across this again recently, it occurred to me that there might be a way to generalize Vassar's suggestion in such a way as to deal with Tarleton's more abstract formulation of the problem. I'm curious about the extent to which folks have thought about this. (Looking further through the comments on the original post, I found essentially the same idea in a comment by g, but it wasn't discussed further.)
The idea is that the Kolmogorov complexity of "3^^^^3 units of disutility" should be much higher than the Kolmogorov complexity of the number 3^^^^3. That is, the utility function should grow only according to the complexity of the scenario being evaluated, and not (say) linearly in the number of people involved. Furthermore, the domain of the utility function should consist of low-level descriptions of the state of the world, which won't refer directly to words uttered by muggers, in such a way that a mere discussion of "3^^^^3 units of disutility" by a mugger will not typically be (anywhere near) enough evidence to promote an actual "3^^^^3-disutilon" hypothesis to attention.
This seems to imply that the intuition responsible for the problem is a kind of fake simplicity, ignoring the complexity of value (negative value in this case). A confusion of levels also appears implicated (talking about utility does not itself significantly affect utility; you don't suddenly make 3^^^^3-disutilon scenarios probable by talking about "3^^^^3 disutilons").
What do folks think of this? Any obvious problems?