= 27d567bc2d48d9f40e9ccc20ea370b15)
Benquo comments on St. Petersburg Mugging Implies You Have Bounded Utility - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (163)
Counterargument #1:
If you are god, then the universe allows for "gods" which can arbitrarily alter the state of the universe. Therefore, any utility gains I make have an unknown duration - it's entirely possible that an instant after you grant my utility, you'll take it away. Furthermore, if you are god, you're (a) flipping a coin and (b) requiring a donation, so I strongly suspect you are neither friendly nor omni-benevolent. Therefore, I have no reason to favour "god will help me for $1" over "god will hurt me for $1" - you could just as easily be trying to trap me, and punish anyone who irrationally sends you $1.
1b) I have no reason to select you as a likely god candidate, compared to the ~infinite number of people who exist across all of space-time and all Everett branches.
Counterargument #2:
There are finite many states of "N".
2a) Eventually the universe will succumb to heat death. Entropy means that we can't gain information from the coin flip without approaching this state. 2b) Even if you flip coins incredibly fast and in parallel, I will still eventually die, so we can only count the number of coin flips that happen before then.
Counterargument #3:
Assume a utility function which is finite but unbounded. It cannot handle infinity, and thus your mugging relies on an invalid input (infinite utility), and is discarded as malformed.
3b: Assume that my utility function fails in a universe as arbitrary as the one implied by you being god, since I would have witnessed a proof that state(t+1) does not naturally follow state(t)
Counterargument #4:
Carefully assign p(you are god) = 1/N, where N approaches infinity in such a way as to cancel out the infinite sum you are working with. This seems contrived, but my mind assigns p(you are god) = "bullshit, prove it", and this is about the closest I can come to expressing that mathematically ;)
Counterargument #5:
Assign probabilities by frequency of occurrence. There have been no instances of god yet, so p(god) = 0. Once god has been demonstrated, I can update off of this 0, unlike with Bayesian statistics. My utility function may very well be poorly designed, and I believe this can still allow for FAI research, etc.: social standing, an interest in writing code, peer pressure, etc. all provide motivations even if p(FAI) = 0. One could also assume that even where p(x) = 0, a different function rewards utility for investigating and trying to update even zero-probability events (in which case I'd get some utility from mailing you $1 to satisfy my curiosity, although I suspect not enough to overcome the cost of setting up a PayPal account and losing $1)
Counterargument #1 is similar to argument against Pascal's wager that weights Christianity and anti-Christianity equally. Carl's comment addresses this sort of thing pretty well. That TimFreeman has asserted that you should suspect he is a god is (very small) positive evidence that he is one, that he has the requisite power and intelligence to write a lesswrong post is also very small but positive evidence, &c.
Counterargument #2 implies the nonexistence of gods. I agree that gods are implausible given what we know, but on the other hand, the necessity of entropy and heat-death need not apply to the entire range of UN(X),
I don't understand Counterargument #3. Could you elaborate a little?
Counterargument #4 seems similar to Robin Hanson's argument against the 3^^^3 dust specks variant of Pascal's Mugging, where if I recall correctly he said that you have to discount by the improbability of a single entity exercising such power over N distinct persons, a discount that monotonically scales positively with N. If the discount scales up fast enough, it may not be possible to construct a Pascal's Mugging of infinite expected value. You could maybe justify a similar principle for an otherwise unsupported claim that you can provide N utilons.
Counterargument #5 raises an interesting point: the post implicitly assumes a consistent utility function that recognizes the standard laws of probability, an assumption that is not satisfied by the ability to update from 0.
It's playing on the mathematical difference between infinite and unbounded.
In plain but debatably-accurate terms, infinity isn't a number. If my utility function only works on numbers, you can no more give it "infinity" than you can give it an apple.
As a couple examples: Any given polygon has 'n' sides, and there are thus infinite many polygons, but no polygon has 'infinity' sides. Conversely, there are infinitely many real numbers such that 0 < x < 1, but x is bounded (it has finite limits).
So I'm asserting that while I cannot have "infinity" utility, there isn't any finite bound on my utility: it can be 1, a million, 3^^^3, but not "infinity" because "infinity" isn't a valid input.
Utility doesn't have to take infinity as an argument in order to be infinite. It just has to have a finite output that can be summed over possible outcomes. In other words, if Sum(p X U(a) + (1-p) X U(^a)) is a valid expression of expected utility, then by induction, Sum X U(i)) should also be a valid expression for any finite n. When you take the limit as n->infinity you run into the problem of no finite expectation, but an arbitrarily large finite sum (which you can get with a stopping rule) ought to be able to establish the same point.
I still don't understand 3b. TimFreeman wasn't postulating an acausal universe, just one in which there are things we weren't expecting.
magfrump seems to have nailed it. I find it interesting how controversial that one has been :)
For infinite sums, basically, if the sum is infinite, then any finite probability gives it infinite expected utility (infinity * [1/N] = infinity). If both the sum and probability are finite, then one can argue the details (N * [1/N^2] < 1). The math is different between an arbitrarily large finite and an infinite. Or, at least, I've always assumed Pascal's Wager relied on that, because otherwise I don't see how it produces an infinite expected utility regardless of scepticism.
If the utility can be arbitrarily large depending on N, then an arbitrarily large finite skepticism discount can be overcome by considering a sufficiently large N.
Of course a skepticism discount factor that scales with N might be enough to obviate Pascal's Wager.