= 27d567bc2d48d9f40e9ccc20ea370b15)
conchis comments on Exterminating life is rational - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (272)
Here's a possible problem with my analysis:
Suppose Omega or one of its ilk says to you, "Here's a game we can play. I have an infinitely large deck of cards here. Half of them have a star on them, and one-tenth of them have a skull on them. Every time you draw a card with a star, I'll double your utility for the rest of your life. If you draw a card with a skull, I'll kill you."
How many cards do you draw?
I'm pretty sure that someone who believes in many worlds will keep drawing cards until they die. But even if you don't believe in many worlds, I think you do the same thing, unless you are not maximizing expected utility. (Unless chance is quantized so that there is a minimum possible possibility. I don't think that would help much anyway.)
So this whole post may boil down to "maximizing expected utility" not actually being the right thing to do. Also see my earlier, equally unpopular post on why expectation maximization implies average utilitarianism. If you agree that average utilitarianism seems wrong, that's another piece of evidence that maximizing expected utility is wrong.
If I draw cards until I die, my expected utility is positive infinity. Though I will almost surely die and end up with utility 0, it is logically possible that I will never die, and end up with a utility of positive infinity. In this case, 10 + 0(positive infinity) = positive infinity.
The next paragraph requires that you assume our initial utility is 1.
If you want, warp the problem into an isomorphic problem where the probabilities are different and all utilities are finite. (Isn't it cool how you can do that?) In the original problem, there's always a 5/6 chance of utility doubling and a 1/6 chance of it going to 1/2. (Being dead isn't THAT bad, I guess.) Let's say that where your utility function was U(w), it is now f(U(w)), where f(x) = 1 - 1/(2 + log_2 x). In this case, the utilities 1/2, 1, 2, 4, 8, 16, . . . become 0, 1/2, 2/3, 3/4, 4/5, 5/6, . . . . So, your initial utility is 1/2, and Omega will either lower your utility to 0 or raise it by applying the function U' = U/(U + 1). Your expected utility after drawing once was previously U' = 5/3U + 1/2; it's now... okay, my math-stamina has run out. But if you calculate expected utility, and then calculate the probability that results in that expected utility, I'm betting that you'll end up with a 1/2 probability of *ever dying.
(The above paragraph surrounding a nut: any universe can be interpreted as one where the probabilities are different and the utility function has been changed to match... often, probably.)