Stuart_Armstrong comments on Extreme risks: when not to use expected utility - Less Wrong

4 Post author: Stuart_Armstrong 23 October 2009 02:40PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (36)

You are viewing a single comment's thread. Show more comments above.

Comment author: Stuart_Armstrong 24 October 2009 08:19:35AM 1 point [-]

Interesting thoughts on the mugger. But you still need a theory able to deal with it, not just an understanding of the problems.

For the second part, you can get a good decision theory for the "Name any rational number less than ten, and you get that much utility," by giving you a certain fraction of negutility for each digit of your definition; there comes a time when the time wasted adding extra '9's dwarfs the gain in utility. See Tolstoy's story How Much Land Does a Man Need for a traditional literary take on this problem.

The "Name any rational number, and you get that much utility" problem is more tricky, and would be a version of the "it is rational to spend infinity in hell" problem. Basically if your action (staying in hell; or specifying your utility) give you more ultimate utility than you lose by doing so, you will spend eternity doing your utility-losing action, and never cash in on your gained utility.