g comments on Pascal's Mugging: Tiny Probabilities of Vast Utilities - Less Wrong

39 Post author: Eliezer_Yudkowsky 19 October 2007 11:37PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (334)

Sort By: Old

You are viewing a single comment's thread.

Comment author: g 21 October 2007 01:37:00PM 0 points [-]

Stephen, no problem. Incidentally, I share your doubt about the optimality of optimizing *expected* utility (though I wonder whether there might be a theorem that says anything coherent can be squeezed into that form).

CC, indeed there are many infinities (not merely infinitely many, not merely more than we can imagine, but more than we can describe), but so what? Any sort of infinite utility, coupled with a nonzero finite probability, leads to the sort of difficulty being contemplated here. Higher infinities neither help with this nor make it worse, so far as I can see. (I suppose it's worth considering that it might conceivably make sense for an agent's utilities to live in some structure "richer" than the usual real numbers, like Conway's surreal numbers, where there are infinities and infinitesimals aplenty. But I think there are technical difficulties with this sort of scheme; for instance, doing calculus over the surreals is problematic. And of course we actually only have finite brains, so whatever utilities we have are presumably representable in finite terms even if they feature incommensurabilities of the sort that might be modelled in terms of something like the surreal numbers. But all this is a separate issue.)