Andreas_Giger comments on Pinpointing Utility - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (154)
Yes, I am aware of that. The biggest trouble, as you have elaborately explained in your post, is that people think they can perform mathematical operations in VNM-utility-space to calculate utilities they have not explicitly defined in their system of ethics. I believe Eliezer has fallen into this trap, the sequences are full of that kind of thinking (e.g. torture vs dust specks) and while I realize it's not supposed to be taken literally, "shut up and multiply" is symptomatic.
Another problem is that you can only use VNM when talking about complete world states. A day where you get a tasty sandwich might be better than a normal day, or it might not be, depending on the world state. If you know there's a wizard who'll give you immortality for $1, you'll chose $1 over any probability<1 of $2, and if the wizard wants $2, the opposite applies.
VNM isn't bad, it's just far, far, far too limited. It's somewhat useful when probabilities are involved, but otherwise it's literally just the concept of well-ordering your options by preferability.
Turns out this is not actually true: 1 day is 1, 2 days is 1.5, 3 days is 1.75, etc, immortality is 2, and then you can add quality. Not very surprising in fact, considering immortality is effectively infinity and |ℕ| < |ℝ|. Still, I'm pretty sure the set of all possible world states is of higher cardinality than ℝ, so...
(Also it's a good illustration why simply assigning utility to 1 day of life and then scaling up is not a bright idea.)
You can talk about probability distributions over world-states as well. When I say "tasty sandwich day minus normal day" I mean to refer to the expected marginal utility of the sandwich, including the possibilities with wizards and stuff. This simplifies things a bit, but goes to hell as soon as you include probability updating, or actually have to find that value.