= 27d567bc2d48d9f40e9ccc20ea370b15)
elspood comments on Pinpointing Utility - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (154)
I've been very entertained by this framing of the problem - very fun to read!
I find it strange that you claim the date with Satan is clearly the best option, but almost in the same breath say that the utility of whaling in the lake of fire is only 0.1% worse. It sounds like your definition of clarity is a little bit different from mine.
On the Satan date, souls are tortured, steered toward destruction, and tossed in a lake of fire. You are indifferent to those outcomes because they would have happened anyway (we can grant this a premise of the scenario). But I very much doubt you are indifferent to your role in those outcomes. I assume that you negatively value having participated in torture, damnation, and watching others suffer, but it's not immediately clear if you had already done those things on the previous 78044 days.
Are you taking into account duration neglect? If so, is the pain of rape only slightly worse than burning in fire?
This probably sounds nitpicky; the point I'm trying to make is that computing utilities using the human brain has all kinds of strange artifacts that you probably can't gloss over by saying "first calculate the utility of all outcomes as a number then compare all your numbers on relative scale". We're just not built to compute naked utilities without reference anchors, and there does not appear to be a single reference anchor to which all outcomes can be compared.
Your system seems straightforward when only 2 or 3 options are in play, but how do you compare even 10 options? 100? 1000? In the process you probably do uncover examples of your preferences that will cause you to realize you are not VNM-compliant, but what rule system do you replace it with? Or is VNM correct and the procedure is to resolve the conflict with your own broken utility function somehow?
TL;DR: I think axiom #1 (utility can be represented as a single real number) is false for human hardware, especially when paired with #5.
That was one of the major points. Do not play with naked utilities. For any decision, find the 0 anchor and the 1 anchor, and rank other stuff relative to them.
Yep, you are not VNM compliant, or the whole excercise would be worthless. The philosophy involved in actually making your preferences consistent is hard of course. I swept that part under the rug.
I understood your major point about the radioactivity of the single real number for each utility, but I got confused by what you intended the process to look like with your hell example. I think you need to be a little more explicit about your algorithm when you say "find the 0 anchor and the 1 anchor". I defaulted to a generic idea of moral intuition about best and worst, then only made it as far as thinking it required naked utilities to find the anchors in the first place. Is your process something like: "compare each option against the next until you find the worst and best?"
It is becoming clear from this and other comments that you consider at least the transitivity property of VNM to be axiomatic. Without it, you couldn't find what is your best option if the only operation you're allowed to do is compare one option against another. If VNM is required, it seems sort of hard to throw it out after the fact if it causes too much trouble.
What is the point of ranking other stuff relative to the 0 and 1 anchor if you already know the 1 anchor is your optimal choice? Am I misunderstanding the meaning of the 0 and 1 anchor, and it's possible to go less than 0 or greater than 1?
Yes, approximately.
I consider all the axioms of VNM to be totally reasonable. I don't think the human decision system follows the VNM axioms. Hence the project of defining and switching to this VNM thing; it's not what we already use, but we think it should be.
VNM is required to use VNM, but if you encounter a circular preference and decide you value running in circles more than the benefits of VNM, then you throw out VNM. You can't throw it out from the inside, only decide whether it's right from outside.
Expectation. VNM isn't really useful without uncertainty. Without uncertainty, transitive preferences are enough.
If being a whale has utility 1, and getting nothing has utility 0, and getting a sandwich has utility 1/500, but the whale-deal only has a probability of 1/400 with nothing otherwise, then I don't know until I do expectation that the 1/400 EU from the whale is better than the 1/500 EU from the sandwich.
I think I have updated slightly in the direction of requiring my utility function to conform to VNM and away from being inclined to throw it out if my preferences aren't consistent. This is probably mostly due to smart people being asked to give an example of a circular preference and my not finding any answer compelling.
I think I see the point you're trying to make, which is that we want to have a normalized scale of utility to apply probability to. This directly contradicts the prohibition against "looking at the sign or magnitude". You are comparing 1/400 EU and 1/500 EU using their magnitudes, and jumping headfirst into the radiation. Am I missing something?
You are allowed to compare. Comparison is one of the defined operations. Comparison is how you decide which is best.
I'm uneasy with this "normalized". Can you unpack what you mean here?
What I mean by "normalized" is that you're compressing the utility values into the range between 0 and 1. I am not aware of another definition that would apply here.
Your rule says you're allowed to compare, but your other rule says you're not allowed to compare by magnitude. You were serious enough about this second rule to equate it with radiation death.
You can't apply probabilities to utilities and be left with anything meaningful unless you're allowed to compare by magnitude. This is a fatal contradiction in your thesis. Using your own example, you assign a value of 1 to whaling and 1/500 to the sandwich. If you're not allowed to compare the two using their magnitude, then you can't compare the utility of 1/400 chance of the whale day with the sandwich, because you're not allowed to think about how much better it is to be a whale.
There's something missing here, which is that "1/400 chance of a whale day" means "1/400 chance of whale + 399/400 chance of normal day". To calculate the value of "1/400 chance of a whale day" you need to assign a utility for both a whale day and a normal day. Then you can compare the resulting expectation of utility to the utility of a sandwhich = 1/500 (by which we mean a sandwich day, I guess?), no sweat.
The absolute magnitudes of the utilities don't make any difference. If you add N to all utility values, that just adds N to both sides of the comparison. (And you're not allowed to compare utilities to magic numbers like 0, since that would be numerology.)
I notice we're not understanding each other, but I don't know why. Let's step back a bit. What problem is "radiation poisoning for looking at magnitude of utility" supposed to be solving?
We're not talking about adding N to both sides of a comparison. We're talking about taking a relation where we are only allowed to know that A < B, multiplying B by some probability factor, and then trying to make some judgment about the new relationship between A and xB. The rule against looking at magnitudes prevents that. So we can't give an answer to the question: "Is the sandwich day better than the expected value of 1/400 chance of a whale day?"
If we're allowed to compare A to xB, then we have to do that before the magnitude rule goes into effect. I don't see how this model is supposed to account for that.
You can't just multiply B by some probability factor. For the situation where you have
p(B) = x, p(C) = 1 - x, your expected utility would bexB + (1-x)C. ButxBby itself is meaningless, or equivalent to the assumption that the utility of the alternative (which has probability1 - x) is the magic number 0. "1/400 chance of a whale day" is meaningless until you define the alternative that happens with probability 399/400.For the purpose of calculating
xB + (1-x)Cyou obviously need to know the actual values, and hence magnitudes of x, B and C. Similarly you need to know the actual values in order to calculate whether A < B or not. "Radiation poisoning for looking at magnitude of utility" really means that you're not allowed to compare utilities to magic numbers like 0 or 1. It means that the only thing you're allowed to do with utility values is a) compare them to each other, and b) obtain expected utilities by multiplying by a probability distribution.Unless you rescale everything so that magic numbers like 0 and 1 are actually utilities of possibilities under consideration.
But that's like cutting corners in the lab; dangerous if you don't know what you are doing, but useful if you do.
If you don't conform to VNM, you don't have a utility function.
I think you mean to refer to your decision algorithms.
No, I mean if my utility function violates transitivity or other axioms of VNM, I more want to fix it than to throw out VNM as being invalid.
then it's not a utility function in the standard sense of the term.
I think what you mean to tell me is: "say 'my preferences' instead of 'my utility function'". I acknowledge that I was incorrectly using these interchangeably.
I do think it was clear what I meant when I called it "my" function and talked about it not conforming to VNM rules, so this response felt tautological to me.