= 27d567bc2d48d9f40e9ccc20ea370b15)
pjeby comments on Post Your Utility Function - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (273)
My point was that even if you can make a tradeoff, you're likely to have at least some disutility for Omega making that tradeoff for you, rather than letting you make the tradeoff yourself.
My own personal observation, though, is that people don't usually make good tradeoffs by weighing and combining the utility and disutility for each of their options; they're happier (and their lives are more generally functional), when they work to maximize utility and then satisfice disutility, in that order.
Our hardware doesn't do well at cross-comparison, but it can handle, "Which of these do I like best?", followed by "What am I willing to trade off to get what I like best?" (It can also handle the reverse, but that road leads to a dysfunctional and ever-shrinking "comfort zone".)
I assume that this is because the two affect systems were intended for approach and avoidance of predators, prey, and mates, rather than making rational tradeoffs between a wide array of future options.
Each system is quite capable of ranking threats or opportunities within its own value system, but there doesn't seem to be a register or readout in the system that can hold a "pleasure minus pain" value. What appears to happen instead is that the conscious mind can decide to switch off the negative input, if there's an inner consensus that the worst-case downside is stlil manageable.
This mechanism, however, appears to only operate on one goal at at time; it doesn't seem to work to try to cram all your options into it at once.
In the aggregate, these mechanisms would be really difficult to model, since the disutility/worst-case scenario check often depends on the examination of more than one possible future and contemplating possible mitigations, risks, etc.
I guess what I'm trying to say is that not only is goal+context important, there's also a cross-time or over-time input component as well, and that I don't really see anything that allows a person's preferences to be absolute, because the "tradeoff" part is something that can happen consciously or unconsciously, and is very sensitive to the steps undertaken to make the tradeoff. But despite this sensitivity, the emotional affect of having made the choice is the same - we defend it, because we own it.
In contrast, a rational weighing of ratios and scores can easily produce a different felt-sensation about the decision: one of not really having decided at all!
If a person "decides" based only on the objective/numerical criteria (even if this includes scoring and weighing his or her emotional responses!), this ownership/territory mechanism does not kick in, with resulting negative consequences for that person's persistence and commitment.
For example, if you "decide" to go on a diet because you're 20 pounds overweight, you may stop eating healthily (or at least cease to do so consistently) as you approach your desired weight.
Now, that's not to say you can't weigh all the objective information, and then make a decision that's not conditional upon those facts, or is conditional upon those facts only at the point of time you originally received them. I'm just saying that if you just weigh up the facts and "let the facts decide", you are just begging for an akrasia problem.
This is why, btw, effective decision makers and successful people tend to talk about "listening to all the input first, and then making their own decision".
It's because they need to make it theirs -- and there's no way for the math to do that, because the calculation has to run on the right brain hardware first. And the mathematical part of your brain ain't it.
OK, there's a lot of food for thought in there, and I can't possibly hope to clarify everything I'd ideally like to, but what I think you're saying is:
(FWIW, I was only claiming 1.) I'm fairly sympathetic to 2(a), although I would have thought we could get better at it with the right training. I can see how 2(b) could be a problem, but I guess I'm not really sure (i) that akrasia is always an issue, and (ii) why (assuming we could overcome 2(a)) we couldn't decide mathematically, and then figure out how to "own" the decision afterwards. (This seems to have worked for me, at least; and stopping to do the math has at sometimes stopped me "owning" the wrong decision, which can be worse than half-heartedly following through on the right one.)
P.S.
I didn't think anyone was suggesting Omega should make the trade-off. I certainly wasn't.
To own it, you'd need to not mathematically decide; the math could only ever be a factor in your decision. There's an enormous gap between, "the math says do this, so I guess I'll do that", and "after considering the math, I have decided to do this." The felt-experience of those two things is very different, and it's not merely an issue of using different words.
Regarding getting better at making decisions off of mathematics, I think perhaps you miss my point. For humans, the process by which decision-making is done, has consequences for how it's implemented, and for the person's experience and satisfaction regarding the decision itself. See more below...
I'd like to see an actual, non-contrived example of that. Mostly, my experience is that people are generally better off with a 50% plan executed 100% than a 100% plan executed 50%. It's a bit of a cliche -- one that I also used to be skeptical/cynical about -- but it's a cliche because it's true. (Note also that in the absence of catastrophic failure, the primary downside of a bad plan is that you learn something, and you still usually make some progress towards your goals.)
It's one of those places where in theory there's no difference between theory and practice, but in practice there is. We just think differently when we're considering something from when we're committed to it -- our brains just highlight different perceptions and memories for our attention, so much so that it seems like all sorts of fortunate coincidences are coming your way.
Our conscious thought process in System 2 is unchanged, but something on the System 1 level operates differently with respect to a decision that's passed through the full process.
I used to be skeptical about this, before I grasped the system 1/system 2 distinction (which I used to call the "you" (S2) vs. "yourself" (S1) distinction). I assumed that I could make a better plan before deciding to do something or taking any action, and refused to believe otherwise. Now I try to plan just enough to get S1 buy-in, and start taking action so I can get feedback sooner.
Sure. I don't think this is inconsistent with what I was suggesting, which was really just that that the math could start the process off.
All of which I agree with; but again, I don't see how this rules out learning to use math better.
Fair enough. The examples I'm thinking of typically involve "owned" decisions that are more accurately characterised as 0% plans (i.e. do nothing) or -X% plans (i.e. do things that are actively counterproductive).
P.S. Yes, I'm avoiding the concrete example request. I actually have a few, but they'd take longer to write up than I have time available at the moment, and involve things I'm not sure I'm entirely comfortable sharing.
I already explained: you select options by comparing their positive traits. The devil is in the details, of course, but as you might imagine I do entire training CDs on this stuff. I've also written a few blog articles about this in the past.
I don't understand the question. If you're asking how I'd know whether I made the best possible decision, I wouldn't. Maximizers do very badly at long-term happiness, so I've taught myself to be a satisficer. I assume that the decision to invest something for the long term is better than investing nothing, and that regarding an ethical decision I will know by the consequences and my regrets or lack thereof whether I've done the "right thing"... and I probably won't have to wait very long for that feedback.
One can imagine a person who has committed emotionally to the maxim "shut up and multiply (when at all possible)" and made it an integral part of their identity. For such an individual, the commitment precedes the act of doing the math, and the enormous gap referred to above does not exist.
If such an individual existed, they would still have the same problem of shifting decisions, unless they also included a commitment to not recalculate before a certain point.
Consider, e.g. Newcomb's problem. If you do the calculation before, you should one-box. But doing the calculation at the actual time, means you should two-box.
So, to stick to their commitments, human beings need to precommit to not revisiting the math, which is a big part of my point here.
Your hypothetical committed-to-the-math person is not committed to their "decisions", they are committed to doing what the math says to do. This algorithm will not produce the same results as actual commitment will, when run on human hardware.
To put it more specifically, this person will not get the perceptual benefits of a committed decision for decisions which are not processed through the machinery I described earlier. They will be perceptually tuned to the math, not the situation, for example, and will not have the same level of motivation, due to a lack of personal stake in their decision.
In theory there's no difference between theory and practice, but in practice there is. This is because System 2 is very bad at intuitively predicting System 1's behavior, as we don't have a built-in reflective model of our own decision-making and motivation machinery. Thus, we don't know (and can't tell) how bad our theories are without comparing decision-making strategies across different people.
This is incorrect. You are doing something very wrong if changing the time when you perform a calculation changes the result. That's an important issue in decision theory being reflectively consistent.
That's the major point I'm making: that humans are NOT reflectively consistent without precommitment... and that the precommitment in question must be concretely specified, with the degree of concreteness and specificity required being proportional to the degree of "temptation" involved.
That may usually be the case, but this is not a law. Certain people could conceivably precommit to being reflectively consistent, to follow the results of calculations whenever the calculations are available.
Of course they could. And they would not get as good results from either an experiential or practical perspective as the person who explicitly committed to actual, concrete results, for the reasons previously explained.
The brain makes happen what you decide to have happen, at the level of abstraction you specify. If you decide in the abstract to be a good person, you will only be a good person in the abstract.
In the same way, if you "precommit to reflective consistency", then reflective consistency is all that you will get.
It is more useful to commit to obtaining specific, concrete, desired results, since you will then obtain specific, concrete assistance from your brain for achieving those results, rather than merely abstract, general assistance.
Edit to add: In particular, note that a precommitment to reflective consistency does not rule out the possibility of one's exercising selective attention and rationalization as to which calculations to perform or observe. This sort of "commit to being a certain kind of person" thing tends to produce hypocrisy in practice, when used in the abstract. So much so, in fact, that it seems to be an "intentionally" evolved mechanism for self-deception and hypocrisy. (Which is why I consider it a particularly heinous form of error to try to use it to escape the need for concrete commitments -- the only thing I know of that saves one from hypocrisy!)
I can't understand you.
Newcomb's problem is a bad example to use here, because it depends on which math the person has committed to, e.g., Eliezer claims to have worked out a general analysis that justifies one-boxing...
The personal stake I envision is defending their concept of their own identity. "I will do this because that's the kind of person I am."
Then their perception will be attuned to what kind of person they are, instead of the result. You can't cheat your brain - it tunes in on whatever you've decided your "territory" is, whatever you "own". This is not a generalized abstraction, but a concrete one.
You know how, once you buy a car, you start seeing that model everywhere? That's an example of the principle at work. Notice that it's not that you start noticing cars in general, you notice cars that look like yours. When you "own" a decision, you notice things specifically connected with that particular decision or goal, not "things that match a mathematical model of decision-making". The hardware just isn't built for that.
You also still seem to be ignoring the part where, if your decisions are made solely on the basis of any external data, then your decision is conditional and can change when the circumstances do, which is a bad idea if your real goal or intent is unconditional.
I've already mentioned how a conditional decision based on one's weight leads to stop-and-start dieting, but another good example is when somebody decides to start an exercise program when they're feeling well and happy, without considering what will happen on the days they're running late or feeling depressed. The default response in such cases may be to give up the previous decision, since the conditions it was made under "no longer apply".
What I'm saying is, it doesn't matter what conditions you base a decision on, if it is based solely on conditions, and not on actually going through the emotional decision process to un-conditionalize it, then you don't actually have a commitment to the course of action. You just have a conditional decision to engage in that course, until conditions change.
And the practical difference between a commitment and a conditional decision is huge, when it comes to one's personal and individual goals.
Thank you for this interesting discussion. Although I posed the "emotionally committed to math" case as a specific hypothetical, many of the things you've written in response apply more generally, so I've got a lot more material to incorporate into my understanding of the pjeby model of cognition. (I know that's a misnomer, but since you're my main source for this material, that's how I think of it.) I'm going to have to go over this exchange more thoroughly after I get some sleep.
Of course, there are presumably situations where one's decision should change with the conditions. (I do get that there's a trade-off between retaining the ability to change with the right conditions and opening yourself up to changing with the wrong conditions though.)
The trade-off optimum is usually in making decisions aimed at producing concrete results, while leaving one's self largely free to determine how to achieve those results. But again, the level of required specificity is determined by the degree of conflict you can expect to arise (temptations and frustrations).