ialdabaoth comments on Tell Culture - Less Wrong

109 Post author: BrienneYudkowsky 18 January 2014 08:13PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (217)

You are viewing a single comment's thread. Show more comments above.

Comment author: ialdabaoth 01 February 2014 04:11:44PM *  1 point [-]

Rewrite your utility function.

Examples:

Original preference - "I just want a car to get to work."

Environmental change: "Here, everyone gets a car for free."

Adjusted preference - "Okay, then what I REALLY want is a faster car than anyone else has."

...

Original preference - "I just want to be able to eat."

Environmental change: "Here, there's enough food to go around forever."

Adjusted preference - "Okay, then what I REALLY want is for me to eat while those guys have to watch, starving."

...

Original preference - "I just want to feel safe."

Environmental change: "Here, you're in a space where everyone is your ally and no one can hurt you."

Adjusted preference - "Okay, then what I REALLY want is for us to all gang up on the people who made us feel unsafe, and make THEM feel persecuted for a change."

Comment author: MugaSofer 01 February 2014 11:26:43PM *  1 point [-]

Original preference - "I just want to be able to eat."

Environmental change: "Here, there's enough food to go around forever."

Adjusted preference - "Okay, then what I REALLY want is for me to eat while those guys have to watch, starving."

You mean "What I REALLY want is to eat better than everybody else", surely. Gourmet food or organic or hand-prepared etc. etc.

Unless this is intended to imply the current famines etc. are the product of a conspiracy and not civilizational inadequacy, in which case yes, that would of course be evidence that my model of human nature is wrong an yours is the correct one, if it's true.

Comment author: TheOtherDave 01 February 2014 07:51:13PM 1 point [-]

The way you formulate this opposes the idea that the "adjusted" preference was actually the preference all along, and the originally stated preference was simply an incorrect description of the system's actual preferences. Is that deliberate, or just an incidental artifact of your phrasing?

Comment author: ialdabaoth 01 February 2014 10:49:59PM *  2 points [-]

It's an artifact of my phrasing. In my experience, people do truly want good things, until those things become universally available - at which point they switch goals to something zero-sum. When they do so, they often phrase it themselves as if they really wanted the zero-sum thing all along, but that's often a part of trying to distance themselves from their lower-status past.

Of course, I'm describing something that I only have personal and anecdotal evidence for; I'd REALLY like to be pointed towards either a legitimate, peer-reviewed description of a cognitive bias that would explain what I'm observing. (And I'd be at least equally happy if it turned out to be my cognitive bias that's causing me to perceive people in this way.)

Comment author: TheOtherDave 01 February 2014 11:49:49PM 2 points [-]

In my experience, people do truly want good things, until those things become universally available - at which point they switch goals to something zero-sum.

What would you expect to experience differently if, instead, people truly want zero-sum things, but they claim to want good things until the universal availability of good things makes that claim untenable?

Comment author: ialdabaoth 02 February 2014 12:03:06AM *  1 point [-]

I'll need some time to think on this. This might just be my tendency to find the most charitable interpretation, even if other interpretations might be more parsimonious.

Comment author: Eugine_Nier 02 February 2014 03:02:19AM 0 points [-]

What do you mean by "trully want"? See the phenomenon Eliezer describes here.

Comment author: TheOtherDave 02 February 2014 06:26:20AM 0 points [-]

I intend the phrase to refer to whatever ialdabaoth meant by it when I quoted them.

Comment author: MugaSofer 01 February 2014 11:28:24PM *  0 points [-]

When they do so, they often phrase it themselves as if they really wanted the zero-sum thing all along, but that's often a part of trying to distance themselves from their lower-status past.

So it was deliberate, and not an artifact of your phrasing. Did you perhaps misread the grandparent?