I could have got that child sponsored. I could have kept my job, and Mary could have stopped crying that evening. She’d have thanked me for coming by, and after I left she would have cuddled on the couch with her new Sponsor Child, tears drying as she found hope in the world.
But I didn’t do it. Instead I apologized for interrupting her grief, and left.
Because I am not a Meat Fucker.
Translation: you fell prey to a different Dark meme, one that linked your behavior to a perception of status, which you then pursued at the expense of real utility for at least two real people besides yourself.
In other words, your vaunted "honesty" actually equals nothing more than selfish egoism: you prefer to pride yourself on it more than you prefer actually helping people.
That is, the rhetoric around honesty you're using is nothing more than the creed of the Confessor ("the ultimate sin is the exercise of command"), which is equally countered and mirrored by that of the Kiritsugu (who would see you as flawed for being "one who refuses to help").
It's not obvious that either of these is a truly correct stance, vs. just being different from each other. Neither post...
This is a revealing story about the double-binds in influence and persuasion. To me, your hypothetical form of influence probably would have been "Dark Arts," but not just for the reasons you describe.
Actually, I'd prefer to taboo the term "Dark Arts" for a while, because I feel it's poisoned the well on the subject of influence and persuasion (like the word "manipulation"). The problem with the term "Dark Arts" is that it conflates both ethical and unethical forms of influence, and creates an "Ugh Field" around the subject of influence in general. Let's talk about how influence and persuasion work, and then later we will decide what's ethical and what's unethical.
The irony of a post criticizing the Dark Arts is that this post is full of rhetoric and persuasion itself, whether intentionally or unintentionally. I'd like to examine some of the language you use in this post ("Dark Arts", "Jedi Mind Tricks", "hacking", "Murder Pill"). In general, I consider these forms of emotive language about influence and persuasion to inhibit understanding these topics.
...When you take advantage of the Dark A
There was a step somewhere in here I missed. You're clear that these tactics violate your morals, and you specifically do not allege that learning about them will help us avoid them (and nor will anything else). What's your purpose in teaching them?
Clarification: if you walk into the leather shop, with $300 burning a hole in your pocket, and see a nice jacket - and the guy behind the counter knows what he's doing (and gives enough of a shit) you will walk out of their, happy with your new jacket.
Saying there's no defense whatsoever was a bit of an exagerration; it's true that some people are more resistant than others (though not infinitely resistant), but even if you're weak you can always avoid the situations in the first place.
Having a blanket policy of "I don't do anything at the door" is highly effective.
It's sort of like a bullet; you can't resist it, but you can avoid it [assuming for the sake of analogy you ignore kevlar].
You know what? I will write a piece on defence, once I'm done with the other pieces in the series. Hopefully what I was trying to convey will be clearer at that point.
As for my purpose - 1) it's fun; this is stuff I've thought a lot about, and the community here is smart enough to understand it, 2) I get LW points and ego validation for writing it, and 3) the information's already out there, I figure it might as well be in the hands of the Good Guys.
This is a followup to the thread you linked, but I'll mention it here because it may be relevant to the current conversation.
I spent a little while with TheValliant (offsite) trying to pin down a definition of "manipulation"--specifically, to divide it from acceptable forms of nonverbal communication. The facets which came to mind immediately were "trying to get someone to do something they otherwise wouldn't have," which is clearly too broad, and "displaying exaggerated emotion," which is sometimes valid (when you're trying to communicate an emotion and your natural expression is too subtle). We got from there to "trying to make someone feel an emotion they otherwise wouldn't have," which still isn't right, because it covers gestures of affection.
What we eventually settled on was this: "Using emotion to bypass someone's normal decision-making process." That is, creating emotions in someone else for the purpose of getting them to do something. This phrasing also makes it pretty clear why we find it abhorrent: it's opening a back door into someone else's brain, and about as invasive as that makes it sound.
The reason I felt a need to pin it down is that TheValliant, like you, has a policy of not tolerating it under any circumstances, and it seemed to me that that required understanding what it was. So now I'm curious--does the above definition match the thing you hate?
To illustrate another example where the "avoid buyer's remorse" principle is overbroad (which may or may not be the principle you are advocating), let's talk about cookies.
You're having a dinner party, and everyone is stuffed. You bring out some freshly baked cookies for dessert. Guest: "Oh no, I'm stuffed." You: "They are warm and soft!" Guest: "Well, that does sound good, I'll have just one." Guest eats a cookie. Later, Guest looks a bit queasy and is obviously regretting eating so much.
This is another case that falls afoul of the "avoid creating buyer's remorse" principle, but doesn't deserve such a negative term as "manipulation."
The fact is that a lot of people enjoy taking actions that they may later regret. It's not a moral requirement to protect people from themselves. As long as they are making the decision with free will (or the closest thing to it that humans have), and informed consent exists, then it's valid for people to take responsibility for the risks of their behavior. We should assume that people can assess their best interests, unless we have reasons to believe otherwise (for instance, if you know that ...
Influencing someone to take an action that you know she will regret afterwards is manipulative.
What if you also "manipulate" so that it will also not be regretted afterwards?
In marketing at least, there is the concept of customer retention through anticipating and countering buyers' remorse... which mostly, AFAICT, consists of providing a customer with arguments to use to explain to co-workers, friends, relatives, spouses, or whomever why their purchase decision was a good one. This strongly implies that at least in the purchasing arena, the main reason people come to have buyers' remorse (besides crappy products) is that the purchase makes them look bad in the eyes of others.
Hence the marketing adage that the primary function of facts and logic in a sales pitch is to provide the customer with a rationale that lets them prove to themselves and others that they made the right decision... but only after they've been convinced to make that decision based on emotion.
I believe there's some discussion in the PUA field of similar "buyer's remorse" issues and providing the same sort of supporting rationale, except that such rationales are more to allow those women...
Yay, ethics of influence and seduction, one of my favorite subjects. I'm still figuring out my thoughts, so I would appreciate it if people tell me if I'm making incorrect assumptions.
First, I'll state that I prefer the terms "unethical influence" and "unethical influence" to "manipulation" and "non-manipulation," because people use "manipulation" to mean too many different things.
Second, in ethical discussions, we should distinguish between things that would be a good things to do that aren't morally required, and things that are morally required (Kant called these "imperfect duties" vs "perfect duties"). Also, we should distinguish between ethical courses of action, and empathetic courses of action. There are lots of cases where there's an action that's a good and empathetic thing to do, but actually requiring it as a perfect duty would screw everything up.
Physical coercion forces you to do something that you don't want to do, don't enjoy while you're doing it, and regret doing afterwards.
There is clearly a perfect duty against physical coercion, and that duty applies even if someone happens to enjoy what y...
In the case of sex, I propose a different demarcation criterion between ethical and unethical social influence than yours. The dimension I'm most concerned with is not remorse or lack thereof after the fact, but rather the reasons for consent at the moment of consent. In the past, I proposed the following definition for ethical seduction on a pickup blog:
Taking intentional action to create a context that raises someone’s chances of wanting to be sexual with you of their own free will out of anticipation of intrinsic enjoyment of the experience.
The "anticipation of intrinsic enjoyment of the experience" criterion is important, because it gets rid of cases where people consent to sex out of feelings of obligation, pity, merely because the other person wants it, or because they had trouble putting on the brakes. This notion is similar to the notion of "enthusiastic consent," but without the confusing connotations that "enthusiastic" may hold.
I'm not sure whether this is a perfect or merely an imperfect duty. The argument for this principle being a perfect duty is that if someone has sex with you for reasons other than anticipating inherent enjoyment o...
No matter how hard I try and kick it, there it is: Honesty.
Where in particular do you perceive the lie in the situation you described? Would her donation not have sponsored the child? Would the donation not have made her feel better? Or is that you do not believe she should feel better for sponsoring a child; that in your mind it would be dishonest for her to displace her current grief with this balm?
Of course if the donation would not have sponsored the child, then it would be dishonest to claim that it would.
I can also imagine the donation not actually making her feel better. It would be possible to simply overwhelm her (e.g., intimidate them with a whirlwind of emotional stimuli) into doing something she didn't want to do. This is in the direction of being mean/bullyish. People can push boundaries and this is being pushy...but we don't often describe it explicitly, perhaps because it requires such a high emotional intelligence to identify and name it.
Finally, the third case is that you don't believe sponsoring children is a feel-good thing. Which would be a strong indication that that wasn't the right job for you, but wouldn't mean that sponsoring a child wasn't the right thing for her.
The bit about Sheila from accounting prompts a fun question for those LWers who subscribe to specific theories about gender relationships (PUA, feminism or whatever): within a couple, do you predict the events "husband cheats" and "wife cheats" to be independent, correlated or anticorrelated? To me it sounds like it could just as easily go one way or the other, and each answer has many rationalizations that immediately spring to mind. (Won't spell them out to avoid spoiling your fun.)
Correlated, because time spent apart causes infidelity for both. Also, cultural norms make a difference, and both are exposed to the same culture.
Anti-correlated, because differences in libido causes a higher probability of infidelity on the part of whoever wants more sex, and a lower probability on the part of whoever wants less.
I'm not sure which of these I find more convincing.
Anticorrelated: Sheila's attraction to Hubby implies Hubby has qualities which make him appealing to women, which makes Wifey less likely to cheat.
Correlated because cheating by one partner implies less mate-guarding by the other. A partner who doesn't bother to mate guard is less concerned about losing that partner, and thus more likely to cheat.
Also both partners are more likely to cheat if the passion has died down in the relationship, or if they're in the midst of a conflict.
Interesting that given the choice between leaving the woman to cry and an obvious route to make her feel better, you felt that the correct, moral thing to do was to leave her miserable.
There would certainly seem to have been a third option: comfort her without selling her anything...
FWIW, I second Relsqui's curiosity about why you think this sequence is a good idea.
I really can't help but correct your math: 12! would be if you could choose how to order the 12 toppings, but that's not what you're doing - you're choosing a subset of them - that's 2^12 options. Don't worry, you were only off by five orders of magnitude.
Sorry about that.
Would we expect those who consciously learned the Dark Arts to be more or less responsible with them (that is, using them ethically) than those who are naturally skilled?
Out of interest, how many of the people who listened to your pitch subsequently gave donations?
In Ian M Banks' novel 'Excession', the term meatfucker is used as an derogatory term for AIs that violate a cultural taboo again reading human minds.
This is the only comment I've ever seen here where I've vacillating between voting up or down, rather than just leaving it alone. I can recognize that it's either really good or really bad, but I can't decide which one, so congratulations, I guess.
Dark Arts done right... people need to get used to being manipulated a little to at least get the feel of it happening, or to learn a bit of it if that doesn't work. Experience as a net troll does wonders to keep me from getting riled up. But honesty does not have the least collateral damage trolling does.
1) Any group of mostly men dedicated to any sort of hardware--guns, cars, for all I know chainsaws. For some reason men don't just form attachments to the specific tool they actually have (Cars, Guns, Hammers, scapels are all tools), they have to defend the BRAND and/or model (Mopar, 1911, Macintosh are the most vociferous defenders).
2) Any group that thinks unreasonably highly of itself. Atheists, Mensa, Vegetarians/Vegans, Hippies, Mac Users.
3) Any group that feels under threat, either from the march of technology, a conspiracy, or just reality damaging their world view like Socialists, Vegetarians/Vegans, 1911 owners and Mac Users.
I salute your ability to troll all of these groups in a post about what kind of groups are easy to troll. I almost started to argue on some of these points before I saw your game.
When you manipulate a person's distressed emotional state to convince them into an action, even when that action is a positive one (sponsoring a child) there is always the potential that the person who was manipulated will later realize the manipulation.
Essentially - If he had convinced the crying woman to sponsor the child she very well have felt better on an ongoing and permanent basis (a positive result for everyone involved), however there is also the potential that at some later date she would have realized the manipulation.
My understanding of being ...
I'm pretty sure I don't like being Sold. I generally have a pretty good idea of what I do or don't want, and (to the extent that I recognize it), I dislike being manipulated.
You might be right about people in general, but it seems like a hard thing to check on whether people like being Sold as distinct from whether it's possible to Sell them.
So, is the story real, and why did you include the spider (I reckon that is not real, too perfect)?
I notice that I am confused. Manipulation may have occurred if someone's free will has been overridden. There was a nice post about the compatibility between free will and determinism, which I agree with, but I find the issue of manipulation shows I still have a fuzzy idea about what free will means.
The work on Friendly AI gives me a fairly good idea about how it's possible to avoid being manipulative to a person, but the answer - following their extrapolated volition - is much too high a standard. What if you want to be neutral rather than selflessly and altrusticly benevolent towards them?
Still, their extrapolated volition seems to be relevant.
Fascinating post!
While the Dark Arts are a Power, it’s how you use them that matters, like any other tool. I’m running mind-games on people, but I usually won’t; I’m also good at fighting, but I don’t assault people for no reason. I find both concepts repulsive.
I agree with your reasoning here. Along these lines, I'm curious about something:
Do you think learning this stuff made you better or worse as a rationalist?
I’d like to tell you all a story.
Once upon a time I was working for a charity – a major charity – going door-to-door to raise money while pretending it wasn’t sales.
This story happened on my last day working there. I didn’t know that at the time; I wouldn’t find out until the following morning when my boss called me up to fire me, but I knew it was coming. For weeks I’d been fed up with the job, milking it for the last few dollars I could pull out, hating every minute of it but needing the money. The Sudden Career Readjustment would come as a relief.
So on that day, my last day, I was moving slowly. I knocked on one particular door and there was no response. I had little desire to walk to the next one, however, and there was an interesting spider who’d built its web below the doorbell. I tapped its belly with the tip of my pen, and it reacted with aggression – trying to envenom and ensnare the tip of my ballpoint. I must have been playing with it for a good minute or so when the door suddenly opened.
A distraught woman stood before me. After a brief period of Relating I launched into my pitch.“So you’re probably wondering why there’s a bald weirdo at your door? Actually I’m just coming around with Major Charity1 on an emergency campaign. You’ve heard of us, right? Brilliant! So obviously you’ve thought of getting involved, right? That’s awesome! You see, the reason I’m coming around is for these guys – some of our emergency cases...”2
I handed her the pictures of the Developing World Children (yeah, it was one of those charities). She took them, a wistful look on her face.
“Oh God, don’t show me these. I’m such a Rescuer.”
“Rescuer? Do you have a Rescue Dog?” [Where I’m from, abused animals brought into a new home are called ‘Rescue Dogs’.]
“No, I...”
“You mean your personality? You care about people, don’t you?”
She nodded slowly. Her face began to crumble.
“I’m sorry – I can’t look at these children,” she handed back the photographs, “Not right now. I’ve been crying all day and I just can’t deal with those emotions...”
I took back the children, a look of honest sympathy on my face. The Demon Wheel began spinning. I could see that she was on the verge of crying again. My gut told me that her father had recently died, but the actual cause didn’t matter. I could discover that information. The upcoming dialogue played itself out in my mind...
"Oh jeez, what happened? Oh my god, seriously..?” Head tilted as an Alpha confidant enough for Beta behaviour, looking down and shaking, “I’m lucky enough to have never been through that. Were the two of you close?” As she talks I nod, prompting her until she breaks out in tears. I put down my binder and step into her house, embracing her as she cries on my shoulder.
She sniffles.
“I’m sorry... sorry to do this to you.”
“No, don’t be. Listen... Mary, is it? What you’re going through is normal. It’s nothing to be ashamed of…” Cue personal anecdote, then pause for a beat. “Listen, about the Major Charity thing; this is something you’ve always wanted to do, isn’t it? Yeah, I can tell. You’re a caring person, after all. I tell you what: we’ll get you set up with this little boy – he’s from Ecuador, and we’re trying to get him eating a healthy diet. We’re going to make you his super hero today. And then you’ll know – Mary, you’ll know that even at your darkest moment, you still have the strength in you to save a life.
“And you know what else?” I reach out to touch her arm, “Tonight you’re going to sleep like a baby knowing that you did this. So you go and get your Credit Card and I’ll start filling out the form.”
* * *
I could have done it. I could have got that child sponsored. I could have kept my job, and Mary could have stopped crying that evening. She’d have thanked me for coming by, and after I left she would have cuddled on the couch with her new Sponsor Child, tears drying as she found hope in the world.
But I didn’t do it. Instead I apologized for interrupting her grief, and left.
Because I am not a Meat Fucker.
* * *
All my life, I’ve had this bad habit. No matter how hard I try and kick it, there it is: Honesty. I can’t tell you how many times it’s dug me into a hole. As far as concepts go, it’s about as foolish and utopian as Truth and Justice, and I know that, but I just can’t seem to let it go. That’s a large part of the reason I left Mary alone to her tears – backed off, rather than digging into her psyche to recalibrate a few clusters of neuron.
The other half is my status as a card-carrying (union-dues-paid-in-full) Anarchist. The way I look at things, the only time you can justify using the Jedi Mind Trick on somebody is when your ethics would stand clean with murdering them as well.
Sending Storm Troopers on a Wild Droid Chase is one thing; scamming Waddo out of a distributor cap for your CGI Space Plane is another.
When you take advantage of the Dark Arts, you’re not simply tricking people into giving you what you want; you’re making them want to give it to you. You’re hacking into their brain and inserting a Murder Pill; afterwards they will literally thank you for doing so (the only sponsor I ever met who wasn’t glad that I’d come by was the lady whose 6 year old daughter I primed into wanting it). In ninety percent of the situations where the Dark Arts are useful or possible, you can’t do it out of spite; when you realign someone’s desires to match your own they want to do what you want them to do.
And yet there’s no clear distinction between using these skills and regular social interaction. Manipulation works best when you’re sincere about it. Ethically speaking it’s a grey, wavy line.
The thing is, we all like to be Sold, Led, Dominated; if I walk into Subway, and I ask the kid at the counter to give me his Best Submarine Sandwich, I want him to tell me what I want, and make me love it after it’s paid for. The last thing he should do is say that “They’re all good!” and make me regret the [(5 breads)x(16 meats)x(212 Toppings)-1] subs that I didn’t get.3 Retail is the Dark Arts Done Right (usually). The Sales Lady figures out what I want, uses her expertise to find the best fit, and then kills the cognitive dissonance that could ruin my enjoyment of the product; “You really pull off that colour. Seriously, that jacket looks great on you – you see how these lines naturally compliment your shoulders? Of course you can!”
Sexual dynamics are similar; if somebody’s drinking in public at 2 in the morning it’s because they’re on the market. Let’s say a ‘faithful wife’ goes to the club one weekend while her husband is out of town, and she has a few drinks with a bunch of college boys she just met. One of them happens to be a PUA. When it comes to things like date rape drugs, or taking advantage of a person who’s sloppy-drunk there is a clear line in the sand. But in this hypothetical the woman’s relatively sober. It’s just that the young rake is so damned charming!
Meanwhile her husband’s having a few pints at the hotel bar with Sheila from accounting, and she just keeps making eyes at him…
Neither Sheila nor the PUA is responsible for the ensuing infidelity. If the husband and wife didn’t want it in the first place, they would have never availed themselves to the temptation. If, on the other hand, you meet somebody at a Neighbourhood Watch meeting, and spend the next three months seducing them… that’s when you’ve got to start questioning your ethics. Anybody is going to be vulnerable at some time or another.
While the Dark Arts are a Power, it’s how you use them that matters, like any other tool. I’m running mind-games on people, but I usually won’t; I’m also good at fighting, but I don’t assault people for no reason. I find both concepts repulsive.
That’s the end of my moralizing on the matter. The upcoming series is going to be purely descriptive in nature, exploring different strategies for manipulating others. I’ll provide tactical examples showing how these strategies can be put into play, but for the most part each battlefield is unique; these are broader methods that apply across the board. What you do with these techniques is up to you.
As for defence… I don’t think I’ll have much to say about that. When done properly, the victim doesn’t realize it until it’s already over, and by then it doesn’t matter. You’re aware that the AI manipulated you into opening the box, but you’re going to open it anyways because that’s your new utility function. It’s like a game of Roshambo, or when you’re thinking about joining Facebook: the only way to win is not to play.
Endnotes
1. Major Charity’s methods of acquiring funding don’t have any bearing on whether or not it’s an effective charity. Whether or not the money going overseas actually makes a difference is a question I cannot answer.
2. The repetition here is intentional. I was trying to prime key concepts.
3. My theory as to what is going on with these sub places and their myriad of options: the target is not a new customers, those people are going to be intimidated by all the choices, and the restaurants know that. Rather, it is to provide ‘fresh’ options so that their current customers don’t get bored and go elsewhere.