= 27d567bc2d48d9f40e9ccc20ea370b15)
Esar comments on The Power of Reinforcement - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (467)
But treating human beings, especially adults, like animals is characteristically unethical. Applying some system of reinforcement where someone has asked you to effectively treat their behavior is innocuous enough, as is of course treating yourself.
But generally manipulating the behavior of other people by means other than convincing them that they should behave in a certain way seems to me to be almost definitional of a dark art. If that's not controversial, then I think this article should be qualified appropriately: never do this to other people without their explicit consent.
It seems to me like the flow is in the reverse direction: many unethical manipulations involve treating adults like animals. But people who skillfully use positive reinforcement are both more pleasant to be around and more effective- which seems like something ethical systems should point you towards, not away from.
That's a fair point: I may have been treating a conditional like a bi-conditional. I think my sense of the matter is this: if a friend told me that he spent a lot of our time together thinking through ways to positively reinforce some of my behaviors, even to my benefit, I would become very suspicious of him. I would feel that I'd been treated as a child or a dog. His behavior would seem to me to be manipulative and dishonest, and I think I would feel this way even if I agreed that the results of his actions were on the whole good and good for me.
Do you think this sort of reaction on my part would be misguided? Or am I on to something?
I agree with you that your autonomy is threatened by the manipulations of others. But threats only sometimes turn into harm- distinguishing between manipulations you agree with and disagree with is a valuable skill.
Indeed, there's a general point that needs to be made about human interaction, and another about status, but first a recommendation: try to view as many of your actions as manipulations as possible. This will help separate out the things that, on reflection, you want to do and the things that, on reflection, you don't want to do. For example:
Emphasis mine. The reaction- of calling his behavior manipulative and dishonest- feels like it punishes manipulation, which you might want to do to protect your autonomy. But it actually punishes honesty, because the trigger was your friend telling you! Now, if your friend wants to change you, they'll need to try to do it subtly. Your reaction has manipulated your friend without his explicit consent- and probably not in the direction you wanted it to.
So, the general point: human social interaction is an incredibly thorny field, in part because there are rarely ways to learn or teach it without externalities. Parents, for example, tell their children to share- not because sharing is an objective moral principle, but because it minimizes conflict. As well, some aspects of human social interaction are zero sum games- in which people who are skilled at interaction will lose if others get better at interaction, and thus discourage discussions that raise general social interaction skills.
The status interpretation: generally, manipulation increases the status of the manipulator and decreases the status of the manipulated. Resistance to manipulation could then be a status-preserving move, and interest in manipulation could be a status-increases move. What articles like this try to do is lower the status effects of manipulation (in both directions)- Luke proudly recounts the time Eliezer manipulated him so that he could better manipulate Eliezer. If being molded like this is seen more positively, then resistance to being molded (by others in the community) will decrease, and the community will work better and be happier. As well, I suspect that people are much more comfortable with manipulations if they know how to do them themselves- if positive reinforcement is a tool used by creepy Others, it's much easier to dislike than if it's the way you got your roommate to finally stop annoying you.
This, with extra emphasis!
I'm confused, not only by the beginning of this comment, but by several others as well.
I thought being a LessWronger meant you no longer thought in terms of free will. That it's a naive theory of human behavior, somewhat like naive physics.
I thought so, anyway. I guess I was wrong? (This comment still up voted for amazing analysis.)
Autonomy and philosophical free will are different things. Philosophical free will is the question "well, if physical laws govern how my body acts, and my brain is a component of my body, then don't physical laws govern what choices I make?", to which the answer is mu. One does not need volition on the level of atoms to have volition on the level of people- and volition on the level of people is autonomy.
(You will note that LW is very interested in techniques to increase one's will, take more control over one's goals, and so on. Those would be senseless goals for a fatalist.)
Thanks for clarifying that. I should note that I am very interested in techniques for self-improvement, too. I am currently learning how to read. (Apparently, I never knew :( ) And also get everything organized, GTD-style. (It seems a far less daunting prospect now than when I first heard of the idea, because I'm pseudo-minimalist.)
I still am surprised at the average LWers reaction here. Probably because it's not clear to me the nature of 'volition on the level of people'. Not something to expect you to answer, clarifying the distinction was helpful enough.
I think it's misguided personally. You're already being manipulated this way by your environment whether or not you realize it.
Well, I'm claiming that this kind of manipulation is often, even characteristically, unethical. Since my environment is not capable of being ethical or unethical (that would be a category mistake, I think) then that's not relevant to my claim.
I was referring though to the case of your friend using reinforcement to alter your behavior in a way that would benefit you. I just have a hard time seeing someone trying to help you as an unethical behavior.
It does depend on whose definition of 'help' they're using.
Good point. Do you think it would be ethical if they were helping to fulfill your preferences?
Usually, yes, though there are several qualifications and corner cases.
Agreed, there probably are.
That's fair. I should tone down my point and say that doing this sort of thing is disrespectful, not evil or anything. Its the sort of thing parents and teachers do with kids. With your peers, unsolicited reinforcement training is seen as disrespectful because it stands in leau of just explaing to the person what you think they should be doing.
In my experience, telling other people how I think they should behave is also often seen as disrespectful.
Often it is, we agree. But it's the 'telling' there that's the problem. A respectful way to modify someone's behavior is to convince them to do something different (which may mean convincing them to subject themselves to positive reinforcement training). The difference is often whether we appeal to someone's rationality, or take a run at their emotions.
I agree that there are respectful ways to convince me to do something different, thereby respectfully modifying my behavior.
Many of those ways involve appealing to my rationality.
Many of those ways involve appealing to my emotions.
There are also disrespectful ways to convince me to do something different.
Many of those ways involve appealing to my rationality.
Many of those ways involve appealing to my emotions.
Well this runs into the problem of giving unsolicited advice. Most people don't respond well to that. I think it's probably difficult for most rationalists to remember this since we are probably more open to that.
But your environment includes people, dude.
This shouldn't be a puzzle. Reinforcement happens, consciously or subconsciously. Why in the name of FSM would you choose to relinquish the power to actually control what would otherwise happen just subconsciously?
How is that not on the face of it a paragon, a prototype of optimization? Isn't that optimizing is, more or less-consciously changing what is otherwise unconscious?
Oh, you're definitely on to something, and it's something important.
That said, I don't think what you're on to has to do with whether and when it's ethical to manipulate people's behavior.
So what am I on to then?
Roughly, that we often respond to others' ability to cause us harm (whether by modifying our behavior or our bank accounts or our internal organs or whatever other mechanism) as a threat, independent of their likelihood of causing us harm.
So if you demonstrate, or even just tell me about, your ability to do these things, then while depending on the specific context, my specific reaction will be somewhat different... my reaction to you knowing my bank PIN number will be different from my reaction to you knowing how to modify my behavior or how to modify the beating of my heart or how to break into my home... they will all have a common emotional component: I will feel threatened, frightened, suspicious, attacked, violated.
That all is perfectly natural and reasonable. And a common and entirely understandable response to that might be for me to declare that, OK, maybe you are able do those things, but a decent or ethical person never will do those things. (That sort of declaration is one relatively common way that I can attempt to modify your likelihood of performing those actions. I realize that you would only consider that a form of manipulation if I realize that such declarations will modify your likelihood of performing those actions. Regardless, the declaration modifies your behavior just the same whether I realize it or not, and whether it's manipulation or not.)
But it doesn't follow from any of that that it's actually unethical for you to log into my bank account, modify my heartbeat, break into my home, or modify my behavior. To my mind, as I said before, the determiner of whether such behavior is ethical or not is whether the result leaves me better or worse off.
Breaking into my home to turn off the main watervalve to keep my house from flooding while I'm at work is perfectly ethical, indeed praiseworthy, and I absolutely endorse you doing so. Nevertheless, I suspect that if you told me that you spent a lot of time thinking about how to break into my home, I would become very suspicious of you.
Again, my emotional reaction to your demonstrated or claimed threat capacity is independent of my beliefs about your likely behaviors, let alone my beliefs about your likely intentions.
This seems very implausible to me. I often encounter people with the ability to do me great harm (a police officer with a gun, say), and this rarely if ever causes me to be angry, or feel as if my dignity has been infringed upon, or anything like that. Yet these are the reactions typically associated with finding out you've been intentionally manipulated. Do you have some independent reason to believe this is true?
Yes, but no reasons I can readily share. And, sure, I might be wrong.
I don't think I would be suspicious of him, as long as I agreed with the behaviours he was trying to reinforce. (I don't know for sure–my reactions are based only on a thought experiment.) I think I would be grateful, both that he cared enough about me to put that much time and effort in, and that he considered me emotionally mature enough to tell me honestly what he was doing.
However, I do think that being aware of his deliberate reinforcement might make it less effective. Being reinforced for Behaviour A would feel less like "wow, the world likes it when I do A, I should do it more!" and more like "Person X wants me to do A", which is a bit less motivating.
Really? So say I tell you that all those times that I smiled at you and asked how you were doing were part of a long term plan to change the way you behave. The next day I smile and ask you how you're doing. Has my confession done nothing to change the way you think about my question?
I'm saying that things like smiles and friendly, concerned questions have a certain importance for us that is directly undermined by their being used for for the purposes of changing our behavior. I don't think using them this way is always bad, but it seems to me that people who generally treat people this way are people we tend not to like once we discover the nature of their kindness.
Like I said, thoughts experiments about "how would I feel if X happened" are not always accurate. However, when I try to simulate that situation in my head, I find that although I would probably think about his smile and question differently (and be more likely to respond with a joke along the lines of "trying to reinforce me again, huh?") I don't think I would like him less.
Anyway, I think I regularly use smiles and "how are you doing?" to change the way people behave...namely, to get strangers, i.e. coworkers at a new job, to start liking me more.
Well, I guess I'll tap out then. I'm not sure how to voice my position at this point.
Your position is that you have a certain emotional response to knowing someone is trying to modify your behaviour. My position is that I have a different emotional response. I can imagine myself having an emotional response like yours...I just don't. (Conversely, I can imagine someone experiencing jealousy in the context of a relationship, but romantic jealousy isn't something I really experience personally.) I don't think that makes either of us wrong.
Well, my position is that doing things like asking how someone is doing so as to reinforce behavior rather than because you want to know the answer is ethically bad. I used the example of the friend to try to motivate and explain that position, but at some point if you are totally fine with that sort of behavior, I don't have very much to argue with. I think you're wrong to be fine with that, but I also don't think I can mount a convimcing argument to that effect. So you've pretty much reached the bottom of my thoughts on the matter, such as they are.
I'm curious about whether your reasons for considering this kind of behaviour "unethical" are consequentialist (i.e. a world where people do X is going to be worse overall than a world where no one does X) or deontological (there are certain behaviours, like lying or stealing, that are just bad no matter what world they take place in, and using social cues to manipulate other people is a behaviour that falls into that class.)
Can you express your personal ethics explicitly and clarify where it comes from?
.... And here begins the debate.
What do we do? What do we think about this piece of freaking powerful magic-science?
I vote we keep it a secret. Some secrets are too dangerous and powerful to be shared.
I think the cat is out of the bag on this one.
This statement without context is clearly incorrect; there are all sorts of behaviors we can ethically execute with respect to both humans and other animals. I understand that what you and the OP both mean to connote is particular behaviors which we restrict in typical contexts only to non-human animals, but if you're going to label them as unethical when applied to humans it helps to specify what behaviors and context those are.
That's a little more specific, but not too much, as I'm not really sure what you mean by "convincing" here.
That is, if at time T1 I don't exhibit behavior B and don't assert that I should exhibit B, and you perform some act A at T2 after which I exhibit B and assert that I should exhibit B, is A an act of convincing me (and therefore OK on your account) or not (and therefore unethical on your account)? How might I test that?
This, on the other hand, is clear. Thank you.
I disagree with it strongly.
That story doesn't trouble you at all?
For most people, there's lots of low hanging fruit from trying to recognize when they are reinforcing and punishing behaviors of others. Also, positive reinforcement is more effective at changing behavior than positive punishment.
But that doesn't mean that we should embrace conditioning-type behavior-modification wholesale. I'm highly doubtful that conditioning responses are entirely justifiable by decision-theoretic reasons. And "not justifiable by decision theoretic reasons" is a reasonable definition of non-rational. Which implies that relying on those types of processes to change others behaviors might be unethical.
Does it trouble me at all? I suppose. Not a huge amount, but some. Had Esar said "Doing this to people without their consent is troubling" rather than "never do this to other people without their explicit consent" I likely wouldn't have objected.
My response to the rest of this would mostly be repeating myself, so I'll point to here instead.
More generally, "conditioning-type behavior-modification" isn't some kind of special category of activity that is clearly separable from ordinary behavior. We modify one another's behavior through conditioning all the time. You did it just now when you replied to my comment. Declaring it unethical across the board seems about as useful as saying "never kill a living thing."
You seem to know what I mean, so I won't go into a buch of unnecessary qualifications.
Not necessarily. Is the meaning of 'convince' really unclear? Threatening someone with a gun seems to satisfy your description, but it's obviously not a case of convincing. I'm not sure what you're unclear about.
If you care to explain why, please do so.
Sure.
The easiest way to get at it is with an example.
Suppose I decide I want my coworkers to visit my desk more often at work, and therefore begin a practice of smiling at everyone who visits, keeping treats on my desk and inviting visitors to partake, being nicer to people when they visit me at my desk than I am at other times, and otherwise setting up a schedule of differential reinforcement designed to increase the incidence of desk-visiting behavior, and I do all of that without ever announcing to anyone that I'm doing it or why I'm doing it, let alone securing anyone's consent. (Let alone securing everyone's consent.)
Do you consider that an example of unethical behavior? I don't.
Now, maybe you don't either. Maybe it's "obviously" not an example of manipulating the behavior of other people by means other than convincing them that they should behave in a certain way. I don't really know, since you've declined to clarify your constraints. But it sure does seem to match what you described.
You're right that this doesn't seem quite unethical, but it is awfully creepy and I'm not sure how to pull my intuitions apart there. Sitting across from someone who is faking affection and smiles and pleasantries so as to manipulate my behavior would cause me to avoid them like the plague.
In professional environments I find this happens all the time, and when the fake friendliness is discovered as such, the effect reverses considerably. If it's terribly important to something's being effective that the person you're doing it to doesn't know what's going on, it's probably bad.
(nods) Absolutely. I could have also framed it to make it seem far creepier, or to make it seem significantly less creepy.
In particular, the use of loaded words like "faking" and "manipulate" ups the creepy factor of the description a lot. The difference between faking affection and choosing to be affectionate is difficult to state precisely, but boy do we respond to the difference between the words!
I agree that most activities which depend on my ignorance for their effectiveness are bad. I even agree that a higher percentage of activities which depend on my ignorance for their effectiveness are bad than the equivalent percentage of activities that don't so depend.
That said, you seem to be going from that claim to the implicit claim that they are bad by virtue of depending on my ignorance. That's less clear to me.
I'll put it simply: if someone asks me about my kids, neither to be polite nor because they care, but because they want to change the way I behave, then they're (in most cases) being manipulative and insincere. While perhaps they're not wronging me, per se, it's certainly not something that speaks well of them, ethically speaking. If you find this controversial, then you surprise me.
It would be bad advice, I think, to encourage people to use positive reinforcement on others when their ignorance is necessary for it to be effective. Not just practically bad advice, as people are pretty good at picking up on fake friendliness. But full stop ethically damaging advice, if taken seriously. I'm not saying that every such case is going to be unethical, but I'm not in the business of lawlike ethical principles anyway.
No, what I said was that behaviors which depend on someone's ignorance for their effectiveness are often also bad behaviors. I didn't say anything one way or the other about a stricter relation between the two properties, but I'll say now that I don't think they're unrelated.
What do you think being polite is?
I agree that asking you about your kids solely to change your behavior is manipulative.
I also agree that it's insincere. (Which is an entirely distinct thing.)
I would also say that asking you about your kids solely to be polite is insincere.
I would not agree that any of these are necessarily unethical.
I am not quite sure what you mean by "ethically damaging advice."
I agree with you that it's not always unethical to positively reinforce others without their knowledge.
I would agree that "Positively reinforcing others without their knowledge is a good thing to do, do it constantly" is advice that, if taken seriously, would often lead me to perform unethical acts. I can accept calling it unethical advice for that reason, I suppose.
But I also think that "Positively reinforcing others without their knowledge is a bad thing to do, never do it." is unethical advice in the same (somewhat unclear) sense.
I agree that behaviors that depend on others' ignorance are often also bad behaviors.
Behaviors that depend on others' knowledge are also often bad behaviors.
Agreed on all counts. In fact, it doesn't look like we disagree at all, judging from your comment.
Oh good!
When you started out by saying "never do this," I concluded otherwise.
I'm pleased to discover I was wrong.
You and Esar both: Taboo 'creepy'? Particularly with an eye to 'why is it important that this situation evokes this emotion'?
Well, I think it's important because IMHO that negative emotional response is what underlies the (incorrect) description of the corresponding behavior as unethical. But I expect Esar would find that implausible.
'Taboo with an eye to this question', not 'answer this question'. I'd already noticed the pattern that people consider finding something creepy to be sufficient reason to label it unethical, but that observation isn't useful for very much beyond predicting other peoples' labeling habits.
Oh, I see.
Sorry, misunderstood.
I could replace "creepy" everywhere it appears with "emotionally disquieting", but I'm not sure what that would help. I figured using the same language Esar was using would be helpful, but I may well have been wrong.
I think it's false to suggest that pleasantries are being outright faked. This person is probably not sitting there going, "Oh, woe is me, I have to pay the horrible price of smiling and being nice to these imbeciles in order to make them give me what I want; I would never do that otherwise." In fact, why would he even want his coworkers to visit his desk more if he had such utter contempt for them that he had to fake affection wholesale?
Rather, like many people, there's a part of him which would probably like to be a nicer person overall, but he can't always bring himself to live up to the ideal. "People will visit my desk more" is a good immediate incentive to be a better person. The coworker who wants more people to visit their desk is also affected by the results of his own behavior. He'll probably be happier because of the visitations, and his happiness would cause him to smile more, and the very act of smiling would make him even more happy. After a while the "initial motivation," whether it was 100% selfish "I want people to visit my desk more; damn their own desires" or the 100% altruistic "I want to manipulate myself into being a nicer person," or, more likely, a mixture of the two, has faded away, and all that remains is the slightly modified, more pleasant person.
I don't understand how using friendly behavior to reinforce people visiting one's desk precludes that behavior being genuine. You seem to be dismissing the possibility that the person in question feels real affection, and is smiling because they are in fact happy that their desk is being visited. Just because they are using their (real) positive response to coworkers visiting their desk as positive reinforcement doesn't mean that their behavior is "fake" in any way.
Just like a woman who feels a surge of affection towards her husband when he puts away the laundry, and kisses or praises him.
Yes, it's positive reinforcement, but it's also a genuine response.