Bit of a tangent, but if you ever run across someone for whom this doesn't seem to work, check the hypothesis that they don't parse praise as a positive reinforcer. I don't know how common this is, but I actually have to make a conscious effort to keep it from acting as a mild punishment in most cases when it's applied to me. (Ditto M&Ms in the given context, I expect. Attention Bad.)

Good post! Thank you for writing it Luke =)

"Eventually it hit me that the same techniques might work on that stubborn but lovable species, the American wife." "Back in Maine, I began thanking Amy if she threw one dirty shirt into the hamper. If she threw in two, I'd kiss her." "...After two years of exotic animal training, my marriage is far smoother, my wife much easier to love."

Thank you Luke for this beautifully written post.

A while ago I saw a kindly waitress give my friend's two year old daughter a small cookie in a restaurant. Various emotions flickered across her tiny face, and then she made a decision, accompanied by a small smile.

She broke the cookie into three pieces and gave them to her brothers. Completely unprompted.

I couldn't believe my eyes. I asked my friend, who is a lecturer in experimental psychology, whether altruism was normal amongst very young siblings.

He looked a bit smug and said "Well we put a lot of reinforcement into that."

I hadn't really thought about what that meant until now. Your clear writing has made it obvious.

As a result of your post, I think I'm going to try deliberately modifying some of my own behaviours this way, and maybe try the techniques on some friends. (The first time, by the way, that I've changed my behaviour as a result of reading less wrong, rather than just treating it as philosophical crack.)

For friends it seems that sincere praise / avoiding criticism would be good, but what would you recommend as rewards to self? I'm pretty sure that nicotine and pizza slices would work for me, but I'm also sure that those aren't things I want to do more of.

I like this article because it is reasonably short, but very clear and highly actionable.

Reason #228 I'm crazy and irrational: Without conscious attention to the reinforcement process, my behaviors are selected for reinforcement almost at random. The process selecting behaviors for reinforcement has tons of steps in it like "Did I happen to glance in the direction of the bag of M&Ms right now?" instead of "Is the thing I'm doing now something I want to reinforce?"

To help someone improve at dance or sport, ignore poor performance but reward good performance immediately, for example by shouting "Good!" (Buzas & Allyon 1981) The reason you should ignore poor performance if you say "No, you're > doing it wrong!" you are inadvertently punishing the effort. A better response to a mistake would be to reinforce the effort: "Good effort! You're almost there! Try once more."

I've noticed in pilates classes with one specific teacher you get positive feedback in one specific situation - when you're having trouble, and have just barely managed something basic. This leads to the association that whenever you get positive comments you know you're doing badly.

Eagerly awaiting "The Power of Punishment".

The lead article conflates two process: habits and incentives. The very term "reinforcement" dates back to before the distinction was well-understood. Only in the last decade has it been known that habit operates from a neurology distinct from incentives. (The habit mechanism is in a much older part of the brain.) Only the first story, Yudkowsky and the jellybeans, deals clearly with reinforcement of habit. The others are probably primarily adjustment of incentives.

In using habit and incentive, different rules apply. Incentives require that the subject discern the contingency. The processes Skinner studied as "reinforcement" are mostly about incentives. You adjust schedules of reinforcement to alter the organism's expectancies. For incentive effects, consistent reinforcement is not usually best, as the results are subject to extinction soon after the organism stops getting the reward.

Habits, on the other hand, are blind. The organism doesn't need to see any contingency. Yudkowsky continued to be nice even after he no longer received the jellybeans. To form habits, as opposed to incentive structures, consistency is key.

In short, as a general rule, you want consi... (read more)

That's why I tried to stay positive when talking about the new SI website. Especially with technical changes like that, the (vocal) negative response can be overwhelming.

To help someone improve at dance or sport, ignore poor performance but reward good performance immediately, for example by shouting "Good!" (Buzas & Allyon 1981) The reason you should ignore poor performance if you say "No, you're doing it wrong!" you are inadvertently punishing the effort. A better response to a mistake would be to reinforce the effort: "Good effort! You're almost there! Try once more."

I got a demonstration of how true this is yesterday when, during my taekwondo class, I was paired up with one of the senior black belt students, who has some but not a lot of experience teaching. He was supposed to be fixing up my poomsae (same thing as a kata in karate) and each time he watched me do it, I would finish and he would immediately launch into a description of what I was doing wrong. His feedback was pretty useful–specific, with demonstrations of exactly what to change in order to do it right–but without any prelude of "yay, good job!" or even "okay, the punches were way better that time...now let's work on the stances", I found myself getting really discouraged. Reminding myself that I wasn't actually doing wors... (read more)

I read this post last night. I was in the office late, not because I had a great deal to do, but because I was procrastinating. After reading it, I asked my friend to give me a quick call to say congratulations in a half an hour if I'd finished all the work. It took me 10 minutes to finish! :)

1. Nice post SIAI! Have an $5 donation!

2. I tried a similar reinforcement technique on myself but it didn't stick because I couldn't find a reliable trigger condition for dispensing the reward.

3. Does this mean that we should stop punishing ourselves for procrastination?

On Skype with Eliezer, I said: "Eliezer, you've been unusually pleasant these past three weeks. I'm really happy to see that, and moreover, it increases my probability than an Eliezer-led FAI research team will work. What caused this change, do you think?"

Eliezer replied: "Well, three weeks ago I was working with Anna and Alicorn, and every time I said something nice they fed me an M&M."

If I recall my high school psychology class correctly, you can get a stronger and more persistent effect by secretly rolling a dice and note the number, and when Eliezer says that many nice things, give him an M&M, roll the dice again for a new target number of nice things.

Daniel Kahneman in Thinking, Fast and Slow:

I had stumbled onto a significant fact of the human condition: the feedback to which life exposes us is perverse. Because we tend to be nice to other people when they please us and nasty when they do not, we are statistically punished for being nice and rewarded for being nasty.

There reason for that lies in regression to the mean when training (example of flight instructors in the israel airforce):

I pointed out to the instructors that what they saw on the board coincided with what we had heard about the performance of aerobatic maneuvers on successive attempts: poor performance was typically followed by improvement and good performance by deterioration, without any help from either praise or punishment.

Since positive reinforcement is so counterintuitive: don't forget to reward yourself for rewarding somebody for good behaviour! :)

The central lesson I learned from exotic animal trainers is that I should reward behavior I like and ignore behavior I don't. After all, you don't get a sea lion to balance a ball on the end of its nose by nagging. The same goes for the American husband.

Back in Maine, I began thanking Scott if he threw one dirty shirt into the hamper. If he threw in two, I'd kiss him. Meanwhile, I would step over any soiled clothes on the floor without one sharp word, though I did sometimes kick them under the bed. But as he basked in my appreciation, the piles became smaller.

My wife, if pulling that kind of stunt, would quickly find that her affections were shunned and her thanks were met with clear contempt (after she was asked politely not to do that the first time). It is almost certainly not in her interests to produce a pavlovian association between her affections and attempts to control me against my wishes. My aversion to hostile takeover of internal motivations is much stronger than my desire for the affections of any particular individual.

This would be entirely different if I had made a prior agreement regarding shirts and hampers. Making it motivationally easier and more enjoyable to do things I am willing to do is to be encouraged.