Lots of cool videos of operant conditioning and animals are here (also click on the other chapters).

I just read Don't Shoot The Dog, and one of the interesting bits was that it seemed like getting trained the way it described was fun for the animals, like a good game. Also as the skill was learnt the task difficulty level was raised so it wasn't too easy. And the rewards seemed somewhat symbolic - a clicker, and being fed with food that wasn't officially restricted outside the training sessions.

Thinking about applying it to myself, having the reward not be too important outside the game/practise means I'm not likely to want to bypass the game to get the reward directly. Having the system be fun means it's improving my quality of life in that way in addition to any behaviour change.

I haven't done much about ramping up the challenge. How does one make doing the dishes more challenging?

But I did make sure to make the rewards quicker/more frequent by rewarding subtasks.

Wow, thanks for this great article that was the final piece of information that tipped me over towards getting my shit together. Within 10 minutes after reading it and browsing the comments, I was on my bicycle going to buy small treats I like, that I now give myself for every achieved small goal (~2-10 min of work).

I now wonder though if maybe I should give myself another reinforcer when starting to work with a new goal, otherwise maybe I will only strive for finishing as fast as possible, but starting with a new small goal won't be that much reinforced? Maybe this is my mind trying to get more candy though, so I would be thankful for outside perspective.

related: http://gettingstronger.org/2012/01/hormesis-and-the-limbic-brain/

"Reprogramming the amygdala. This is the indirect way to re-program the hypothalamus, by altering the amygdaloid reward circuitry that feeds it. There are a number approaches to achieving this, some of which I’ve outlined in previous articles, but all of them fall generally under the umbrella of classical or Pavlovian conditioning. There are a few basic strategies:

Extinction. An addictive response becomes weaker and eventually dies out when you stop responding to a triggering cue. This approach works, but can take a long time and requires patience and discipline.
Cue exposure or deconditioning. This involves deliberate, repeated and provocative exposure to the triggering cue, withholding the response. After some initial discomfort, this approach proceeds rapidly and can be quite effective. Success is improved the more realistic and varied the presentation of the cue.
Putting on cue. A new cue is developed and the behavior is only allowed in the presence of this cue. It could be a special sound, or a location. Then the special cue is withheld and the behavior disappears.
Counter conditioning. This involves the substitution of an alternative behavior to actively displace the old reward circuitry. It can be very effective.

"

Any non-sugary food related reinforcer?

So, reinforcement with M&Ms doesn't translate into an addiction for extrinsic rewards and the reduction of intrinsic motivation?

I'm missing something here, I know.

The lead article conflates two process: habits and incentives. The very term "reinforcement" dates back to before the distinction was well-understood. Only in the last decade has it been known that habit operates from a neurology distinct from incentives. (The habit mechanism is in a much older part of the brain.) Only the first story, Yudkowsky and the jellybeans, deals clearly with reinforcement of habit. The others are probably primarily adjustment of incentives.

In using habit and incentive, different rules apply. Incentives require that the subject discern the contingency. The processes Skinner studied as "reinforcement" are mostly about incentives. You adjust schedules of reinforcement to alter the organism's expectancies. For incentive effects, consistent reinforcement is not usually best, as the results are subject to extinction soon after the organism stops getting the reward.

Habits, on the other hand, are blind. The organism doesn't need to see any contingency. Yudkowsky continued to be nice even after he no longer received the jellybeans. To form habits, as opposed to incentive structures, consistency is key.

In short, as a general rule, you want consistency to reward habits and considerable randomness to create lasting incentives.

But the difference extends also to the ethical questions raised. Altering others' incentives for our own benefit is part of ordinary human interaction. If his colleagues surreptitiously timed the offer of jellybeans to Yudkowsky when he acted nice, this is something else; the ethical reason is that Yudkowsky need not recognize what he's being rewarded for to be affected by the jellybeans.

Both habit and incentive are "powerful." But they're powerful for different reasons, in different ways; and to apply them effectively and ethically requires different procedures.

This post may have the highest upvotes per comment I've ever seen. Anyone got access to the database want to confirm that?

Edit: relevant quotes from the post:

When trying to maintain order in a class, ignore unruly behavior and praise good behavior (Madsen et al. 1968; McNamara 1987).

To help someone improve at dance or sport, ignore poor performance but reward good performance immediately, for example by shouting "Good!" (Buzas & Allyon 1981) The reason you should ignore poor performance if you say "No, you're doing it wrong!" you are inadvertently punishing the effort. A better response to a mistake would be to reinforce the effort: "Good effort! You're almost there! Try once more."

Reward opinion-expressing to get people to express their opinions more often

Now that we all know this, shouldn't we abolish downvotes? From my personal experience the emotional impact of a downvote is extremely frustrating and not helpful at all. The message I get from a downvote is "You are wrong!" or "What you said doesn't agree with the group consensus so we will punish you for it!". I don't see this as constructive in any sense.

I read this post last night. I was in the office late, not because I had a great deal to do, but because I was procrastinating. After reading it, I asked my friend to give me a quick call to say congratulations in a half an hour if I'd finished all the work. It took me 10 minutes to finish! :)

What expert timing, Luke! Just two days ago, I came across the fascinating practice of clicker training for horses - http://www.theclickercenter.com, while reading Kathy Sierra's old blog - http://headrush.typepad.com/creating_passionate_users/2006/03/clicker_trained.html.

My only problem is that I need to train my own behaviour rather than someone else's. I'm going to try to use these techniques on myself, although I'm not sure if that's supposed to work.

Lessons learned:

• continue to mentally /ignore people and posts I don't care for on IRC and online forums

• never comment on bad posts or explain my downvote on LW

• be more generous with upvoting good contributions and give a short praise when warranted.

The reason you should ignore poor performance if you say "No, you're doing it wrong!" you are inadvertently punishing the effort. A better response to a mistake would be to reinforce the effort: "Good effort! You're almost there! Try once more.

I am probably unusual in this regard, but I think I would find both approaches equally aggravating. If someone points out that I've made a mistake, anything other than a concise detailing of exactly how what I did differs from what I was supposed to do, is just going to irritate me. Also, my brain tends to interpret being ignored as a signal that I'm doing correctly.

Daniel Kahneman in Thinking, Fast and Slow:

I had stumbled onto a significant fact of the human condition: the feedback to which life exposes us is perverse. Because we tend to be nice to other people when they please us and nasty when they do not, we are statistically punished for being nice and rewarded for being nasty.

There reason for that lies in regression to the mean when training (example of flight instructors in the israel airforce):

I pointed out to the instructors that what they saw on the board coincided with what we had heard about the performance of aerobatic maneuvers on successive attempts: poor performance was typically followed by improvement and good performance by deterioration, without any help from either praise or punishment.

Since positive reinforcement is so counterintuitive: don't forget to reward yourself for rewarding somebody for good behaviour! :)

Note that there are many circumstances when it is right to criticise. For instance group brainstorming exercises are more productive if the participants criticise each others ideas.

Does this still work if I reinforce myself? Every time I read 5 lesswrong articles in a day, I give myself a reward. Or every time i have a cigarette, I kick a brick wall with no shoes on. If i was consistent with this for a long time, would it work?

This seems to contradict the very powerful effect of learning from failure and corrective feedback. See http://www.wired.com/wiredscience/2011/10/why-do-some-people-learn-faster-2/ for an accessible overview.

I'd conjecture this works better when someone can already perform the desired behavior and wants to form a habit, whereas learning from failure comes in when new information needs to be stored and reorganized.

The central lesson I learned from exotic animal trainers is that I should reward behavior I like and ignore behavior I don't. After all, you don't get a sea lion to balance a ball on the end of its nose by nagging. The same goes for the American husband.

Back in Maine, I began thanking Scott if he threw one dirty shirt into the hamper. If he threw in two, I'd kiss him. Meanwhile, I would step over any soiled clothes on the floor without one sharp word, though I did sometimes kick them under the bed. But as he basked in my appreciation, the piles became smaller.

My wife, if pulling that kind of stunt, would quickly find that her affections were shunned and her thanks were met with clear contempt (after she was asked politely not to do that the first time). It is almost certainly not in her interests to produce a pavlovian association between her affections and attempts to control me against my wishes. My aversion to hostile takeover of internal motivations is much stronger than my desire for the affections of any particular individual.

This would be entirely different if I had made a prior agreement regarding shirts and hampers. Making it motivationally easier and more enjoyable to do things I am willing to do is to be encouraged.

Maybe it's because behaviorist techniques like reinforcement feel like they don't respect human agency enough. But if you aren't treating humans more like animals than most people are, then you're modeling humans poorly.

But treating human beings, especially adults, like animals is characteristically unethical. Applying some system of reinforcement where someone has asked you to effectively treat their behavior is innocuous enough, as is of course treating yourself.

But generally manipulating the behavior of other people by means other than convincing them that they should behave in a certain way seems to me to be almost definitional of a dark art. If that's not controversial, then I think this article should be qualified appropriately: never do this to other people without their explicit consent.

To help someone improve at dance or sport, ignore poor performance but reward good performance immediately, for example by shouting "Good!" (Buzas & Allyon 1981) The reason you should ignore poor performance if you say "No, you're > doing it wrong!" you are inadvertently punishing the effort. A better response to a mistake would be to reinforce the effort: "Good effort! You're almost there! Try once more."

I've noticed in pilates classes with one specific teacher you get positive feedback in one specific situation - when you're having trouble, and have just barely managed something basic. This leads to the association that whenever you get positive comments you know you're doing badly.

Thank you Luke for this beautifully written post.

A while ago I saw a kindly waitress give my friend's two year old daughter a small cookie in a restaurant. Various emotions flickered across her tiny face, and then she made a decision, accompanied by a small smile.

She broke the cookie into three pieces and gave them to her brothers. Completely unprompted.

I couldn't believe my eyes. I asked my friend, who is a lecturer in experimental psychology, whether altruism was normal amongst very young siblings.

He looked a bit smug and said "Well we put a lot of reinforcement into that."

I hadn't really thought about what that meant until now. Your clear writing has made it obvious.

As a result of your post, I think I'm going to try deliberately modifying some of my own behaviours this way, and maybe try the techniques on some friends. (The first time, by the way, that I've changed my behaviour as a result of reading less wrong, rather than just treating it as philosophical crack.)

For friends it seems that sincere praise / avoiding criticism would be good, but what would you recommend as rewards to self? I'm pretty sure that nicotine and pizza slices would work for me, but I'm also sure that those aren't things I want to do more of.

http://en.wikipedia.org/wiki/Love_bombing

This is getting creepy.