I remember a poster that hung on the wall of my seventh grade classroom. It went like this:

Watch your thoughts, for they become words.
Watch your words, for they become actions.
Watch your actions, for they become habits.
Watch your habits, for they become your character.
Watch your character, for it becomes your destiny.

It was as a competitive swimmer that these words were the most meaningful to me. Most sports are ultimately about the practice, about repeating an action over and over and over again, so that actions become habits and habits become character. The fleeting thought that I really hate getting up at 5:00 am for swim practice is just that: a fleeting thought. But if I justified it with words, speaking it aloud to my parents or siblings or friends, it became a fact that others knew about me, much realer than just a wispy thought. The action of forgetting-on-purpose to set my alarm, or faking sick, was a logical next step. And one missed practice might not be huge, in the long run, but it led easily to a habit of missing practice, say, once a week. A year of this, and I would start to think of myself as the kind of person who missed practice once a week, because after all, isn’t it silly of anyone to expect a twelve-year-old to get up at 5:00 three times a week? And that attitude could very easily have led, over a couple of years, to quitting the team. 

Related: Purchase Fuzzies and Utilons Separately

We genuinely want to do good in the world; but also, we want to feel as if we're doing good, via heuristics that have been hammered into our brains over the course of our social evolution. The interaction between these impulses (in areas like scope insensitivity, refusal to quantify sacred values, etc.) can lead to massive diminution of charitable impact, and can also suck the fun out of the whole process. Even if it's much better to write a big check at the end of the year to the charity with the greatest expected impact than it is to take off work every Thursday afternoon and volunteer at the pet pound, it sure doesn't feel as rewarding. And of course, we're very good at finding excuses to stop doing costly things that don't feel rewarding, or at least to put them off.

But if there's one thing I've learned here, it's that lamenting our irrationality should wait until one's properly searched for a good hack. And I think I've found one.

Not just that, but I've tested it out for you already.

This summer, I had just gone through the usual experience of being asked for money for a nice but inefficient cause, turning them down, and feeling a bit bad about it. I made a mental note to donate some money to a more efficient cause, but worried that I'd forget about it; it's too much work to make a bunch of small donations over the year (plus, if done by credit card, the fees take a bigger cut that way) and there's no way I'd remember that day at the end of the year.

Unless, that is, I found some way to keep track of it.

So I made up several jars with the names of charities I found efficient (SIAI and VillageReach) and kept a bunch of poker chips near them. Starting then, whenever I felt like doing a good deed (and especially if I'd passed up an opportunity to do a less efficient one), I'd take a chip of an appropriate value and toss it in the jar of my choice. I have to say, this gave me much more in the way of warm fuzzies than if I'd just waited and made up a number at the end of the year.

And now I've added up and made my contributions: $1,370 to SIAI and $566 to VillageReach.

In which I attempt to apply findings from behavioral psychology to my own life.

Behavioral Psychology Finding #1: Habituation

The psychological process of "extinction" or "habituation" occurs when a stimulus is administered repeatedly to an animal, causing the animal's response to gradually diminish.  You can imagine that if you were to eat your favorite food for breakfast every morning, it wouldn't be your favorite food after a while.  Habituation tends to happen the fastest when the following three conditions are met:

• The stimulus is delivered frequently
• The stimulus is delivered in small doses
• The stimulus is delivered at regular intervals

Source is here.

Applied Habituation

I had a project I was working on that was really important to me, but whenever I started working on it I would get demoralized.  So I habituated myself to the project: I alternated 2 minutes of work with 2 minutes of sitting in the yard for about 20 minutes.  This worked.

Related to: Ugh Fields, Are Wireheads Happy?

In his post Ugh Fields, Roko discussed "temporal difference learning", the process by which the brain propagates positive or negative feedback to the closest cause it can find for the feedback. For example, if he forgets to pay his bills and gets in trouble, the trouble (negative feedback) propagates back to thoughts about bills. Next time he gets a bill, he might paradoxically have even more trouble paying it, because it's become associated with trouble and negative emotions, and his brain tends to unconsciously flinch away from it.

He links to the associated Wikipedia article:

The TD algorithm has also received attention in the field of neuroscience. Researchers discovered that the firing rate of dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) appear to mimic the error function in the algorithm. The error function reports back the difference between the estimated reward at any given state or time step and the actual reward received. The larger the error function, the larger the difference between the expected and actual reward. When this is paired with a stimulus that accurately reflects a future reward, the error can be used to associate the stimulus with the future reward.

Dopamine cells appear to behave in a similar manner. In one experiment measurements of dopamine cells were made while training a monkey to associate a stimulus with the reward of juice. Initially the dopamine cells increased firing rates when exposed to the juice, indicating a difference in expected and actual rewards. Over time this increase in firing back propagated to the earliest reliable stimulus for the reward. Once the monkey was fully trained, there was no increase in firing rate upon presentation of the predicted reward. This mimics closely how the error function in TD is used for reinforcement learning.

So if I understand this right, the monkey hears a bell and is unimpressed, having no expectation of reward. Then the monkey gets some juice that tastes really good and activates (opioid dependent?) reward pathways. The dopamine system is pretty surprised, and broadcasts that surprise back to all the neurons that have been especially active recently, most notably the neurons that activated upon hearing the bell. These neurons are now more heavily associated with the dopamine system. So the next time the monkey hears a bell, it has a greater expectation of reward.

And in this case it doesn't matter, because the monkey can't do anything about it. But if it were a circus monkey, and its trainer was trying to teach it to do a backflip to get juice, the association between backflips and juice would be pretty useful. As long as the monkey wanted juice, merely entertaining the plan of doing a backflip would have motivational value that promotes the correct action.

The Sinclair Method is a promising technique for treating alcoholics that elegantly demonstrates these pathways by sabotaging them.

Note: this post is basically just summarizing some of PJ Eby's freely available writings on the topic of pain/gain motivation and presenting them in a form that's easier for the LW crowd to digest. I claim no credit for the ideas presented here, other than the credit for summarizing them.

EDIT: Note also Eby's comments and corrections to my summary at this comment.

Eby proposes that we have two different forms of motivation: positive ("gain") motivation, which drives us to do things, and negative ("pain") motivation, which drives us to avoid things. Negative motivation is a major source of akrasia and is mostly harmful for getting anything done. However, sufficiently large amounts of negative motivation can momentarily push us to do things, which frequently causes people to confuse the two.

To understand the function of negative motivation, first consider the example of having climbed to a tree to avoid a predator. There's not much you can do other than wait and hope the predator goes away, and if you move around, you risk falling out of the tree. So your brain gets flooded with signals that suppress activity and tell it to keep your body still. It is only if the predator ends up climbing up the tree that the danger becomes so acute that you're instead pushed to flee.

What does this have to do with modern-day akrasia? Back in the tribal environment, elicting the disfavor of the tribe could be a death sentence. Be cast out by the tribe, and you likely wouldn't live for long. One way to elict disfavor is to be unmasked as incompetent in some important matter, and a way to avoid such an unmasking is to simply avoid doing anything where to consequences of failure would be severe.

You might see why this would cause problems. Sometimes, when the pain level of not having done a task grows too high - like just before a deadline - it'll push you to do it. But this fools people into thinking that negative consequences alone will be a motivator, so they try to psyche themselves up by thinking about how bad it would be to fail. In truth, this is only making things worse, as an increased chance of failure will increase the negative motivation that's going on.

Reply to: Practical Advice Backed By Deep Theories

Inspired by what looks like a very damaging reticence to embrace and share brain hacks that might only work for some of us, but are not backed by Deep Theories. In support of tinkering with brain hacks and self experimentation where deep science and large trials are not available.

Eliezer has suggested that, before he will try a new anti-akraisia brain hack:

[…] the advice I need is from someone who reads up on a whole lot of experimental psychology dealing with willpower, mental conflicts, ego depletion, preference reversals, hyperbolic discounting, the breakdown of the self, picoeconomics, etcetera, and who, in the process of overcoming their own akrasia, manages to understand what they did in truly general terms - thanks to experiments that give them a vocabulary of cognitive phenomena that actually exist, as opposed to phenomena they just made up.  And moreover, someone who can explain what they did to someone else, thanks again to the experimental and theoretical vocabulary that lets them point to replicable experiments that ground the ideas in very concrete results, or mathematically clear ideas.

This doesn't look to me like an expected utility calculation, and I think it should. It looks like an attempt to justify why he can't be expected to win yet. It just may be deeply wrongheaded.

I submit that we don't "need" (emphasis in original) this stuff, it'd just be super cool if we could get it. We don't need to know that the next brain hack we try will work, and we don't need to know that it's general enough that it'll work for anyone who tries it; we just need the expected utility of a trial to be higher than that of the other things we could be spending that time on.

So… this isn't other-optimizing, it's a discussion of how to make decisions under uncertainty. What do all of us need to make a rational decision about which brain hacks to try?

• We need a goal: Eliezer has suggested "I want to hear how I can overcome akrasia - how I can have more willpower, or get more done with less mental pain". I'd push cost in with something like "to reduce the personal costs of akraisia by more than the investment in trying and implementing brain hacks against it plus the expected profit on other activities I could undertake with that time".

• We need some likelihood estimates:
  
  • Chance of a random brain hack working on first trial: ?, second trial: ?, third: ?
  • Chance of a random brain hack working on subsequent trials (after the third - the noise of mood, wakefulness, etc. is large, so subsequent trials surely have non-zero chance of working, but that chance will probably diminish): →0
  • Chance of a popular brain hack working on first (second, third) trail: ? (GTD is lauded by many many people; your brother in law's homebrew brain hack is less well tried)
  • Chance that a brain hack that would work in the first three trials would seem deeply compelling on first being exposed to it: ?
    (can these books be judged by their covers? how does this chance vary with the type of exposure? what would you need to do to understand enough about a hack that would work to increase its chance of seeming deeply compelling on first exposure?)
  • Chance that a brain hack that would not work in the first three trials would seem deeply compelling on first being exposed to it: ? (false positives)
  • Chance of a brain hack recommended by someone in your circle working on first (second, third) trial: ?
  • Chance that someone else will read up "on a whole lot of experimental psychology dealing with willpower, mental conflicts, ego depletion, preference reversals, hyperbolic discounting, the breakdown of the self, picoeconomics, etcetera, and who, in the process of overcoming their own akrasia, manages to understand what they did in truly general terms - thanks to experiments that give them a vocabulary of cognitive phenomena that actually exist, as opposed to phenomena they just made up.  And moreover, someone who can explain what they did to someone else, thanks again to the experimental and theoretical vocabulary that lets them point to replicable experiments that ground the ideas in very concrete results, or mathematically clear ideas", all soon: ? (pretty small?)
  • What else do we need to know?
• We need some time/cost estimates (these will vary greatly by proposed brain hack):
  • Time required to stage a personal experiment on the hack: ?
  • Time to review and understand the hack in sufficient detail to estimate the time required to stage a personal experiment?
  • What else do we need?

… and, what don't we need?

• A way to reject the placebo effect - if it wins, use it. If it wins for you but wouldn't win for someone else, then they have a problem. We may choose to spend some effort helping others benefit from this hack, but that seems to be a different task - it's irrelevant to our goal.

How should we decide how much time to spend gathering data and generating estimates on matters such as this? How much is Eliezer setting himself up to lose, and how much am I missing the point?