Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Require contributions in advance

Viliam 08 February 2016 12:55PM

If you are a person who finds it difficult to tell "no" to their friends, this one weird trick may save you a lot of time!


Scenario 1

Alice: "Hi Bob! You are a programmer, right?"

Bob: "Hi Alice! Yes, I am."

Alice: "I have this cool idea, but I need someone to help me. I am not good with computers, and I need someone smart whom I could trust, so they wouldn't steal my idea. Would you have a moment to listen to me?"

Alice explains to Bob her idea that would completely change the world. Well, at the least the world of bicycle shopping.

Instead of having many shops for bicycles, there could be one huge e-shop that would collect all the information about bicycles from all the existing shops. The customers would specify what kind of a bike they want (and where they live), and the system would find all bikes that fit the specification, and display them ordered by lowest price, including the price of delivery; then it would redirect them to the specific page of the specific vendor. Customers would love to use this one website, instead of having to visit multiple shops and compare. And the vendors would have to use this shop, because that's where the customers would be. Taking a fraction of a percent from the sales could make Alice (and also Bob, if he helps her) incredibly rich.

Bob is skeptical about it. The project suffers from the obvious chicken-and-egg problem: without vendors already there, the customers will not come (and if they come by accident, they will quickly leave, never to return again); and without customers already there, there is no reason for the vendors to cooperate. There are a few ways how to approach this problem, but the fact that Alice didn't even think about it is a red flag. She also has no idea who are the big players in the world of bicycle selling; and generally she didn't do her homework. But after pointing out all these objections, Alice still remains super enthusiastic about the project. She promises she will take care about everything -- she just cannot write code, and she needs Bob's help for this part.

Bob believes strongly in the division of labor, and that friends should help each other. He considers Alice his friend, and he will likely need some help from her in the future. Fact is, with perfect specification, he could make the webpage in a week or two. But he considers bicycles to be an extremely boring topic, so he wants to spend as little time as possible on this project. Finally, he has an idea:

"Okay, Alice, I will make the website for you. But first I need to know exactly how the page will look like, so that I don't have to keep changing it over and over again. So here is the homework for you -- take a pen and paper, and make a sketch of how exactly the web will look like. All the dialogs, all the buttons. Don't forget logging in and logging out, editing the customer profile, and everything else that is necessary for the website to work as intended. Just look at the papers and imagine that you are the customer: where exactly would you click to register, and to find the bicycle you want? Same for the vendor. And possibly a site administrator. Also give me the list of criteria people will use to find the bike they want. Size, weight, color, radius of wheels, what else? And when you have it all ready, I will make the first version of the website. But until then, I am not writing any code."

Alice leaves, satisfied with the outcome.


This happened a year ago.

No, Alice doesn't have the design ready, yet. Once in a while, when she meets Bob, she smiles at him and apologizes that she didn't have the time to start working on the design. Bob smiles back and says it's okay, he'll wait. Then they change the topic.


Scenario 2

Cyril: "Hi Diana! You speak Spanish, right?"

Diana: "Hi Cyril! Yes, I do."

Cyril: "You know, I think Spanish is the most cool language ever, and I would really love to learn it! Could you please give me some Spanish lessons, once in a while? I totally want to become fluent in Spanish, so I could travel to Spanish-speaking countries and experience their culture and food. Would you please help me?"

Diana is happy that someone takes interest in her favorite hobby. It would be nice to have someone around she could practice Spanish conversation with. The first instinct is to say yes.

But then she remembers (she knows Cyril for some time; they have a lot of friends in common, so they meet quite regularly) that Cyril is always super enthusiastic about something he is totally going to do... but when she meets him next time, he is super enthusiastic about something completely different; and she never heard about him doing anything serious about his previous dreams.

Also, Cyril seems to seriously underestimate how much time does it take to learn a foreign language fluently. Some lessons, once in a while will not do it. He also needs to study on his own. Preferably every day, but twice a week is probably a minimum, if he hopes to speak the language fluently within a year. Diana would be happy to teach someone Spanish, but not if her effort will most likely be wasted.

Diana: "Cyril, there is this great website called Duolingo, where you can learn Spanish online completely free. If you give it about ten minutes every day, maybe after a few months you will be able to speak fluently. And anytime we meet, we can practice the vocabulary you have already learned."

This would be the best option for Diana. No work, and another opportunity to practice. But Cyril insists:

"It's not the same without the live teacher. When I read something from the textbook, I cannot ask additional questions. The words that are taught are often unrelated to the topics I am interested in. I am afraid I will just get stuck with the... whatever was the website that you mentioned."

For Diana this feels like a red flag. Sure, textbooks are not optimal. They contain many words that the student will not use frequently, and will soon forget them. On the other hand, the grammar is always useful; and Diana doesn't want to waste her time explaining the basic grammar that any textbook could explain instead. If Cyril learns the grammar and some basic vocabulary, then she can teach him all the specialized vocabulary he is interested in. But now it feels like Cyril wants to avoid all work. She has to draw a line:

"Cyril, this is the address of the website." She takes his notebook and writes 'www.duolingo.com'. "You register there, choose Spanish, and click on the first lesson. It is interactive, and it will not take you more than ten minutes. If you get stuck there, write here what exactly it was that you didn't understand; I will explain it when we meet. If there is no problem, continue with the second lesson, and so on. When we meet next time, tell me which lessons you have completed, and we will talk about them. Okay?"

Cyril nods reluctantly.


This happened a year ago.

Cyril and Diana have met repeatedly during the year, but Cyril never brought up the topic of Spanish language again.


Scenario 3

Erika: "Filip, would you give me a massage?"

Filip: "Yeah, sure. The lotion is in the next room; bring it to me!"

Erika brings the massage lotion and lies on the bed. Filip massages her back. Then they make out and have sex.


This happened a year ago. Erika and Filip are still a happy couple.

Filip's previous relationships didn't work well, in long term. In retrospect, they all followed a similar scenario. At the beginning, everything seemed great. Then at some moment the girl started acting... unreasonably?... asking Filip to do various things for her, and then acting annoyed when Filip did exactly what he was asked to do. This happened more and more frequently, and at some moment she broke up with him. Sometimes she provided explanation for breaking up that Filip was unable to decipher.

Filip has a friend who is a successful salesman. Successful both professionally and with women. When Filip admitted to himself that he is unable to solve the problem on his own, he asked his friend for advice.

"It's because you're a f***ing doormat," said the friend. "The moment a woman asks you to do anything, you immediately jump and do it, like a well-trained puppy. Puppies are cute, but not attractive. Have you ready any of those books I sent you, like, ten years ago? I bet you didn't. Well, it's all there."

Filip sighed: "Look, I'm not trying to become a pick-up artist. Or a salesman. Or anything. No offense, but I'm not like you, personality-wise, I never have been, and I don't want to become your - or anyone else's - copy. Even if it would mean greater success in anything. I prefer to treat other people just like I would want them to treat me. Most people reciprocate nice behavior; and those who don't, well, I avoid them as much as possible. This works well with my friends. It also works with the girls... at the beginning... but then somehow... uhm... Anyway, all your books are about manipulating people, which is ethically unacceptable for me. Isn't there some other way?"

"All human interaction is manipulation; the choice is between doing it right or wrong, acting consciously or driven by your old habits..." started the friend, but then he gave up. "Okay, I see you're not interested. Just let me show you the most obvious mistake you make. You believe that when you are nice to people, they will perceive you as nice, and most of them will reciprocate. And when you act like an asshole, it's the other way round. That's correct, on some level; and in a perfect world this would be the whole truth. But on a different level, people also perceive nice behavior as weakness; especially if you do it habitually, as if you don't have any other option. And being an asshole obviously signals strength: you are not afraid to make other people angry. Also, in long term, people become used to your behavior, good or bad. The nice people don't seem so nice anymore, but they still seem weak. Then, ironicaly, if the person well-known to be nice refuses to do something once, people become really angry, because their expectations were violated. And if the asshole decides to do something nice once, they will praise him, because he surprised them pleasantly. You should be an asshole once in a while, to make people see that you have a choice, so they won't take your niceness for granted. Or if your girlfriend wants something from you, sometimes just say no, even if you could have done it. She will respect you more, and then she will enjoy more the things you do for her."

Filip: "Well, I... probably couldn't do that. I mean, what you say seems to make sense, however much I hate to admit it. But I can't imagine doing it myself, especially to a person I love. It's just... uhm... wrong."

"Then, I guess, the very least you could do is to ask her to do something for you first. Even if it's symbolic, that doesn't matter; human relationships are mostly about role-playing anyway. Don't jump immediately when you are told to; always make her jump first, if only a little. That will demonstrate strength without hurting anyone. Could you do that?"

Filip wasn't sure, but at the next opportunity he tried it, and it worked. And it kept working. Maybe it was all just a coincidence, maybe it was a placebo effect, but Filip doesn't mind. At first it felt kinda artificial, but then it became natural. And later, to his surprise, Filip realized that practicing these symbolic demands actually makes it easier to ask when he really needed something. (In which case sometimes he was asked to do something first, because his girlfriend -- knowingly or not? he never had the courage to ask -- copied the pattern; or maybe she has already known it long before. But he didn't mind that either.)


The lesson is: If you find yourself repeatedly in situations where people ask you to do something for them, but at the end they don't seem to appreciate what you did for them, or don't even care about the thing they asked you to do... and yet you find it difficult to say "no"... ask them to contribute to the project first.

This will help you get rid of the projects they don't care about (including the ones they think they care about in far mode, but do not care about enough to actually work on them in near mode) without being the one who refuses cooperation. Also, the act of asking the other person to contribute, after being asked to do something for them, mitigates the status loss inherent in working for them.

Lesswrong Survey - invitation for suggestions

4 Elo 08 February 2016 08:07AM

Given that it's been a while since the last survey (http://lesswrong.com/lw/lhg/2014_survey_results/)


It's now time to open the floor to suggestions of improvements to the last survey.  If you have a question you think should be on the survey (perhaps with reasons why, predictions as to the result, or other useful commentary about a survey question)


Alternatively questions that should not be included in the next survey, with similar reasons as to why...

Conveying rational thinking about long-term goals to youth and young adults

10 Gleb_Tsipursky 07 February 2016 01:54AM
More than a year ago, I discussed here how we at Intentional Insights intended to convey rationality to young adults through our collaboration with the Secular Student Alliance. This international organization unites over 270 clubs at colleges and high schools in English-speaking countries, mainly the US, with its clubs spanning from a few students to a few hundred students. The SSA's Executive Director is an aspiring rationalist and CFAR alum who is on our Advisory Board.

Well, we've been working on a project with the SSA for the last 8 months to create and evaluate an event aimed to help its student members figure out and orient toward the long term, thus both fighting Moloch on a societal level and helping them become more individually rational as well (the long-term perspective is couched in the language of finding purpose using science) It's finally done, and here is the link to the event packet. The SSA will be distributing this packet broadly, but in the meantime, if you have any connections to secular student groups, consider encouraging them to hold this event. The event would also fit well for adult secular groups with minor editing, in case any of you are involved with them. It's also easy to strip the secular language from the packet, and just have it as an event for a philosophy/science club of any sort, at any level from youth to adult. Although I would prefer you cite Intentional Insights when you do it, I'm comfortable with you not doing so if circumstances don't permit it for some reason.

We're also working on similar projects with the SSA, focusing on being rational in the area of giving, so promoting Effective Altruism. I'll post it here when it's ready.  

Request for help with economic analysis related to AI forecasting

6 ESRogs 06 February 2016 01:27AM

[Cross-posted from FB]

I've got an economic question that I'm not sure how to answer.

I've been thinking about trends in AI development, and trying to get a better idea of what we should expect progress to look like going forward.

One important question is: how much do existing AI systems help with research and the development of new, more capable AI systems?

The obvious answer is, "not much." But I think of AI systems as being on a continuum from calculators on up. Surely AI researchers sometimes have to do arithmetic and other tasks that they already outsource to computers. I expect that going forward, the share of tasks that AI researchers outsource to computers will (gradually) increase. And I'd like to be able to draw a trend line. (If there's some point in the future when we can expect most of the work of AI R&D to be automated, that would be very interesting to know about!)

So I'd like to be able to measure the share of AI R&D done by computers vs humans. I'm not sure of the best way to measure this. You could try to come up with a list of tasks that AI researchers perform and just count, but you might run into trouble as the list of tasks to changes over time (e.g. suppose at some point designing an AI system requires solving a bunch of integrals, and that with some later AI architecture this is no longer necessary).

What seems more promising is to abstract over the specific tasks that computers vs human researchers perform and use some aggregate measure, such as the total amount of energy consumed by the computers or the human brains, or the share of an R&D budget spent on computing infrastructure and operation vs human labor. Intuitively, if most of the resources are going towards computation, one might conclude that computers are doing most of the work.

Unfortunately I don't think that intuition is correct. Suppose AI researchers use computers to perform task X at cost C_x1, and some technological improvement enables X to be performed more cheaply at cost C_x2. Then, all else equal, the share of resources going towards computers will decrease, even though their share of tasks has stayed the same.

On the other hand, suppose there's some task Y that the researchers themselves perform at cost H_y, and some technological improvement enables task Y to be performed more cheaply at cost C_y. After the team outsources Y to computers the share of resources going towards computers has gone up. So it seems like it could go either way -- in some cases technological improvements will lead to the share of resources spent on computers going down and in some cases it will lead to the share of resources spent on computers going up.

So here's the econ part -- is there some standard economic analysis I can use here? If both machines and human labor are used in some process, and the machines are becoming both more cost effective and more capable, is there anything I can say about how the expected share of resources going to pay for the machines changes over time?

Weekly LW Meetups

1 FrankAdamek 05 February 2016 04:40PM

This summary was posted to LW Main on January 29th. The following week's summary is here.

Irregularly scheduled Less Wrong meetups are taking place in:

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, London, Madison WI, Melbourne, Moscow, Mountain View, New Hampshire, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

continue reading »

Weekly LW Meetups

1 FrankAdamek 05 February 2016 04:39PM

Irregularly scheduled Less Wrong meetups are taking place in:

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, London, Madison WI, Melbourne, Moscow, Mountain View, New Hampshire, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

continue reading »

[Link] How I Escaped The Darkness of Mental Illness

7 Gleb_Tsipursky 04 February 2016 11:08PM
A deeply personal account by aspiring rationalist Agnes Vishnevkin, who shares the broad overview of how she used rationality-informed strategies to recover from mental illness. She will also appear on the Unbelievers Radio podcast today live at 10:30 PM EST (-5 UTC), together with JT Eberhard, to speak about mental illness and recovery.

**EDIT** Based on feedback from gjm below, I want to clarify that Agnes is my wife and fellow co-founder of Intentional Insights.

Meetup : London Rationalish meetup, 7/2/16

1 philh 04 February 2016 04:34PM

Discussion article for the meetup : London Rationalish meetup, 7/2/16

WHEN: 07 February 2016 02:00:00PM (+0000)

WHERE: Shakespeare's Head, 64-68 Kingsway, London WC2B 6AH

(The new experimental name for the diaspora meetup group.)

We had around 30 people last time! (Plus at least four different meal shakes.) Go, us!

This meetup will be social discussion in a pub, with no set topic. If there's a topic you want to talk about, feel free to bring it.

The pub is the Shakespeare's Head in Holborn. There will be some way to identify us.

The event on facebook is visible even if you don't have a facebook account. Any last-minute updates will go there.

We're a fortnightly London-based meetup for members of the rationalist diaspora. The diaspora includes, but is not limited to, LessWrong, Slate Star Codex, rationalist tumblrsphere, and parts of the Effective Altruism movement.

You don't have to identify as a rationalist to attend: basically, if you think we seem like interesting people you'd like to hang out with, welcome! You are invited. You do not need to think you are clever enough, or interesting enough, or similar enough to the rest of us, to attend. You are invited.

People start showing up around two, and there are almost always people around until after six, but feel free to come and go at whatever time.

Discussion article for the meetup : London Rationalish meetup, 7/2/16

A Rationalist Guide to OkCupid

22 Jacobian 03 February 2016 08:50PM

There's a lot of data and research on what makes people successful at online dating, but I don't know anyone who actually tried to wholeheartedly apply this to themselves. I decided to be that person: I implemented lessons from data, economics, game theory and of course rationality in my profile and strategy and OkCupid. Shockingly, it worked! I got a lot of great dates, learned a ton and found the love of my life. I didn't expect dating to be my "rationalist win", but it happened.

Here's the first part of the story, I hope you'll find some useful tips and maybe a dollop of inspiration among all the silly jokes.


Does anyone know who curates the "Latest on rationality blogs" toolbar? What are the requirements to be included?


Gamify your goals: How turning your life into a game can help help you make better decisions and be more productive

10 BayesianMind 03 February 2016 10:48PM

Self-motivated hard work is the primary source of the intense, optimistic engagement known as flow—one of the greatest forms of happiness that makes us come alive with purpose and potential (Csikszentmihalyi, 1975). Sadly, for most people work does not feel so rewarding most of the time. Instead we often have to persevere through long periods of hard, painful, and unrewarding work when we could be doing something much more enjoyable. When faced with this motivational challenge people often give up too easily, get sidetracked, or procrastinate (Steel, 2007). The problem is not that we are not willing or unable to work hard. To the contrary, we crave being productively engaged in challenging tasks. Thus, instead of blaming ourselves for our limited will-power, it may be more productive to take a critical look at the carrots and the sticks that are supposed to help us stay motivated. Who put them there and why? Are these incentives helpful, distracting, irrelevant, or out of sight? If you could place them differently and add new ones, where would they go? Often, the problem is that the rewards we experience in the short run are misaligned with what we want to accomplish. In the short run the extremely valuable work that brings us closer to our cherished goals can be aversive while activities that are irrelevant or even opposed to everything we want to accomplish can be pleasant and rewarding. Hence, when we struggle to be engaged with something that we care about, then perhaps we are not the problem but the incentives are, or as Jane McGonigal (2011) put it "Reality is broken".

So, if reality is broken, then what can we do to fix it?  One approach is to design better incentive structures that make the pursuit of our goals more engaging. If we want to go this way, then there is a lot to be learned from games, because their incentive structures are so well designed that they let people enjoy hard work for many hours on end (McGonigal, 2011). In the past five years, the success of video games has inspired the gamification of education, work, health, and business. Gamification is the use of game elements, like points, levels, badges, and quests to engage, motivate, and nudge people in non-game contexts. There are even tools like SuperBetter and Habitica that individuals like you and I can use to gamify our own lives. Previous studies have shown that gamification can have positive effects on motivation, engagement, behavior, learning outcomes, and health—but only when it is done right (Hamari, Koivisto, & Sarsa, 2014; Roepke, et al., 2015). But when gamification is done wrong it can have negative effects by incentivizing counter-productive behaviors. So far gamification has been an art, and there is very little science about how to do it right. This motivated my advisor and me to develop a practical theory of optimal gamification.

In this blog post I focus on how our theory could be applied in practice. If you would like to learn about the technical details or read more about our experiments, then please take look at our CogSci paper (Lieder & Griffiths, submitted). I will start with a very brief summary of our method, provide an intuitive explanation of what it does, and then dive into how you can implement it in your own life. I will close with an outlook on how our method could be applied to gamify our todo lists.

Level 1: Optimal Gamification

Our method for optimal gamification draws on the theory of Markov decision processes (MDPs; Sutton & Barto, 1998) and the shaping theorem (Ng, Harada, & Russell, 1999). The basic idea is to align each action's immediate reward with its value in the long run. Therefore the points should complement the immediate rewards of doing something (e.g., how painful it is) by the value that it generates in the long run. Concretely, the points awarded for an activity should be chosen such that the right thing to do looks best in the short run when you combine how many points it is worth with how it feels when you do it. Furthermore, the points have to be assigned in such a way that when you undo something you lose as many points as you earned when you did it. We evaluated the effectiveness of our method in two behavioral experiments. Our first experiment demonstrated that incentive structures designed by our method can indeed help people make better, less short-sighted decisions—especially when course of action that is best in the long run is unpleasant in the short run. We also found that less principled approaches to gamification can encourage ruthless rushing towards a goal that causes more harm than good, and we showed that our method is guaranteed to avoid these perils. In the second experiment we found that the optimal incentive structures designed with our method can be effectively implemented using game elements like points and badges. These results suggest that the proposed method provides a principled way to leverage gamification to help people make better decisions.

Our method proceeds in three steps:

1.    Model the situation and the decision-maker's goals and options as a MDP.

2.    Solve the MDP to obtain the optimal value function V* or approximate it.

3.    Set the number of points for progressing from stage s to stage s' to V*(s')-V*(s).  

Intuitively, this means that the number of points that is awarded for doing something should reflect how much better the resulting state (i.e., s') is than the previous one (i.e., s). For instance, achieving a goal is worth 1000 points then completing 10% of the work required to reach the goal should be rewarded with 100 points. So let's think about how you could apply this approach right now without having to solve MDPs.

Level 2: Practical Implications

In my day-to-day life I try to approximate optimal gamification as follows:

1.    Set a concrete goal that you would like to achieve and figure out how many points it is worth, e.g. writing this blog post was worth 1000 points to me.

2.    Set several milestones along the way to the goal to divide the path into small steps that feel very manageable.

3.    For each milestone, determine how far you will have come when you get there as a percentage of the total distance to the goal, e.g. 10%, 20%, 30%, ..., 100% for the first, second, third, ..., and the tenth milestone respectively.

4.    Assign each milestone the corresponding fraction of the total value of achieving the goal, e.g. 100 points, 200 points, 300 points, ..., and 1000 points for the first, second, third, ..., and tenth milestone respectively.

5.    Figure out what you have to do to get from one milestone to the next. If this is a simple activity, then its reward should be the difference between the value of next milestone and the value of the current milestone, e.g. 100 points. If it is a complex sequence of actions, then make it a subgoal and apply steps 1-3 figure out how to achieve it.

6.    Once you are done with step 5, you can add those points to your todo-list.

7.    Now it is time to get things done and reward yourself. You start at 0 points, but whenever you complete one of the steps, you earn as many points as you have assigned to it and can increment your (daily) score.

Earning these points can be very rewarding if you remind yourself what they stand for. If your goal was worth $1,000,000 to you and you assigned 1000 points to it, then 10 points should be worth $10,000 to you. But if this is not rewarding enough for you, you can think of ways that make the points more pleasurable. You could, for instance, make a high-score list that motivates you to beat your personal best day after day or start a high-score competition with your friends. You could also set yourself the goal to achieve a certain number of points by a certain time and promise yourself a treat if you achieve it.

There are many other ways that you could assign points to the items on you todo list. Feel free to do whatever works for you. But it may be useful to keep in mind that the way in which optimal gamification assigns points has several formal properties that are necessary to avoid negative side-effects:

a) Each item's score reflects how valuable is in the long run.

Optimal gamification works because it aligns each action's immediate reward with its long-term value. To help you make better decisions the points should be designed such that the course of action that is best in the long run looks best in the short run. This entails incentivizing unpleasant or unrewarding activities that will pay off later—especially when their less productive alternatives are very rewarding in the short run.

b) Beware of cycles!

The shaping theorem (Ng, et al., 1999) requires that going back and forth between two states receives a net pseudo-reward of zero. When your pseudo-rewards along a circle add up to a positive value, then you may be incentivizing yourself to create unnecessary problems for yourself. This can happen when the action for which you reward yourself can only be executed in an undesirable state, and you do not equally punish yourself for falling back into that state. For instance adding points for losing weight will inadvertently incentivize you to regain weight afterwards unless you subtract at least the same number of points for gaining weight. Similarly, if you reward yourself for solving interpersonal conflicts but don’t punish yourself for creating them, then you may be setting yourself up for trouble. To avoid such problems, creating a problem must be punished by at least as many points as you earn by solving it. 

c) Two ways to achieve the same goal should yield the same number of points.

The shaping theorem also requires that all paths that lead to the same final state (e.g., having submitted a paper by the deadline) should yield the same amount of reward. If this is not the case your pseudo-rewards may bias you towards a suboptimal path. For instance, if you reward your all-nighter on the last night before the deadline by the reward value of a month’s worth of work, you are incentivizing yourself to procrastinate. Similarly, if you reward one activity that leads towards your goal much more heavily than others, then you may be biasing yourself towards a reckless course of action that may achieve the goal at an unreasonably high cost. For instance, rewarding yourself 100 times as much for working 100% on a project than for working on it 50% might lead you to complete the project early at the expense of your health, your friendships, your education, and all your other projects. To avoid this problem, al paths that lead to the same state should yield the same amount of reward. 

d) Pseudo-rewards should be awarded for state-transitions instead of actions.

Many applications of gamification reward "good" actions with points regardless of when or how often these actions are taken. But according to the shaping theorem, the number of points must depend on the state in which the action is taken and the state that it leads to. If your pseudo-rewards were based only on what you do but not on when you do it, then you might keep rewarding yourself for something even when it is no longer valuable, because the underlying state has changed. For instance, at some point your reward for losing weight has to diminish or else you may be setting yourself up for anorexia.

Level 3: Todo-list gamification

Todo list gamification

My first practical application is to manually gamify my todo-list every morning. I find this very helpful and motivating: Assigning points to the items on my todo list makes me realize how much I value them. This is useful for prioritizing important task. Earning points allows me to perceive my progress more more accurately and more vividly. This helps me feel great about getting something important done even when it was only a single item on my todo list and took me a lot of time and effort to accomplish. Conversely, the point scheme also prevents me from feeling so good about checking off small things that I become tempted to neglect the big ones that are much more important. Gamification thereby remedies the todo list's shortcoming that it makes each item seem equally important. I highly recommend gamifying your todo lists. It can be highly motivating. Yet, adding the points manually takes some effort and my point scheme is often somewhat arbitrary and probably suboptimal.

To make todo list gamification easier and more effective, I am planning to develop an easy-to-use website or app that will do optimal gamification for you. Its graphical user interface would allow you to create hierarchical todo-lists, ask you 1 or 2 simple questions about each item on your list and then gamify your todo-list for you. To do this, it will translate your list and your answers into a MDP, compute its optimal value function, and use it to determine how valuable it is to complete each item. The tool could also help you set manageable subgoals and determine what is most important and should be done first. Last but not least, a website or app can also leverage additional game elements to make the points that you earn more rewarding: It can track your productivity and provide instant feedback that makes your progress more salient. It can send you on a quest that gives you a goals along with small actionable steps. The tool could allow you to realize that you are getting ever more productive by visualizing your progress over time. As you become more effective, you level up and your quests will become increasingly more challenging.  It might include a scoreboard that lets you compete with yourself and/or others and win prizes for your performance. Last but not least, if you need an extra push, you can tie your points to social rewards, your favorite treat, money, or access to your favorite music, apps, or websites. There are many more possibilities, and I invite you to think about it and share your ideas. In brief, there is wealth of opportunities to leverage game elements to make goal achievement fun and easy.

Join me on my quest! An adventure awaits. 

Gamification can be a useful tool to make achieving your goals easier and more engaging. However, gamification only works when it is done right. The theory of MDPs and pseudo-rewards provide the formal tools needed to do gamification right. With the help of these tools we can design incentive structures that help people overcome motivational obstacles, do the right thing and achieve their goals. But more research and development needs to be done to make optimal gamification practical.

If you have any thoughts or ideas for what to do next, noticed a problem with the approach, or would like to be part of our team and contribute to building a tool helps people achieve their goals, please send me an e-mail.

References and recommended readings

Csikszentmihalyi, M. (1975). Beyond boredom and anxiety: the experience of play in work and games. San Francisco: Jossey-Bass.

Lieder, F., & Griffiths, T.L. (submitted). Helping people make better decisions using optimal gamification. CogSci 2016. [Manuscript]

McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. New York: Penguin.

McGonigal, J. (2015). SuperBetter: A revolutionary approach to getting stronger, happier, braver and more resilient–powered by the science of games. London, UK: Penguin Press.

Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does gamification work?–A literature review of empirical studies on gamification. In 47th Hawaii international conference on system sciences (pp. 3025–3034).

Ng, A. Y., Harada, D., & Russell, S. (1999). Policy invariance under reward transformations: Theory and application to reward shaping. In I. Bratko & S. Dzeroski (Eds.), Proceedings of the 16th annual international conference on machine learning (Vol. 16, pp. 278–287). San Francisco, CA, USA: Morgan Kaufmann.

Roepke, A. M., Jaffee, S. R., Riffle, O. M., McGonigal, J., Broome, R., & Maxwell, B. (2015). Randomized controlled trial of SuperBetter, a smartphone-based/Internet-based self-help tool to reduce depressive symptoms. Games for health journal4(3), 235-246.

Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA, USA: MIT press.


View more: Next