Much of this material is sourced/summarized from Deliberate Performance: Accelerating Expertise in Natural Settings and “The Power of Intuition”. Both contain much more than what is written here. An earlier draft/version of this was posted as "Productive Use of Heuristics and Biases". Based on feedback in the comments, much has changed. Feedback is appreciated, as are any other ideas on how deliberate practice can be applied.

How does someone incorporate something like deliberate practice into a typical job and everyday life? Deliberate practice is meant to be challenging, and because of this, it is draining. Much of the expertise literature describes a limit of a few hours of deliberate practice per day. If you can sustain more than that, you probably aren’t doing it right.

While the amount you can put in per day is limited, it still takes many hours of this intense practice, so reaching expert performance in a domain takes anywhere from years to decades. Deliberate performance is a concept related to deliberate practice, but perhaps more effective and efficient. Rather than separating practice and performance, the idea is to overlap the two as much as possible. The primary aim is to accelerate the development of expertise, while also improving the productivity of practice. Since you are practicing using normal tasks, you don’t have to set as much time aside where you aren’t working.

Deliberate performance is readily applicable to decision making. You make decisions as you normally would but also record your expectations and thinking behind the decision. What do you think will happen and why? How do you feel about the decision? When you have feedback on how the decision turned out, you can go back to see what you were thinking and how well your expectations matched reality. The idea of recording this information has been recommended by Peter Drucker, and more recently by Daniel Kahneman (Kahneman sees it as a way to reduce hindsight bias).

The term deliberate performance comes from the paper Deliberate Performance: Accelerating Expertise in Natural Settings, where the concept is elaborated into four exercises:

Estimate
One of the simplest ways to do this is to estimate how long a task you have will take. Accurately predicting how long things will take is a valuable skill on its own, but in estimating it, you form a mental model of the task and where the obstacles lie. When your estimate differs from reality, you can go back and see where you went wrong.

Experiment
The simplest way to experiment is just to try something and see what happens. This can be helpful in early stages when you are exploring, but typically, a better way is explicitly state what you expect to occur. As in estimation, if the actual results aren’t what you expect, you can update your mental models. It is usually a good idea to make your experiments cheap, in the sense that you expect failure and change to occur. Steven Spear, in his studies of Toyota, described cheap experiments as preferring bolting to welding, clamping to bolting, taping to clamping, and holding to taping. The more rapidly you can experiment, the more rapidly you can get feedback and update your models.

Experimentation is sort of where the rubber meets the road in deliberate performance. Where the other exercises are more focused on thought, experimentation is more about action. You see how much your mental models match the real world.

Explain
Why did things happen as they did? Try to explain from the available evidence. Explanation gives you an opportunity to make sense of why things turned out the way they did. Whether or not things went as you expected, try to explain why and how it happened. You can also seek out explanations from others, but you should probably form your own first so you can compare.

Extrapolate
What do you know now that you can use to reason about something you don’t know or something that doesn’t yet exist? What do you already know that you can use to solve a new problem? Using induction in proofs is an example of this, as is analogical reasoning. Klein describes an example of the latter in “Sources of Power”. Engineers are trying to figure out how much parts of new airplane designs will cost, but the parts are still being developed. The engineers end up looking at existing parts for analogs. None of the analogs are perfect matches, so they extrapolate the characteristics of multiple parts to create a new “Frankenstein” part.

Feedback and Coaching
Unlike deliberate practice, where the role of a coach is more formal and explicit, deliberate performance has the more realistic expectation that you will not have a coach. Instead, you try to make the most of the feedback you can get. Rather than a coach pointing out mistakes and what you can improve, you find errors in your mental models and ask effective questions. The 4Es will likely prompt some questions, while also giving you some knowledge and experience to make sense of the answers. While the exercises probably sound simplistic, something important happens as you do them: you are forced to form mental models, compare them to reality, and adjust them as needed.

Another useful method for getting feedback is to predict or emulate what an expert would do. When things don’t go as expected, you can then explain what you did and your reasoning to the expert and get feedback. As you find and correct mistakes in your mental models, your “expert emulation” should improve. Seeking explanations to compare to your own is also an example of this.

Using the Exercises
The first example, keeping a decision journal and reflecting on how things turns out, demonstrates that the 4Es can be combined and work well together. In fact, that is probably the best way to use them, looping through them multiple times. Similar to the scientific method, start with an estimate or hypothesis. Then, experiment to see if your hypothesis is correct. Next, attempt to explain what you saw in the results of the experiment in relation to your hypothesis. You’d then finish the first loop by extrapolating from the explanation, which can provide a new hypothesis to test.

One other method that combines multiple exercises is to do what Klein calls a premortem. If you have some large project, start by imagining that what you are trying to do has been a complete failure, and you are now doing a post mortem to understand what went wrong. Assuming things have gone wrong, why did they go wrong? The reasoning behind this technique is that people don’t want something to fail (or look like they want it to fail), but assuming it already has failed reduces that bias. In a sense, this is similar to red teams but easier to implement and less resource intensive. This gives you a chance to extrapolate and estimate, then form an explanation for why you think things might go wrong.

A few more exercises specific to decision making
In “The Power of Intuition”, Klein advocates identifying decisions where problems have occurred, forming exercises based on those decisions, playing through the exercises, and then critiquing your decision making process. The idea is that the most difficult scenarios occur rarely enough that developing expertise in them is impossible without some intervention, they occur too rarely to gain significant experience.

First, identify a decision. When reviewing the decisions, note what makes it difficult, what kinds of errors are often made, how an expert might approach it differently than a novice, and how the decision can be practiced and how you can get feedback.

Identified decisions can then be turned into scenarios which can be repeatedly practiced (typically in groups). Start with describing the events that led to the decision. The players are then told what they are trying to achieve, the context, and the constraints. Try to include a visual representation whenever possible. Even if you don’t work through the exercise, forming it can be useful in and of itself.

After the exercise, critique the decision and the process used to make it. Start with a timeline and identify key judgments. For each of the key judgments, note why it was difficult, how you were interpreting the situation, what cues/patterns you should have been picking up, why you chose to do what you did, and what you would’ve done differently with the benefit of hindsight.

The main point
If we are naturally biased and limited in how rational our decision making can be, it might be more effective to improve intuition and manage biases rather than trying to completely eliminate them and focus on unbiased thinking. I’m not advocating irrationality, instead I’m interested in how to deal with bounded rationality. How can we improve our ability to satisfice? How can we productively use and develop our intuition?

Caveats
While I intend for this post to be informative, it may not be, and I suspect I’ll learn more from writing it and feedback than others will get from reading it. There is much on LessWrong I haven’t read, so I apologize if this is something that was discussed extensively before. If that is the case, I’d appreciate links to those discussions in the comments. One other note, I realize intuition can be interpreted as a dirty word, with good reason. Here, please interpret intuition as a rational heuristic guided by experience. Heuristics are tools, they can be effective when used properly.

Background
This post was prompted by some casual research on expertise and decisionmaking I've been doing for a couple months now. Along the way I came across the work of Herbert Simon and Gary Klein. Simon’s ideas of bounded rationality and satisficing have come up in discussion here before, but I haven’t seen any discussion of Klein, who is an active researcher in the area of intuitive decisionmaking. Simon proposes that rationality in decisionmaking is limited by the constraints of time, available information, and brain/computational capabilities (bounded rationality). Rather than make perfectly rational decisions, we make the best decisions we can given those constraints (satisficing). Klein’s work is primarily focused on how experts actually make decisions in the real world. Put another way, his work is focused on how experts satisfice.

One of Klein’s early research projects was to help the military improve decision making. His proposal was to study firefighters, figuring their experience would be a useful analog to military experience (high pressure, life/death, time...). The major finding was that they did not follow the standard model of good decision making, they did not evaluate multiple options. Instead, they believed they operated by feel. Generally, the firefighters had one option they were considering. After further dialogue, Klein realized that they were forming mental models of the situations based on recognition/experience, which he calls recognition primed decision making. Using those mental models, they would mentally simulate what might occur if they followed their first instinct and go from there. A similar process was seen in other areas of expertise as well. For example, expert chess players aren’t evaluating more options or seeing farther ahead than novices, they just have better recognition/heuristics that are developed through experience. One potential problem with using chess as an analog for other skills is that the rules are clearly defined, all of the information you need is on the board (or in your opponent’s head...). The real world isn’t quite so clean, which which might imply that rationality is generally more limited/bounded than in the case of chess.

From what I’ve been reading, I guess I’m skeptical about efforts to be more rational in everyday life. Certainly it is possible, but it may be unnatural enough to be impractical. Even then, it will still be bounded. If this is the case, it might be more effective to train in some mix of intuitive and rational decisionmaking, while understanding the limits of both. While I haven’t read it yet, I think Daniel Kahneman refers to intuition and rationality as system 1 and 2 in “Thinking Fast and Slow”. While Kahneman has seemed skeptical of our ability to improve decision making and reduce bias in the interviews I’ve read, Klein is a bit more optimistic.

Methods for improving intuition I’ve found so far
(1-5 from Klein/Fadde - Deliberate Performance, 6-7 from Klein - The Power of Intuition)
Intuition comes from experience/expertise. Expertise is developed through deliberate practice. The methods here are intended to accelerate the learning process.

1. Estimation
This is fairly simple, estimate something and compare to reality. How long do you expect a project to take? Think through it and discuss with others, and record your estimate and reasoning. Later, after you’ve completed the project, go back and look at your prediction. Where did you go wrong?

One of the keys to effective and efficient learning is timely feedback. If your project will last months, you’ll probably want to do other estimates in the meantime, preferably with tighter feedback loops. An example given in the paper is giving a presentation. How long do you expect discussion to take on each point? What questions will be asked? After the presentation, compare the prediction to reality. For what it’s worth, I believe Peter Drucker also recommends something similar in his article “Managing Oneself”. If I remember correctly, Drucker recommends keeping a decision journal, where you record the decisions you make and return later to see what you got right or wrong (and why).

2. Extrapolation
In “Sources of Power”, Klein discusses an example of extrapolation. Engineers are trying to estimate the cost and time it takes to build various components of airplanes that haven’t yet been built. Since the parts don’t yet exist in some cases, they need to find a way to provide reasonable estimates. To do this, they try to find an analog. Are there any parts that are similar? How long did they take to produce and how much did they cost? It may take combining several analogs to come up with an estimate. The idea is to use what you have and know to model what you don’t have and know. Again, compare the extrapolation to the actual results when you are able to do so, so you can improve next time.

3. Experimentation
At its most basic, experimentation can be just trying something and seeing what happens. A better way is to form a hypothesis (estimation/prediction/extrapolation) prior to trying.This promotes better learning, because you have a concrete expectation that is either verified or not. If not, you’ve learned something new. Perhaps you’ll even be surprised, and surprises can lead you in new directions.

A more personal example: For the longest time, I had trouble with directions where I currently live. I’d look at a map, figure out how to get somewhere, and try not to vary things too much so I wouldn’t get lost. I rapidly improved by using a GPS, and ignoring it until I needed it. I’d have an idea of how to get there (from looking at maps), but I wouldn’t have it down 100% or I’d try to improvise. The GPS allowed me to experiment without the risk of getting lost, and it provided instant feedback.

4. Explanation
This section is rather short, so I am not entirely sure if I understand it correctly. I think the intent is to try to make sense of what you’ve seen and learned from the other three Es.

One way to do this might be to use what Scott Young calls the Feynman Technique (Learn Faster with the Feynman Technique):

Step 1. Choose the concept you want to understand.

Take a blank piece of paper and write that concept at the top of the page.

Step 2. Pretend you’re teaching the idea to someone else.

Write out an explanation of the topic, as if you were trying to teach it to a new student. When you explain the idea this way you get a better idea of what you understand and where you might have some gaps.

Step 3. If you get stuck, go back to the book.

Whenever you get stuck, go back to the source material and re-learn that part of the material until you get it enough that you can explain it on paper.

Step 4. Simplify your language.

The goal is to use your words, not the words of the source material. If your explanation is wordy or confusing, that’s an indication that you might not understand the idea as well as you thought – try to simplify the language or create an analogy to better understand it.

5. Feedback/Coaching (Emulation?)
Feedback is critical. While you might not have a coach at work, you can still find someone to emulate. Assuming they are available for questions, you can try to predict what they would do in a situation. When something does not go as you’d expect, explain your thinking and ask for feedback. In my experience, other people are busy with their own work, so coaching/mentorship takes a backseat to more urgent matters. Plus, some people often just don’t want to be bothered. In this case, I think the best thing to do is to get good at asking effective questions and be well prepared.

6. Premortem
Imagine results of what you are trying to do are a complete failure and that you are now doing a post mortem to understand what went wrong. What went wrong? The reasoning behind this technique is that people don’t want something to fail (or look like they want it to fail), assuming it already has failed reduces that bias. Apparently the idea for the technique came from Mitchell, Russo, and Pennington - “Back to the Future: Temporal Perspective in the Explanation of Events”. That paper doesn’t entirely support the technique, stating that it produces more reasons that are episodic, but not necessarily better (they were unable to judge the value of the reasons given). Klein uses this technique regularly in meetings, the general impression is that it reduces confidence in the plan, as intended. From there, they try to prepare for the potential problems.

In a sense, this is similar to red teams but easier to implement and less resource intensive.

7. Identify Decisions/Decision Making Exercises/Decision Making Critiques
In “The Power of Intuition”, Klein advocates identifying decisions where problems have occurred. When reviewing the decisions, note what makes it difficult, what kinds of errors are often made, how an expert might approach it differently than a novice, and how the decision can be practiced and you can get feedback.

Those decisions are then turned into scenarios which can be repeatedly practiced (typically in groups). Start with describing the events that led to the decision. The players are then told what they are trying to achieve, the context, and the constraints. Try to include a visual representation whenever possible.

After the exercise, critique the decision and the process used to make it. Start with a timeline and identify key judgments. For each of the key judgments, note why it was difficult, how you were interpreting the situation, what cues/patterns you should have been picking up, why you chose to do what you did, and what you would’ve done differently with the benefit of hindsight.

Concluding thoughts
Is there interest in this topic on LW? I’m not denying that relying on intuition alone can be dangerous, but I am very skeptical that focusing on reducing bias alone will lead to better decisions. In some cases, it may be better to admit that biases are affecting a decision. One other thing to note, is that bias and mistakes are inevitable. It seems a lot of the LW rationality posts I’ve seen focus on reducing or eliminating these mistakes. That is certainly a valid goal (at least the reduction is), but it isn’t enough. Choice/information overload can affect decisions, as can blood sugar levels (Sweet Future: Fluctuating Blood Glucose Levels May Affect Decision Making) and having to go to the bathroom (Tuk M. et al. (2011). Inhibitory Spillover: Increased Urination Urgency Facilitates Impulse Control in Unrelated Domains. Psychological Science.).

Mistakes will happen, so you’ll have to do your best to learn from them and reduce their cost/make recovery easier. At the same time, good heuristics give you a better starting point to apply the rationality techniques. They are complementary. Worry about reducing bias after you have come up with something using expertise/intuition.

Again, this is a draft, written mostly from memory of what I’ve been reading. The primary sources were “Sources of Power” and “The Power of Intuition” (both by Klein), “The Cambridge Handbook of Expertise”, and scattered readings on/by Simon. Also of interest are this interview with Daniel Kahneman and Gary Klein (free registration required unfortunately) and this paper on deliberate performance by Peter Fadde and Klein.