Note: This post is part of my series of posts on forecasting and related topics, related to my contract work for MIRI. While I think the post would interest a sufficiently large fraction of the LessWrong readership for it to be worth posting, it's not of as much general interest as some of my other recent posts. If business strategy and planning don't interest you as topics, this post may not be for you.
I've been reviewing forecasting and related domains as part of contract work for MIRI. One of the related domains I'm looking at is scenario planning (Wikipedia), also known as scenario thinking or scenario analysis. To understand scenario planning, I read the book The Art of the Long View by Peter Schwartz (Amazon) (a fairly interesting read, though whether you consider it value-for-money depends on whether strategy planning and thinking about the future as a whole interest you) and his follow-up book Learnings from the Long View (Amazon). I also skimmed Scenario Planning in Organizations by Thomas J. Chermack (Amazon), which is somewhat drier but is more useful for people who have some understanding of scenario planning and want to implement it in full.
In this post, I discuss some aspects of scenario planning, but the post is not intended to be a standalone summary of scenario planning. For that purpose, I recommend reading the overviews of scenario planning listed below. In this post, I'll look at examples of scenario planning in diverse domains, and I'll consider the relationship between scenario planning and the more conventional, quantitative approach to forecasting.
For longer overviews, consider the books listed above.
Key people
Herman Kahn (Wikipedia): He pioneered the use of scenario planning in the United States military and at the RAND Corporation, and he founded the Hudson Institute.
Pierre Wack (Wikipedia): He spearheaded the use of scenario planning by Royal Dutch Shell in the early 1970s. This is claimed to have helped the company cope better with the OPEC oil shock and environmentalism than its competitors did. Shell was the first private company to use scenario planning in a big, systematic way.
Arie de Geus (Wikipedia): He was the head of Shell Oil Company's Strategic Planning Group. He studied the history of scenario planning and found that its main utility arose from the decision-making processes following the scenario generation, not the scenarios themselves.
Peter Schwartz (Wikipedia): He is the author of the books The Art of the Long View and Learnings from the Long View that describe scenario planning in detail, and also a co-founder of the Global Business Network (Wikipedia), a leading organization offering consulting and training services in scenario planning.
Paul J. H. Schoemaker (Wikipedia): He is an academic studying strategic management and decision-making. He has devoted some research efforts to understanding scenario planning.
A quick summary of scenario planning
Scenario planning involves the generation of (usually two or three) scenarios for how the future might transpire. Here are some typical aspects of scenario planning:
Historically, scenario planning has not been used to replace rigorous, quantitative forecasting. Rather, scenario planning in the corporate world arrived with the intention of replacing a mode of strategic planning where the whole company was supposed to believe in and bet upon a single Official Future. Scenario planning helped introduce the possibility of coping with uncertainty, through a route somewhat different from probabilistic forecasting. (See the last point, though).
According to some scenario planning guidelines, including guidelines offered by Schwartz and Chermack, as well as a suggestion in Megamistakes (a book I blogged about a while back), the scenarios should be chosen to have approximately equal probability. At any rate, the probabilities do not differ by an order of magnitude. So, for instance, it's okay to take scenarios with probabilities of 25%, 35%, and 40%, but not okay to consider scenarios with probabilities of 1%, 10%, and 89%. It's unclear how hard-and-fast this rule is.
Each scenario is described in considerable detail, along with early indicators that that is the relevant scenario. The idea is that, if we observe the early indicators of a particular scenario, we have higher confidence in that scenario than in the other scenarios.
Ultra-optimistic and ultra-pessimistic scenarios are avoided. Even scenarios that unfold in optimistic ways or pessimistic ways usually incorporate some elements of pushback against the optimism or pessimism, as we might expect in real life.
Schwartz describes scenario planning as combining predetermined elements and critical uncertainties. The predetermined elements are what allow us to build a storyline with reasonable confidence, without having to guess at every step. The critical uncertainties are what we vary between the different scenarios.
The consideration of scenarios happens by looking at the conjunction of Social, Technological, Environmental, Economic, and Political (STEEP) factors, according to Schwartz. Other authors have used the acronym PESTEL for political, economic, socio-cultural, technological, environmental, and legal factors (see the Wikipedia page on environmental scanning). These factors may generate both predetermined elements and critical uncertainties for the scenario planning exercise.
The point in time at which the scenarios diverge needs to be chosen carefully. In most real-world situations, the very near future can be forecast to a crude approximation with reasonable confidence. The divergence into scenarios is generally done from the point at which a clear forecast starts becoming difficult to build.
One way of thinking of the above is in terms of trend analysis versus emerging issues analysis (Wikipedia). Trend analysis focuses on the trend (pattern of change) for the things that already exist and that we can observe and (roughly) measure or ballpark. Trend analysis is the domain where quantitative forecasting methods are more useful: we already have data from the past and we can extend that somewhat into the future. But some people, companies, ideas, memes, fashions, etc. that have near-zero influence today may emerge a few years down the line. These are sometimes called emerging issues and the identification of these is called emerging issues analysis. The point of divergence of the scenarios would therefore be as far in the future as it might take for an emerging issue to become a trend. In fast-changing and highly unpredictable domains, the scenarios may start diverging right away, because new issues can emerge anytime. In relatively stable domains, we would expect that for an issue to emerge and become a trend, it would take a few years.
Usually, what transpires in the real world is not any one particular scenario, but a mix of elements from different scenarios.
The utility of scenario analysis is not merely in listing a scenario that will transpire, or a collection of scenarios a combination of which will transpire. The utility is in how it prepares the people undertaking the exercise for the relevant futures. One way it could so prepare them is if the early indicators of the scenarios are correctly chosen and, upon observing them, people are able to identify what scenario they're in and take the appropriate measures quickly. Another way is by identifying some features that are common to all scenarios, though the details of the feature may differ by scenario. We can therefore have higher confidence in these common features and can make plans that rely on them.
The review article The origins and evolution of scenario techniques in long range business planning by Bradfield, linked from the overview section, identifies two schools of scenario planning: the Intuitive Logics school pioneered by Shell, and described by Schwartz and Chermack, and the probabilistic school pioneered by the RAND Corporation. The article notes that while the intuitive logics approach is the approach more commonly associated with scenario planning, the probabilistic school has also developed considerably. Indeed, some of the scenario planning examples we list below belong to the probabilist school.
Who uses scenario planning?
Scenario planning caught on after the 1973 oil shock; prior to that, only Shell and GE engaged in it (see the Bradfield et al. overview for more). A 1981 survey by Linneman and Klein found three predictors of whether a company used scenario planning:
The size of the company: Larger companies were more likely to use scenario planning. Relatedly, in The Art of the Long View, Schwartz says that both large and small companies can benefit from scenario planning, but in different ways. Small companies need to use scenario planning mainly to get a sense for whether their overall business model will remain viable, whereas large companies need to make specific quantitative decisions, such as determine how much to invest in a particular product line. Larger companies therefore need to develop more detailed models, whereas for small companies, scenario planning serves to augment one's gut feel.
The length of the planning horizon: Companies that planned for longer horizons were more likely to use scenario planning. This is consistent with the above observation that scenario planning starts becoming useful when the time horizon is long enough for new issues and players to emerge and for a tight quantitative forecast to therefore be impossible.
The capital-intensivity of the industry: More capital-intensive industries are more likely to use scenario planning. This can be explained by the more long-term nature of capital investments and the difficulty of reallocating such investments quickly. In order to determine whether to make a big investment (such as an oil well or a factory) it's important to get a handle on the different possible scenarios and how profitable the investment would be in each.
Big successes of scenario planning
As noted above, Royal Dutch Shell used scenario planning in the 1970s and this appears to have given the company an advantage over its competitors in coping with the OPEC oil shock and environmentalism. In the 1980s, scenario planners at Shell considered the possibility of the collapse of the Soviet Union when they were brainstorming ways that the price of oil might go down. Schwartz, who describes this in Art of the Long View, writes that at the time, European countries capped their Russian imports at 35% (for political, Cold War-related, reasons), keeping the price of oil high enough to make investing in some expensive oil wells worthwhile. But if the Soviet Union collapsed and the politically motivated caps were removed, then the price would fall, and the oil wells would become unprofitable. The team at Shell identified the possibility that the Soviet Union economy would collapse and that Gorbachev might lead a new country. They also surmised that even if the new economy didn't do well, it would become a corrupt crony-capitalist system and would not return to Leninism. With these scenarios in mind (even though they were not forecast as highly likely) Shell invested in fewer oil wells, and made more investment in technology that would keep the price of oil extraction low enough to keep the oil wells profitable even after a price collapse.
In Art of the Long View and Learnings from the Long View, Peter Schwartz discusses a number of other examples of scenario planning. One example is an advertising firm in the early 1990s that is exposed to a scenario of broadband Internet and starts preparing for that scenario.
Some examples of scenario planning
I looked on the Internet for scenario planning writeups across diverse domains that were publicly available. I list some examples below. The focus is on what I was able to find on the Internet by using a range of search phrases, and the list below is unlikely to be representative of scenario planning.
The book Learnings from the Long View by Schwartz reviews his scenario analyses from his earlier book The Art of the Long View. The Kindle edition of Learnings is priced at $2.99, so it might be worth buying even if you don't want to buy the earlier (and longer) book.
The Global Scenario Group (website, Wikipedia) is a futures studies/scenario analysis group that publishes scenarios for the future. A number of papers with scenario analyses by them can be downloaded here.
The Global Business Network (website (link not working), Wikipedia) does scenario planning for private sector companies, and also trains them in scenario analysis. The URL for their website (as listed on LinkedIn and Wikipedia) isn't working, and I don't know exactly what the situation with them is.
Many of the reports published by the McKinsey Global Institute (website) or by other parts of McKinsey & Company use scenario analysis to explore different possibilities. For instance, here is a scenario analysis on Moore's Law from McKinsey & Company. Other consulting companies also often use scenario analyses in their reports.
Energy: Another major (and closely related) area where scenario analysis is common is energy demand and supply. Because of how intricately energy is interwoven in the modern economy, creating scenarios for energy often requires creating scenarios for many aspects of the future. Shell (the organization to pioneer scenario analysis for the private sector, as described by Schwartz in his book) publishes some of its scenario analyses online at the Future Energy Scenarios page. While the understanding of future energy demand and supply is a driving force for the scenario analyses, they cover a wide range of aspects of society (the STEEP or PESTEL list). For instance, the New Lens Scenario published in 2012 described two candidate futures for how the world might unfold till 2100, a "Mountains" future where governments played a major role and coordinated to solve global crises, and an "Oceans" future that was more decentralized and market-driven. (For a critique of Shell's scenario planning, see here). Shell competitor BP also publishes an Energy Outlook that is structured more as a forecast than as a scenario analysis, but does briefly consider alternative assumptions in a fashion similar to scenario analysis.
Macroeconomic and fiscal analysis: Budget projections made by the Congressional Budget Office (CBO) in the United States consider multiple scenarios along two dimensions: fiscal policy (tax and revenue) and other sources of varuation in economic growth. CBO projections are typically made for the scoring and analysis of specific fiscal proposals made by members of Congress. For instance, here are the CBO projections made in light of Congressman Paul Ryan's proposals. Scenario analysis is also a common tool in macroeconomic analysis. For instance, Moody's U.S. Macroeconomic Outlook Alternative Scenarios used scenario analysis to understand what awaits the United States economy. Unlike most scenario analyses, Moody's specified, for each scenario, its estimated numerical probability that reality would be better than the scenario. But the scenario analysis didn't hinge on believing the probability estimates.
Land use and transportation analysis: The Federal Highway Administration in the United States uses scenario planning to understand different possibilities for future patterns of land use and transportation. See the scenario planning section of their website.
Analysis of the technology sector: In 1986, Monthly Labor Review ran an article titled Computer manufacturing enters a new era of growth with a scenario analysis of different outcomes for the then nascent tech sector and its implications for productivity and employment. The article considered nine scenarios, obtained by combining three scenarios for the level of technological progress with three scenarios for the level of overall economic growth. In 2009, Leva et al. wrote a scenario analysis for the future of the Internet. They described four scenarios: (A) Wild and Free, (B) Isolated Walled Gardens, (C) Content-driven Overlays, (D) Device-Content Bundles. Also, here is a scenario analysis on Moore's Law from McKinsey & Company. Other consulting companies also often use scenario analyses in their reports.
The future of medicine: Here is a scenario analysis considering four scenarios for the future of medicine: Sobriety in sufficiency, risk avoidance, technology on demand, and free market unfettered.
Does scenario planning involve forecasting?
There are many ways in which scenario planning overlaps with forecasting:
The need to choose scenarios that have approximately equal probability, at least up to an order of magnitude, suggests that a scenario planning exercise implicitly includes a probability estimation exercise, even though the probability estimates are very imprecise. Note that the probabilistic school of scenario planning involves explicit probability estimates. Either way, scenario planning does involve making statements about what types of futures are likely and how likely they are.
The clustering together of different sets of events into scenarios is an implicit conditional forecast. It says that if some events in the scenario occur, the scenario as a whole is likely to occur, and in particular, other events in the scenario are likely to occur. In particular, the identification of early indicators for each scenario is tantamount to giving rules for how to forecast the future by keeping an eye out for the early indicators.
The scenarios in scenario analysis generally start diverging a little while after the present. The time period from the present to the point of divergence of the scenarios is the time period where we are essentially making a relatively tight forecast. The techniques used to determine what happens till the point of divergence are standard quantitative forecasting techniques.
Even after the scenarios diverge, quantitative forecasting techniques may be used to fill in the relevant numbers in the various scenarios.
Evaluation of the utility of scenario planning (some random thoughts)
I wrote above: "The utility [of scenario planning] is in how it prepares the people undertaking the exercise for the relevant futures. One way it could so prepare them is if the early indicators of the scenarios are correctly chosen and, upon observing them, people are able to identify what scenario they're in and take the appropriate measures quickly."
What does research say about scenario planning? My impression is that there is very limited research, and it tends to be positive about the effect of scenario planning on long-range development, but there is considerable uncertainty. Bradfield et al. quote Schnaars as saying that there is "a small body of research based on empirical studies of related topics, which ‘offer some evidence as to the value of scenarios’ as a long range planning tool."
In general, evaluating scenario planning seems difficult, because unlike the case of forecasting, where the accuracy of the forecast is a good first proxy for utility, scenario planning does not lend itself to that sort of evaluation. The examples that Schwartz lists provide some anecdotal evidence. But there is the obvious selection versus treatment issue: maybe the pioneers of scenario planning were selected as the sort of people who could plan better for the future, and the scenario planning exercise itself wasn't useful. The selection versus treatment issue could be resolved by looking at whether business planning has become more efficient on the whole as scenario planning has come to be more widely used (because the continued spread of scenario planning is probably a treatment rather than a selection effect). But isolating scenario planning as a causal factor, when so many other aspects of business strategy are changing, is hard.
Another possible approach: how different would the world be today if people didn't use scenario planning? The IPCC reports on climate change may just consider one "official forecast" with a margin of uncertainty. Energy companies might still have an Official Future (again with an expressed margin of uncertainty, but without a discrete set of alternative futures). Based on my (non-expert) assessment, it seems that the quality of business strategy and policy insight in such a world would be worse than in the current world. But perhaps if scenario planning hadn't been developed, other ways of thinking about the future would have caught on more. The upshot is that I tentatively think scenario planning has been useful, but I don't see a clear way of demonstrating this in a scientifically rigorous manner, given the nature of the beast.
Note: This post is part of my series of posts on forecasting and related topics, related to my contract work for MIRI. While I think the post would interest a sufficiently large fraction of the LessWrong readership for it to be worth posting, it's not of as much general interest as some of my other recent posts. If business strategy and planning don't interest you as topics, this post may not be for you.
I've been reviewing forecasting and related domains as part of contract work for MIRI. One of the related domains I'm looking at is scenario planning (Wikipedia), also known as scenario thinking or scenario analysis. To understand scenario planning, I read the book The Art of the Long View by Peter Schwartz (Amazon) (a fairly interesting read, though whether you consider it value-for-money depends on whether strategy planning and thinking about the future as a whole interest you) and his follow-up book Learnings from the Long View (Amazon). I also skimmed Scenario Planning in Organizations by Thomas J. Chermack (Amazon), which is somewhat drier but is more useful for people who have some understanding of scenario planning and want to implement it in full.
In this post, I discuss some aspects of scenario planning, but the post is not intended to be a standalone summary of scenario planning. For that purpose, I recommend reading the overviews of scenario planning listed below. In this post, I'll look at examples of scenario planning in diverse domains, and I'll consider the relationship between scenario planning and the more conventional, quantitative approach to forecasting.
Overviews of scenario planning
Key people
A quick summary of scenario planning
Scenario planning involves the generation of (usually two or three) scenarios for how the future might transpire. Here are some typical aspects of scenario planning:
Who uses scenario planning?
Scenario planning caught on after the 1973 oil shock; prior to that, only Shell and GE engaged in it (see the Bradfield et al. overview for more). A 1981 survey by Linneman and Klein found three predictors of whether a company used scenario planning:
Big successes of scenario planning
Some examples of scenario planning
I looked on the Internet for scenario planning writeups across diverse domains that were publicly available. I list some examples below. The focus is on what I was able to find on the Internet by using a range of search phrases, and the list below is unlikely to be representative of scenario planning.
Does scenario planning involve forecasting?
There are many ways in which scenario planning overlaps with forecasting:
Evaluation of the utility of scenario planning (some random thoughts)
I wrote above: "The utility [of scenario planning] is in how it prepares the people undertaking the exercise for the relevant futures. One way it could so prepare them is if the early indicators of the scenarios are correctly chosen and, upon observing them, people are able to identify what scenario they're in and take the appropriate measures quickly."
What does research say about scenario planning? My impression is that there is very limited research, and it tends to be positive about the effect of scenario planning on long-range development, but there is considerable uncertainty. Bradfield et al. quote Schnaars as saying that there is "a small body of research based on empirical studies of related topics, which ‘offer some evidence as to the value of scenarios’ as a long range planning tool."
In general, evaluating scenario planning seems difficult, because unlike the case of forecasting, where the accuracy of the forecast is a good first proxy for utility, scenario planning does not lend itself to that sort of evaluation. The examples that Schwartz lists provide some anecdotal evidence. But there is the obvious selection versus treatment issue: maybe the pioneers of scenario planning were selected as the sort of people who could plan better for the future, and the scenario planning exercise itself wasn't useful. The selection versus treatment issue could be resolved by looking at whether business planning has become more efficient on the whole as scenario planning has come to be more widely used (because the continued spread of scenario planning is probably a treatment rather than a selection effect). But isolating scenario planning as a causal factor, when so many other aspects of business strategy are changing, is hard.
Another possible approach: how different would the world be today if people didn't use scenario planning? The IPCC reports on climate change may just consider one "official forecast" with a margin of uncertainty. Energy companies might still have an Official Future (again with an expressed margin of uncertainty, but without a discrete set of alternative futures). Based on my (non-expert) assessment, it seems that the quality of business strategy and policy insight in such a world would be worse than in the current world. But perhaps if scenario planning hadn't been developed, other ways of thinking about the future would have caught on more. The upshot is that I tentatively think scenario planning has been useful, but I don't see a clear way of demonstrating this in a scientifically rigorous manner, given the nature of the beast.