Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Kahneman's Planning Anecdote

25 Post author: Eliezer_Yudkowsky 17 September 2007 04:39PM

Followup toPlanning Fallacy

From "Timid Choices and Bold Forecasts: Cognitive Perspective on Risk Taking" by Nobel Laureate Daniel Kahneman and Dan Lovallo, in a discussion on "Inside and Outside Views":

In 1976 one of us (Daniel Kahneman) was involved in a project designed to develop a curriculum for the study of judgment and decision making under uncertainty for high schools in Israel.  When the team had been in operation for about a year, with some significant achievements already to its credit, the discussion at one of the team meetings turned to the question of how long the project would take.  To make the debate more useful, I asked everyone to indicate on a slip of paper their best estimate of the number of months that would be needed to bring the project to a well-defined stage of completion: a complete draft ready for submission to the Ministry of education.  The estimates, including my own, ranged from 18 to 30 months.

At this point I had the idea of turning to one of our members, a distinguished expert in curriculum development, asking him a question phrased about as follows:

"We are surely not the only team to have tried to develop a curriculum where none existed before.  Please try to recall as many such cases as you can.  Think of them as they were in a stage comparable to ours at present.  How long did it take them, from that point, to complete their projects?"

After a long silence, something much like the following answer was given, with obvious signs of discomfort:  "First, I should say that not all teams that I can think of in a comparable stage ever did complete their task.  About 40% of them eventually gave up.  Of the remaining, I cannot think of any that was completed in less than seven years, nor of any that took more than ten."

In response to a further question, he answered:  "No, I cannot think of any relevant factor that distinguishes us favorably from the teams I have been thinking about.  Indeed, my impression is that we are slightly below average in terms of our resources and potential."

Facing the facts can be intolerably demoralizing.  The participants in the meeting had professional expertise in the logic of forecasting, and none even ventured to question the relevance of the forecast implied by our expert's statistics: an even chance of failure, and a completion time of seven to ten years in case of success.  Neither of these outcomes was an acceptable basis for continuing the project, but no one was willing to draw the embarrassing conclusion that it should be scrapped.

So, the forecast was quietly dropped from active debate, along with any pretense of long-term planning, and the project went on along its predictably unforeseeable path to eventual completion some eight years later.

Comments (8)

Sort By: Old
Comment author: michaelkeenan 10 April 2014 06:57:09AM 12 points [-]

Re-reading this many years later, I noticed something:

In 1976 one of us (Daniel Kahneman) was involved in a project designed to develop a curriculum for the study of judgment and decision making under uncertainty for high schools in Israel...the project went on along its predictably unforeseeable path to eventual completion some eight years later.

There exists a curriculum for the study of judgment and decision making under uncertainty for high schools! Someone spent eight years developing it! Where can we get this curriculum?

Comment author: Regex 17 June 2016 04:41:41PM 3 points [-]

Apparently it "never saw daylight". I bet he'd still have a copy for the materials if one were to get in contact with him. How much of that wouldn't be in Thinking Fast and Slow though?

Comment author: ChristianKl 17 June 2016 06:34:57PM 1 point [-]

Thinking Fast and Slow isn't about how to teach high school students. The curriculum might have ideas about how to go about that.

Comment author: Regex 18 June 2016 02:55:22AM *  1 point [-]

I agree. Nowhere else are we likely to get something optimized for that especially since it took nearly a decade to create.

Comment author: ChristianKl 18 June 2016 10:17:41AM 0 points [-]

It took a long time to create but it's nearly 4 decades old. A lot has happened since then.

Comment author: Jayson_Virissimo 18 June 2016 10:06:34PM 0 points [-]

True, but high school curricula have changed very little in the last four decades.

Comment author: ChristianKl 19 June 2016 11:53:55AM 0 points [-]

True, but high school curricula have changed very little in the last four decades.

Most of the high school curricula is about subjects where the knowledge base that's supposed to be communicated didn't change much in the last 4 decades.

But look at math. 4 decades ago students where taught how to use slide rulers. There was no Core math four decades ago.