The root cause of progress is precision in memetic transfer of information. In the past, knowledge was lost due to oral spread of information. Generations could inherit information from the past by telling the stories from the past, and we invented sacred canonical text to prevent too much informational loss in the most important domain of morals and cooperation, but we didn't have reliable mechanisms to copy information from one generation to the next. First writing, then manual copying, then printing and ultimately, the atomization of symbolic representations of ideas (words, sentences) into moveable letters in a printing machine changed that. We were able to spread ideas reliably on a massive scale and preserve that information for generations, so each could iterate on each other in detail. Then the industrial revolution and boom, there's your hockeystick.
That's one important flywheel, but there are others. To paraphrase another comment I made here: If we lost all of our institutions, wealth, and infrastructure, except for our ability to precisely transfer information, growth would slow way down. So the only way to fully understand long-term growth is to understand all of these overlapping, interacting flywheels.
Absolute growth would slow down but relative growth would increase.
For example, if there's only one working 30 foot sailboat in year zero, and next year there are 1000 such sailboats, that's a 100000% growth rate. And there's definitely more than enough people alive who could hand build such sailboat from wooden planks and easily built pre-industrial revolution tools.
But of course this assumes the preservation of memetic information.
Conversely if humanity lost its accumulated memetic information in this area and adjacent areas, but all equipment, tools, institutions, etc., were preserved, it would not be possible to build a single one within a year since it takes more than a year to relearn the process via trial-and-error.
Maybe if there was a fully automated factory that had a 'make sailboat' button then this would be possible. Though that just moves the need for accurate transfer of information from humans to the factory machines.
Either way you look at it, the 'make sailboat' knowledge would be by far the biggest determinant in growth rate of sailboat production post-catastrophe, at least for the first few years. Once the knowledge has been reacquired then growth rates will skyrocket compared to the world where only information has been preserved. This seems to indicate the biggest 'flywheel' would be time variant.
Brad Delongs book that’s about to come out seems to be aimed at this question (at least from information about it before reading it)
Good question, probably because energy doesn't seem pivotal for the specific case of the threshing machine? It was clearly a crucial piece of infrastructure in many other cases. I consider it part of the technology flywheel, along with other fundamental enabling technologies such as precision manufacturing. There's a good argument that it is the most important of all such fundamental technologies.
So in your specific example of the threshing machine:
Surplus energy is required such that enough of the population are freed from subsistence and agriculture to specialize in other things.
Even more surplus energy is required for the creation/upkeep of cities, which are a prerequisite for technological innovation/growth (high density of different specialists living alongside eachother, as well as a labour force for factories/mass production).
And the railroads that enabled the widespread distribution of threshing machines - obviously highly energy intensive, and coincided with massive growth in coal use (steam engines originally invented for pumping water out of coal mines)
You seem to be labelling energy as a technology? If so, I think this is wrong. Energy is a fundamental input. Certainly technology is involved in capture/extraction/utilisation. But... hmm there's a quote 'Labour without energy is a corpse, capital (substitute technology here) without energy is a sculpture'.
My position is that all technological growth is dependent on sufficient surplus energy. That certain levels of technological and social complexity have minimum surplus energy requirements.
So to answer the question in your title directly: because there wasn't enough surplus energy available. This was the single most important bottleneck at every stage in our development. It remains the single most important bottleneck today.
Vaclav Smil is one of the better people to read on this subject. And if I remember correctly he has an interesting example about the plough (in Energy and Civilization) - an even more headscratching example of 'why wasn't this invented waaaay earlier?'
Your examples of energy usage enabling further economic growth are good ones, particularly the railroads, which absolutely depended on the ability to harness wood or coal for locomotion. But I disagree with how you interpret these examples and the rest of your analysis.
Re:
You seem to be labelling energy as a technology? If so, I think this is wrong. Energy is a fundamental input.
Well, first, there are such things as energy technologies. The steam engine is a technology. Processes to create coke from coal, or to refine crude oil, are technologies. These technologies are what make all of that energy accessible and usable.
I don't know what it means for energy to be a “fundamental input.”
When you say:
So to answer the question in your title directly: because there wasn't enough surplus energy available.
I don't think this does answer the question, because technological/industrial progress is what made that surplus energy available. It didn't just become available for some other reason. The surplus was created by progress itself. So it can't be used to explain progress.
In short, surplus energy is not exogenous to technological or economic growth, it is endogenous.
>Well, first, there are such things as energy technologies. The steam engine is a technology. Processes to create coke from coal, or to refine crude oil, are technologies. These technologies are what make all of that energy accessible and usable.
To quote my post:
>Certainly technology is involved in capture/extraction/utilisation. But... hmm there's a quote 'Labour without energy is a corpse, capital (substitute technology here) without energy is a sculpture'.
And back to you (emphasis mine):
>I don't think this does answer the question, because technological/industrial progress is what made that surplus energy available. It didn't just become available for some other reason. The surplus was created by progress itself. So it can't be used to explain progress.
I agree that technologies increased our access to surplus energy. I strongly disagree that they 'created' it.
Fossil fuels are exogenous to tech, right? They're an energy store that was created long before homo sapiens turned up. And the quality/quantity of this energy store is huge, gargantuan... it's the greatest treasure trove in the history of our planet.
But without such a dense & economical energy source available... you don't get an industrial revolution. Technological progress plateaus.
This is what I mean by a fundamental input - the energy store has to exist in the first place, in order to be harnessed by technology. The surplus is not being 'created', it is being 'harvested'.
I think I'm derailing your topic somewhat, as we discuss more I think I understand more about your thought and I don't think this 'nuance' of energy is very relevant within this framework.
Outside of it though, hugely important. The belief that we can 'invent energy' has fairly disastrous consequences for our civilization.
The physical fact of hydrocarbons sitting in the ground is exogenous, yes. But that was true since before humans existed, so it doesn't explain progress. You need an explanation for why we didn't start using those fuels on a large scale until the 1700s or so. And the proximal explanation for that is technology.
Thanks Jason, great post.
Are there particular knowledge gaps you feel the less wrong community has with respect to the types of progress you've reviewed? Any gaps that are perhaps deeper here than in other similarly thoughtful communities?
Why was progress so slow in the past?
Knowledge development feeds back on itself. So when you have a little knowledge you get a slow speed of further development, and when you have a lot of knowledge you get a fast speed. The more knowledge we get, the faster we go.
Yes, but that's only one of many flywheels. If we lost all of our wealth, infrastructure, and institutions, but kept all of our knowledge, growth would slow way down. The only way to fully understand long-term growth is to understand all of these overlapping, interacting flywheels.
I’d add that “our knowledge” is heavily physically embodied in objects. An example is computer code. At some point, somebody wrote every line. Years later, nobody may really know why or how certain parts of the codebase was written, but the software still works.
Likewise, much knowledge is contingent on the physical structure of the world. I have knowledge of my relationship with my colleagues, of my lab’s layout, etc. Take away the specific people and objects and my knowledge is useless.
GDP per capita is growing ~linear on a logarithmic scale though:
Any data we have on GDP per capita in the 18th century is probably dubious. In the 10th century ? Not even dubious.
Yes, but we can still be confident in the broad pattern. Paul Romer explains:
My conviction that the rate of growth in GDP per capita at the technological frontier had to be increasing over time sprang from a simple calculation. Suppose the modern rate of growth of real GDP per capita (that is the growth rate after taking out the effects of inflation) is equal to 2% per year and that income per capita in year 2000 is $40,000. If this rate had prevailed for the last 1,000 years, then in the year 1000, income per capita measured in the purchasing power of dollars today would have been $0.0001, or 0.01 cents. This is way too small to sustain life. If the growth rate had been falling over time instead of remaining constant, then the implied measure of GDP per capita in the year 1000 would have been even lower.
He concludes:
Reasonable people can differ about what the future holds, but the simple calculation that first got me thinking about this (and which no doubt influenced how Maddison did the backward projections to come up with his estimates) leaves no room for doubt about what happened in the past. The rate of growth of GDP per capita has increased over time.
On account of sounding dumb, but needing to point out, progress is set by baseline of some sort.
However qualitatively our society does live differently in many ways.
Most things in the past required huge time investment, there was low security for life, and generally the product be it crops, or food, were small compared to these days.
So the constraints people dealt with were huge.
Its fallacy though to think we don't have the exact same issues in today's world.
We have washing machines and microwaves and tools that speed things up so we can do the more progressive things so to speak.
The problem is we are subject to biological evolution and therefore progress needs more than quantity.
For instance dinosaurs are extinct yet I bet most dinosaurs would boast that they are huge and bigger than other dinosaurs therefore they made more progress and therefore are the bestests of all.
Human evolution can be the same issue.
On scale of evolution the danger we face is not that we are so good at adaptation and use of tools and changing environments.
We face the problem of change.
All change leads to extinction of some sort. Change is dangerous, but we live in a world were change is becoming a constant variable.
And there is no amount of progress that protects humans from this tricky problem.
If not today 1000 years from now, we might be just as extinct as dinosaurs and from the wood works will crawl out some weird creature of tiny stature like mammals did.
We should therefore not assume the fallacy that bit of fossil fuels, and combustion engines or rocketry will make us better at surviving.
We have better chance of surviving today as individuals, but the behemoth civilization became, is by far something that has never existed before.
Its also not something humans ever did on this mass world wide scale with such pacing and such incredible complex balance of activity.
So yes once people overcame certain constraints that were constant we grew.
So as with all things considering something new, you need lots of trail and error to know what works.
In this sense nature does not have friends or good side.
Nature can kill off anything, and there were creatures that lived for millions of years and went in way of dodo in a single instance as if they never existed.
We have to understand that progress in many ways is limited in certain dimensions for all people.
We as humans are extremely good at predicting certain catastrophic scenarios, but we should not think for one second that nature cannot do more than we can imagine.
We have funny things like bees dying off.
A single species could end our civilization.
And this assuming bees are the only species.
Who knows what other species could die off and kill humanity in basically few years.
So my basic question is what progress really is?
If its survival we made some progress, but we added a layer of problems that never threatened humanity.
If its number of humans on this planet and their long life, that is progress, but it comes at a cost.
Cost we will have to pay back at some point in worlds history to simply go on.
What explains the hockey-stick shape of world GDP over time, with seemingly no progress for thousands of years, followed by soaring growth?
The first question you might ask is: is this just an exponential curve? If so, then the explanation is simple: we see a constant growth rate every year, and the steep upward slope is just what exponential curves look like. There’s no more mystery than there would be about the shape of a population curve.
To check this, we can plot the same numbers on a logarithmic y-axis. In such a chart, exponential curves become straight lines. But when we do this, we still see an upwards slope. That is, the growth rate has increased over time:
Nor is it just a consequence of population growth, because we see the same pattern in GDP per capita:
I’ve previously said that a core reason for this is that progress compounds, creating a flywheel effect. Here’s another way of looking at the same idea. Why wasn’t the threshing machine invented in, say, the 1300s? Consider all the barriers to such a thing:
Human capital
First, who would have invented it?
Would it have been a farmer? (Well over half the workforce were farmers.) When would he have found the time to tinker? Labor productivity and incomes were low; there wasn’t much spare time or material for inventing.
Who else could have done it? There was no established professional class of inventors, engineers, or entrepreneurs. The closest were skilled craftsmen who made machines, such as clockmakers or millwrights. Overall there were many fewer inventors per capita than today (ok, I don’t have data on this right now, but I’m pretty confident in this assertion).
If it were, say, a millwright, he would have to learn enough about machines to go beyond the kinds that he had been taught to make through apprenticeship, and invent something entirely new. Where would this knowledge have come from? There was no printed material, and no mechanics’ institutes.
And if our inventor did have mechanical skill, why why would he decide to apply it to a practical invention for farmers? It was more prestigious and lucrative to make clockwork novelties for the aristocracy. Even if the inventor did have a practical bent, there were social taboos against labor-saving devices.
Manufacturing
Suppose that despite all of this, some inventive person is determined to make a threshing machine and acquires the resources and time to experiment. Maybe he avoids the trap of making a machine that mimicks human motions, and hits on the idea of a rotating drum with teeth.
He will find that making the machine work reliably is very difficult. It requires a high degree of skill in the mechanic that crafts it by hand. There are no machine tools to create precision parts. Wood is too soft for precision work; metal parts are required. Machines that are shoddily constructed break easily or bruise the grain instead of threshing it. (See my full post on the threshing machine for elaboration.)
Distribution
Suppose our inventor overcomes all these obstacles, and manages to create a practical threshing machine. What next?
He could use it on his own farm, if he is a farmer. If that’s all he does, then his invention has had no significant impact on the economy and no impact at all on history. This is not what we are seeking to explain.
To matter for progress, the invention needs to be distributed. And here our inventor faces more obstacles.
Who is the market for his invention? Will other farmers be receptive to it? They need to change their methods and take a risk on something new, something they are not used to doing.
Even if they are willing to take that risk, do they have the capital? Buying a piece of agricultural equipment is an investment that won’t pay off right away. Do farmers have enough money saved for such an investment? Likely not, given low incomes, and there is no financial infrastructure to give loans for such purposes. Would such an investment even pay off on a small farm? It may require a certain level of scale to be worth it.
Market creation
But suppose there is a market that is willing and able to pay for threshing machines. How would our inventor, now turned entrepreneur, serve that market?
He may be able to serve his village or town, through word of mouth and local dealings. But that small market is probably not enough to support a business dedicated to threshing machines—and again, if only one town were served, it would have a limited impact on progress. Our aspiring threshing machine tycoon needs to server a larger, regional or even national market.
How is he going to promote his product? There are no newspapers or other media, not even the printing press. There may be occasional local fairs (although, again, the attendees are typically not prepared to consider new inventions).
If potential customers do hear about the product, how do they order one? There is no postal service to send messages or money. Similarly, how would the product be transported to the customer from wherever it is assembled? There are no locomotives, and the wagon roads are in poor condition. River or canal transport might be possible, but that won’t go the last mile to each customer.
And if the business takes off, will our entrepreneur be able to source enough raw materials to keep up production? All of the problems of finding customers apply to finding suppliers as well.
Financial and legal infrastructure
Even if these obstacles could be overcome, the entrepreneur is going to need capital to get started. And again, there is very little in the way of any kind of financial market to support speculative investments like this.
Suppose our plucky hero is very enterprising and decides to crowdfund his effort by collecting small investments from a large number of people. He has no way to form a corporation for this purpose, because corporate law has not yet been developed. (A partnership would not be practical with a large number of partners, especially since there was no limited liability.)
If there were the infrastructure needed to start businesses, the entrepreneur might find himself facing competition from others who steal his ideas and copy his machine. If he had royal favor, he might be granted a monopoly, but there was no patent office where he could send an application, nor any established rule awarding patents to inventions.
And if he overcame all of the above obstacles, he might find that he faced opposition from those whom his progress threatened, such as farmhands who took on manual threshing work. They might oppose him by seeking legal restrictions on his business, or by illegal means such as smashing and burning machinery. (I don’t know of this happening to threshing machines, but it certainly happened to textile machinery.) Would the government come to the aid of the inventor, or of the displaced workers, or would they stay out of the whole affair?
How progress actually happened
If we fast-forward through the centuries, we can see how the underpinnings of progress were gradually established. The threshing machine makes a good example because it was an obvious idea that struggled for a long time to be born, so we can see the stages it went through:
By the 1600s at the latest, the idea of applying mechanical ingenuity to practical problems was well established, and we can see people talking about the idea of threshing machines (although I have seen no evidence that any were working yet).
By the 1700s, there were at least a few mechanics in existence who had the skill to create a working threshing machine, but they were only serving their local area. Others announced projects to distribute plans and models for the machine, but they did not expect to make this a business, and instead asked for donations to support the work. By this point there were newspapers where such schemes could be advertised, and postal service for individuals to communicate about them.
By the early 1800s at least, there were inventor/entrepreneurs who had patented threshing machines and were trying to make a business of them. There were farmer’s journals that discussed such inventions and improvements, and many farmers were eager to try new things to improve their productivity. But there was still very little manufacturing capacity, and inventors such as Joseph Pope were still offering to sell plans which could be implemented by a local workman. Soon after, though, Pope was contracting with a specialized machine shop, an engine manufacturer, to make his machine.
By the mid-1800s, railroads would be established that could ship the machines to customers over a wide area. It’s around this time that threshing machines become widely adopted.
Today
To really drive the point home, imagine that the problem of mechanizing threshing had been completely overlooked for the last few hundred years, while all other progress moved forward.
The threshing problem would be solved almost instantly.
There is an entire professional class of entrepreneurs looking for opportunities exactly like this. It would be easy for them to look up data on agricultural processes and cost drivers, and to find that a very large part of grain cost was manual threshing. It would be obvious that this should be mechanized.
Designing the machine would be no problem—there are many professionals with bachelors’ degrees in mechanical engineering who could do the job. They would have standard parts to choose from out of a catalog, such as gears and motors, and they could specify the design quickly and precisely using CAD software. Any specialized parts could be 3D-printed for rapid prototyping. Manufacturing would similarly be no problem, thanks to the enormous infrastructure we have built up for this.
Three companies to solve this problem would be in the next batch for Y Combinator. They would each form a Delaware C-corporation with a simple filing and some standard legal documents, raise millions of dollars within a few days by meeting with investors over Zoom, sign a contract online through DocuSign, and have the money wired immediately to a bank account they set up in twenty minutes with Mercury.
They would establish a website to market the product, complete with spec sheets, promotional videos, etc. They could get a list of the biggest agricultural companies and reach out to them directly by email, promote the product online using targeted advertisements, and fly to large international trade shows to exhibit there. They could take orders online as well, and ship anywhere via UPS, FedEx, or DHL. They would have a global market from day one.
And their customers would be ready, even eager, for such an innovation. They would be used to the idea of saving costs through better technology. They would have full financial accounting statements to show them where their biggest costs are. They would have executives and program managers whose jobs include evaluating new technologies and buying them. There would be standard legal agreements, purchase orders, and payment mechanisms.
In sum, the road for this kind of progress has already been paved—both metaphorically and literally.
The roots of progress?
All of this has been an illustration of the many, overlapping, interacting flywheels of progress that generate super-exponential growth over the very long term.
What I am much less clear on is which of these factors, if any, can be seen as derivative and which are fundamental—if some were inevitable given other, enabling factors. This is a much harder question to answer.
But even if we could answer that, it wouldn’t change the fact that all of these factors are real and important, and progress depends on all of them working together.