Check the section called "derivations" here: it links to a document attempting to list all conceptual breakthroughs in AI, of at least a certain significance, ever http://mediangroup.org/insights with related discussion on forecasting implications here: https://ai.metaculus.com/questions/2920/will-the-growth-rate-of-conceptual-ai-insights-remain-linear-in-2019/

-Reverse Mode Autodiff

-Using GPUs for computing.

These are the two big ones. Yes there are some others, but those two ideas together are the backbone of the current AI and ML boom.

From A 'Brief' History of Neural Nets and Deep Learning, Part 4:

So, why indeed, did purely supervised learning with backpropagation not work well in the past? Geoffrey Hinton summarized the findings up to today in these four points:

1. Our labeled datasets were thousands of times too small.
2. Our computers were millions of times too slow.
3. We initialized the weights in a stupid way.
4. We used the wrong type of non-linearity.

I think this blog series might help provide a partial answer to your question.