[10/50/90% = 2021/2022/2025] 50% chance scenario: At the end of 2020, Google releases their own API with a 100x larger model. In 2021, there is a race for who could scale it to an even larger model, with Manhattan project level investments from nation states like China and the US. Physicists, computer scientists and mathematicians are working on bottlenecks privately, either at top companies or in secret nation-state labs. People have already built products on top of Google's API which make researchers and engineers more productive. At the end of 2021, one country (or lab) reached a GPT-5 like model size but doesn't publish anything. Instead, researchers use this new model to... (read more)
In Developmental Stages of GPTs, orthonormal explains why we might need "zero more cognitive breakthroughs" to reach transformative AI, aka "an AI capable of making an Industrial Revolution sized impact". More specifically, he says that "basically, GPT-6 or GPT-7 might do it".
Besides, Are we in an AI overhang? makes the case that "GPT-3 is the trigger for 100x larger projects at Google, Facebook and the like, with timelines measured in months."
Now, assuming that "GPT-6 or GPT-7 might do it" and that timelines are "measured in months" for 100x larger projects, in what year will there be a 10% (resp. 50% / 90%) chance of having transformative AI and what would the 50% timeline look like?
To be clear, 90% is the probability of getting transformative AI before 2025 conditional on (i) "GPT-6 or GPT-7 might do it" and (ii) we get a model 100x larger than GPT-3 in a matter of months. So essentially the question boils down to how long would it take to get to something like GPT-6 given that we're already at GPT-4 at the end of the year.
-- re scaling hypothesis: I agree that there might be compute bottlenecks. I'm not sure how compute scales with model size, but assuming that we're aiming for a GPT-6 size model, we're actually aiming for roughly 6-9 orders of magnitude bigger models (~100-1000x scale up each),... (read more)