Could democritus have predicted an Intelligence Explosion?
Only if he had been superhumanly able to follow arguments through to the maximum extent of their consequences.
The number and complexity of the intervening steps, listed by morendil, put “FOOM from Atoms alone" firmly beyond the predictive power from limited evidence exhibited by any historic person who in the end turned out to be right. Which is to say, beyond Democritus, as he perhaps marks the upper end of this scale. There is good reason to believe that there is not sufficient working memory available to members of species Homo Sapiens to discover “FOOM from Atoms alone.” I mean, none of the greatest scientists were near that good.
Intelligence explosion follows from physicalism and scientific progress and not much else.
I agree that physicalism affords the possibility of an Intelligence Explosion from the observed existence of minds, and the physical limits to the size and power of possible minds, compared to the size and power of observed minds.
From this we can conclude that superintelligent minds are permitted by physics, but not necessarily that one can jump from intelligence to superintelligence through an intelligence explosion.
We know something about physical limits of Minds. Less is known about the limits to the progress of science, and it's more relevant proxy, engineering. We do not know if the maximally powerful engineering available to evolved minds is above the level required for initiating the chain-reaction of recursive self-improvement, initiating an intelligence explosion towards the physical limits for minds.
This begs the question: Is the ceiling of engineering prowess available to evolved minds lower than the floor of engineering prowess required for recursive self improvement?
To the best of my knowledge, this question is undecided.
Is there good evidence in one direction or the other?
Superintelligence by way of a controlled Intelligence Explosion seems harder than General Artificial Intelligence by Uploading by a large amount. Like intergalactic travel is harder than interstellar travel by a large amount.
An Intelligence Explosion may permitted under physicalism, yet be prohibited by cost and complexity, like intergalactic travel is permitted under physicalism, yet prohibited by cost and engineering complexity.
Thereby saving minds such as we, around the minimum level of intelligence required for general intelligence, from being displaced by minds that are powerful to the maximum degree that the limits-of-physics allow.
Safety wise initiating an Intelligence Explosion seems around as safe as going to the moon by riding on the heat and pressure of a nuclear chain reaction instead of on the tip of a large chemical rocket.
From this we can conclude that a superintelligent minds are permitted by physics, but not necessarily that one can jump from intelligence to superintelligence through an intelligence explosion.
I still don't perceive it to be anywhere close to being "obvious" that an intelligence explosion, as opposed to learning and intelligence amplification, is more than a possibility (in the sense that is far from being certain).
Also see: History of the Friendly AI concept.
The ancient atomists reasoned their way from first principles to materialism and atomic theory before Socrates began his life's work of making people look stupid in the marketplace of Athens. Why didn't they discover natural selection, too? After all, natural selection follows necessarily from heritability, variation, and selection, and the Greeks had plenty of evidence for all three pieces. Natural selection is obvious once you understand it, but it took us a long time to discover it.
I get the same vibe from intelligence explosion. The hypothesis wasn't stated clearly until 1965, but in hindsight it seems obvious. (Michael Vassar once told me that once he became a physicalist he said "Oh! Intelligence explosion!" Except of course he didn't know the term "intelligence explosion." And he was probably exaggerating.)
Intelligence explosion follows from physicalism and scientific progress and not much else. Since materialists had to believe that human intelligence resulted from the operation of mechanical systems located in the human body, they could have realized that scientists would eventually come to understand these systems so long as scientific progress continued. (Herophilos and Erasistratus were already mapping which nerves and veins did what back in the 4th century B.C.)
And once human intelligence is understood, it can be improved upon, and this improvement in intelligence can be used to improve intelligence even further. And the ancient Greeks certainly had good evidence that there was plenty of room above us when it came to intelligence.
The major hang-up for predicting intelligence explosion may have been the the inability to imagine that this intelligence-engineering could leave the limitations of the human skull and move to a speedier, more dependable and scalable substrate. And that's why Good's paper had to wait until the age of the computer.
</ speculation>