Waving my hands: For AI to explode, it needs to have a set of capabilities such that for each capability C₁ in the set, there exists some other capability C₂ in the set such that C₂ can be used to improve C₁. (Probably really a set of capabilities would be necessary instead of a single capability C₂, but whatever.)
It may be that the minimal set of capabilities satisfying this requirement is very large. So the AI bootstrapping approach could fail on an AI with a limited set of capabilities, but succeed on an AI with a larger set of capabilities.
I suspect the real problem for a would-be exploding AI is that C₂ is going to be "a new chip foundry worth $20 billion". Even if the AI can design the plant, and produce enough value that it can buy the plant itself(and that we grant it the legal personhood necessary to do so), it's not going to happen on a Tuesday evening.
The Register talks to Google's Alfred Spector:
Google's approach toward artificial intelligence embodies a new way of designing and running complex systems. Rather than create a monolithic entity with its own modules for reasoning about certain inputs and developing hypotheses that let it bootstrap its own intelligence into higher and higher abstractions away from base inputs, as other AI researchers did through much of the 60s and 70s, Google has instead taken a modular approach.
"We have the knowledge graph, [the] ability to parse natural language, neural network tech [and] enormous opportunities to gain feedback from users," Spector said in an earlier speech at Google IO. "If we combine all these things together with humans in the loop continually providing feedback our systems become ... intelligent."
Spector calls this his "combination hypothesis", and though Google is not there yet – SkyNet does not exist – you can see the first green buds of systems that have the appearance of independent intelligence via some of the company's user-predictive technologies such as Google Now, the new Maps and, of course, the way it filters search results according to individual identity.
(Emphasis mine.) I don't have a transcript, but there are videos online. Spector is clearly smart, and apparently he expects an AI to appear in a completely different way than Eliezer does. And he has all the resources and financing he wants, probably 3-4 orders of magnitude over MIRI's. His approach, if workable, also appears safe: it requires human feedback in the loop. What do you guys think?