Moore's Law hence AI!
How is that such a weak argument? I'm all for smarter algorithms - as opposed to just increasing raw computing power - but given the algorithms that are already in existence (e.g. AIXItl, others) we'd strongly - and based on theoretical results - expect there to exist some hardware threshold, that once crossed, would empower even the current algorithms sufficiently for an AGI-like phenomenon to emerge.
Since we know that exponential growth is, well, quite fast, it seems a sensible conclusion to say "(If) Moore's Law, (then eventually) AGI", without even mandating more efficient programming. That or dispute established machine learning algorithms and theoretical models. While the software-side is the bottle-neck, it is one that scales with computing power and can thus be compensated.
Of course smarter algorithms would greatly lower the aforementioned threshold, but if (admittedly a big if) Moore's Law were to hold true for a few more iterations, that might not be as relevant as we assume it to be.
The number of steps for current algorithms/agents to converge on an acceptable model of their environment may still be very large, but compared to future approaches, we'd expect that to be a difference in degree, not in kind. Nothing that some computronium shouldn't be able to compensate.
This may be important because as long as there's any kind of consistent hardware improvement - not even exponential - that argument would establish that AGI is just a matter of time, not some obscure eventuality.
And there you've given a better theory than most AI experts. It's not Moores's law + reasonable explanation hence AI that's weak, it's just Moores's law on its own...
While going through the list of arguments for why to expect human level AI to happen or be impossible I was stuck by the same tremendously weak arguments that kept on coming up again and again. The weakest argument in favour of AI was the perenial:
Lest you think I'm exaggerating how weakly the argument was used, here are some random quotes:
At least Moravec gives a glance towards software, even though it is merely to say that software "keeps pace" with hardware. What is the common scale for hardware and software that he seems to be using? I'd like to put Starcraft II, Excel 2003 and Cygwin on a hardware scale - do these correspond to Penitums, Ataris, and Colossus? I'm not particularly ripping into Moravec, but if you realise that software is important, then you should attempt to model software progress!
But very rarely do any of these predictors try and show why having computers with say, the memory capacity or the FOPS of a human brain, will suddenly cause an AI to emerge.
The weakest argument against AI was the standard:
Some of the more sophisticated go "Gödel, hence no AI!". If the crux of your whole argument is that only humans can do X, then you need to show that only humans can do X - not assert it and spend the rest of your paper talking in great details about other things.