Aubrey argues for "the singularity" here:
"The singularity and the Methuselarity: similarities and differences" - by Aubrey de Grey
http://www.sens.org/files/sens/FHTI07-deGrey.pdf
He uses the argument from personal incredulity though - one of the weakest forms of argument known.
He says:
"But wait – who’s to say that progress will remain “only” exponential? Might not progress exceed this rate, following an inverse polynomial curve (like gravity) or even an inverse exponential curve? I, for one, don’t see why it shouldn’t. If we consider specifically the means whereby the Singularity is most widely expected to occur, namely the development of computers with the capacity for recursive improvement of their own workings, I can see no argument why the rate at which such a computer would improve itself should not follow an inverse exponential curve, i.e. one in which the time taken to achieve a given degree of improvement takes time X, the time taken to repeat that degree of improvement is X/2, then X/4 and so on."
My reply: to build a fast computer you don't just need to perform computations quickly. You also need to be able to design and perform real-world experiments and tests. While sensor and motor capabilities are improving, they are not doing so with the same doubling time as exists for computers. Consequently, progress in these fields (and overall progress) is correspondingly slower.
Then there's the influence of limits. Clock speed has already run into diminishing returns. By the time the slower-doubling systems have time to double very many times, the faster-doubling ones will have maxed-out - and will be hindering overall progress.
IMO, nobody seems to have thought the "singularity" idea through :-(
I haven't read the whole essay, but the portion that you quoted isn't an argument from incredulity.
An argument from incredulity has the form "Since I can't think of an argument for assigning P low probability, I should assign P high probability.".
Aubrey's argument has the form "Since I can't think of an argument for assigning P low probability, I shouldn't assign P low probability.".
Robin criticizes Eliezer for not having written up his arguments about the Singularity in a standard style and submitted them for publication. Others, too, make the same complaint: the arguments involved are covered over such a huge mountain of posts that it's impossible for most outsiders to seriously evaluate them. This is a problem for both those who'd want to critique the concept, and for those who tentatively agree and would want to learn more about it.
Since it appears (do correct me if I'm wrong!) that Eliezer doesn't currently consider it worth the time and effort to do this, why not enlist the LW community in summarizing his arguments the best we can and submit them somewhere once we're done? Minds and Machines will be having a special issue on transhumanism, cognitive enhancement and AI, with a deadline for submission in January; that seems like a good opportunity for the paper. Their call for papers is asking for submissions that are around 4000 to 12 000 words.
The paper should probably
Devote the second half to discussing the question of FAI, with references to e.g. Joshua Greene's thesis and other relevant sources for establishing this argument.Carl Shulman says SIAI is already working on a separate paper on this, so it'd be better for us to concentrate merely on the FOOM aspect.I have created a wiki page for the draft version of the paper. Anyone's free to edit.