My feedback, much of which has been incorporated into the latest draft:

Moreover, because the "singularity" term is popularly associated with several claims and approaches we will not defend, we must explain what we are not claiming.

Obvious ref here to Yudkowsky's schools essay; better would be an academic link, has his typology appeared anywhere academic yet?

pg 4

overconfident of our predictions

Nitpick, that sounds really weird to my ears although I am not sure it is actually ungrammatical. I suggested deleting 'of our predictions' entirely.

Estimating progress in scientific research output. Imagine a man digging a ten-kilometer ditch. If he digs 100 meters in one day, you might predict the ditch will be finished in 100 days. But what if 20 more diggers join him, and they are all given amphetamines? Now the ditch might not take so long.

Nootropics are not very impressive; strongly suggest changing the metaphor to something involving backhoes, which is fair - as AI software is developed in various dimensions, it can be applied to the task of further output (recursively, per later section 'Accelerated science').

Several factors may decelerate our progress toward the first creation of AI. For example:...Global totalitarianism. [paragraph]

Well hey, maybe my chip fab/bomber suggestion is not so useless after all.

Quantum computing may also emerge during this period. Early worries that quantum computing may not be feasible have been overcome, but it is hard to predict whether quantum computing will contribute significantly to the development of digital intelligence because progress in quantum computing depends heavily on unpredictable insights in quantum algorithms (Rieffel and Polak 2011).

Should mention that as things stand, there aren't any known algorithms that would directly speed up an AI. Look at https://en.wikipedia.org/wiki/Quantum_algorithm - none of the speedups on concrete algorithms are really useful. Shor's algorithm would be useful to a rogue AI cracking security; Grover's algorithm is a nice speedup, but database lookup/list search is well optimized classically and it's hard to imagine a quantum computer with enough qubits to search a useful list and likewise for quantum counting or element counting. Quantum simulation seems like the one exception.

Datasets are expected to increase greatly in the coming decades.

Worth mentioning kryder's law and projections outpacing Moore's law: http://www.dssc.ece.cmu.edu/research/pdfs/After_Hard_Drives.pdf Kryder, Mark H.; Chang Soo Kim (October 2009). "After Hard Drives - What Comes Next?" (PDF). IEEE Transactions on Magnetics 45 (10). doi:10.1109/TMAG.2009.2024163

At age 8, Terrence Tao scored 760 on the math SAT, one of only [2?3?] children ever to do this at such an age; he later went on to [have a lot of impact on math].

I tried to find the first one, but couldn't find that exact one. What I did find was

... INTRODUCTION Terence Tao is the oldest of three children. ... Terry was the first eight year old ever to score 760 (out of a possible 800) on the SAT-M. Only 1~ of college bound 17 and 18 year o!ds in the United States attain a score of 750 or more. ... http://www.springerlink.com/content/gr62714555714348/

(No academic access, couldn't find it jailbroken.) Which is as good a claim, I think.

I don't think the latter really needs much referencing - just say he won a Fields Medal and move on.

will soon make it feasible to compare the characteristics of an entire population of adults with those adults’ full genomes, and, thereby, to unravel the heritable components of intelligence, dilligence, and other contributors to scientific achievement.

Counter-argument: http://www.ncbi.nlm.nih.gov/pubmed/21826061 http://blogs.discovermagazine.com/gnxp/2011/08/half-the-variation-in-i-q-is-due-to-genes/ Intelligence is highly hereditable but it's spread over so many alleles and interactions that embryo selection gets only a few points and one would have to edit half the genome to get to, say, 2 standard deviations above the norm.

The digital intelligence could conceivably edit its own code while it is running, but it could also create a new intelligence that runs independently.

Lots of potential citations here. All of Schmidhuber's Godel machine papers come to mind, as does classic AI using Lisp and Smalltalk to generate and run code (eg. Automated Mathematician and Eurisko).

[concluding paragraph]

May I suggest a Churchill quote I recently found? The final line, specifically:

"So now the Admiralty wireless whispers through the ether to the tall masts of ships, and captains pace their decks absorbed in thought. It is nothing. It is less than nothing. It is too foolish, too fantastic to be thought of in the twentieth century. Or is it fire and murder leaping out of the darkness at our throats, torpedoes ripping the bellies of half-awakened ships, a sunrise on a vanished naval supremacy, and an island well-guarded hitherto, at last defenceless? No, it is nothing. No one would do such things. Civilization has climbed above such perils. The interdependence of nations in trade and traffic, the sense of public law, the Hague Convention, Liberal principles, the Labour Party, high finance, Christian charity, common sense have rendered such nightmares impossible. Are you quite sure? It would be a pity to be wrong. Such a mistake could only be made once—once for all."

--Winston Churchill, 1923, recalling the possibility of war between France and Germany after the Agadir Crisis of 1911, in The World Crisis,vol. 1, 1911-1914, pp. 48-49