Bostrom flies by an issue that's very important:
Suppose that a scientific genius of the caliber of a Newton or an Einstein arises at least once for every 10 billion people: then on MegaEarth there would be 700,000 such geniuses living contemporaneously, alongside proportionally vast multitudes of slightly lesser talents. New ideas and technologies would be developed at a furious pace,
Back up. The population of Europe was under 200 million in 1700, less than a sixth of what it is today. The number of intellectuals was a tiny fraction of the number it is today. And the number of intellectuals in Athens in the 4th century BC was probably a few hundred. Yet we had Newton and Aristotle. Similarly, the greatest composers of the 18th and 19th century were trained in Vienna, one city. Today we may have 1000 or 10,000 times as many composers, with much better musical training than people could have in the days before recorded music, yet we do not have 1000 Mozarts or 1000 Beethovens.
Unless you believe human intelligence has been steadily declining, there is one Einstein per generation, regardless of population. The limiting factor is not the number of geniuses. The number of geniuses, and the amount of effort put into science, is nearly irrelevant to the amount of genius-level work accomplished and disseminated.
The limiting factor is organizational. Scientific activity can scale; recognition or propagation of it doesn't. If you graphed scientific output over the years in terms of "important things discovered and adopted by the community" / (scientists * dollars per scientist), you'd see an astonishing exponential decay toward zero. I measured science and technology output per scientist using four different lists of significant advances, and found that significant advances per scientist declined by 3 to 4 orders of magnitude from 1800 to 2000. During that time, the number of scientific journals has increased by 3 to 4 orders of magnitude, and a reasonable guess is that so did the number of scientists. Total recognized "significant" scientific output is independent of the number of scientists working!
You can't just add scientists and money and get anything like proportional output. The scientific community can't absorb or even be aware of most of the information produced. Nor can it allocate funds or research areas efficiently.
So a critical question when thinking about super-intelligences is, How does the efficiency of intelligence scale with resources? Not linearly. To a first approximation, adding more scientists at this point accomplishes nothing.
On the other hand, merely recognizing and solving the organizational problems of science that we currently have would produce results similar to a fast singularity.
Good post.
First of all, knowledge is partially ordered. A bunch of lesser-known results were required before Einstein could bring together the mathematical tools and physics knowledge sufficient to create relativity. True enough, this finding may have come much later, if not for Einstein, but dozens of others built predecessor results that also required great insight.
Similarly, we should not decry the thousands of biologists who have been cataloging every single protein, its post-translational modifications and its protein-protein interactions in exhaust...
This is part of a weekly reading group on Nick Bostrom's book, Superintelligence. For more information about the group, and an index of posts so far see the announcement post. For the schedule of future topics, see MIRI's reading guide.
Welcome. This week we discuss the fifth section in the reading guide: Forms of superintelligence. This corresponds to Chapter 3, on different ways in which an intelligence can be super.
This post summarizes the section, and offers a few relevant notes, and ideas for further investigation. Some of my own thoughts and questions for discussion are in the comments.
There is no need to proceed in order through this post, or to look at everything. Feel free to jump straight to the discussion. Where applicable and I remember, page numbers indicate the rough part of the chapter that is most related (not necessarily that the chapter is being cited for the specific claim).
Reading: Chapter 3 (p52-61)
Summary
Notes
In-depth investigations
If you are particularly interested in these topics, and want to do further research, these are a few plausible directions, some inspired by Luke Muehlhauser's list, which contains many suggestions related to parts of Superintelligence. These projects could be attempted at various levels of depth.
How to proceed
This has been a collection of notes on the chapter. The most important part of the reading group though is discussion, which is in the comments section. I pose some questions for you there, and I invite you to add your own. Please remember that this group contains a variety of levels of expertise: if a line of discussion seems too basic or too incomprehensible, look around for one that suits you better!
Next week, we will talk about 'intelligence explosion kinetics', a topic at the center of much contemporary debate over the arrival of machine intelligence. To prepare, read Chapter 4, The kinetics of an intelligence explosion (p62-77). The discussion will go live at 6pm Pacific time next Monday 20 October. Sign up to be notified here.