= 27d567bc2d48d9f40e9ccc20ea370b15)
Luke_A_Somers comments on Superintelligence 5: Forms of Superintelligence - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (112)
Bostrom flies by an issue that's very important:
Back up. The population of Europe was under 200 million in 1700, less than a sixth of what it is today. The number of intellectuals was a tiny fraction of the number it is today. And the number of intellectuals in Athens in the 4th century BC was probably a few hundred. Yet we had Newton and Aristotle. Similarly, the greatest composers of the 18th and 19th century were trained in Vienna, one city. Today we may have 1000 or 10,000 times as many composers, with much better musical training than people could have in the days before recorded music, yet we do not have 1000 Mozarts or 1000 Beethovens.
Unless you believe human intelligence has been steadily declining, there is one Einstein per generation, regardless of population. The limiting factor is not the number of geniuses. The number of geniuses, and the amount of effort put into science, is nearly irrelevant to the amount of genius-level work accomplished and disseminated.
The limiting factor is organizational. Scientific activity can scale; recognition or propagation of it doesn't. If you graphed scientific output over the years in terms of "important things discovered and adopted by the community" / (scientists * dollars per scientist), you'd see an astonishing exponential decay toward zero. I measured science and technology output per scientist using four different lists of significant advances, and found that significant advances per scientist declined by 3 to 4 orders of magnitude from 1800 to 2000. During that time, the number of scientific journals has increased by 3 to 4 orders of magnitude, and a reasonable guess is that so did the number of scientists. Total recognized "significant" scientific output is independent of the number of scientists working!
You can't just add scientists and money and get anything like proportional output. The scientific community can't absorb or even be aware of most of the information produced. Nor can it allocate funds or research areas efficiently.
So a critical question when thinking about super-intelligences is, How does the efficiency of intelligence scale with resources? Not linearly. To a first approximation, adding more scientists at this point accomplishes nothing.
On the other hand, merely recognizing and solving the organizational problems of science that we currently have would produce results similar to a fast singularity.
While failure to recognize & propagate new scientific discoveries probably explains some of our apparent deficit of current scientific geniuses, I think a bigger factor is just that earlier scientists ate the low-hanging fruit.
(I have no idea whether a similar effect would kick in for superintelligences and throttle them.)
An upvote is inadequate to express the degree of my agreement with this statement.
When I point out the low-hanging fruit effect to LWers, I do usually get a lot of agreement (and it is appreciated!) but I am starting to wish that someone would dig up some strong contrary evidence.
When the topic of apparent genius deficits and scientific stagnation comes up, people often present multiple explanations, like
but tend to present only anecdotal evidence for each — myself included. And I'm not sure that can be helped; I don't know of readily available evidence which powerfully discriminates between the different explanations.
PhilGoetz has data on scientific & technological progress, but I get the impression that much of it's basically time series of counts of inventions & discoveries, which would establish only the whats and not the whys. Likewise, I think I could substantiate my January comment that cohort explains a substantial part of the variation in scientific eminence. And when I scraped together the data, ran the big regression, and found that birth year accounted for (suppose) 30% of the variance in eminence, that wouldn't refute any of the potential explanations for why cohort correlated with eminence.
A partisan of the scaling hypothesis might say, "Obviously, as science gets bigger over time, it gets less efficient; more recently born scientists just lost the birth year draw".
Someone arguing that scientific stagnation is illusory might say, "Obviously, this is a side effect of overlooking more recent scientific geniuses; scientists are working as effectively as before but we don't recognize that thanks to increasing specialization, or our own complacency, or the difficulty of picking out individual drops from the flood of brilliance, or the fact that we only recognize greatness decades after the fact".
I would say, if I were the kind of person who threw the word "obviously" around willy-nilly, "How many times do you expect general relativity to be invented? Obviously, there are only so many simple but important problems to work on, and when we turn to much harder problems, we make slower and more incremental progress".
Someone most concerned with institutional degradation might say, "Obviously, as science has become more bureaucratic and centralized, that's rendered it more careerist, risk-averse & narrow-minded and less ambitious, so of course later generations of scientists would end up being less eminent, because they're not tackling big scientific questions like they did before".
And we don't get anywhere because each explanation is broadly consistent with the observed facts, and each seems obvious to someone.
I would put forth three lines of argument that might help.
First, what we consider a significant development is put in relation to its context. So, we naturally end up picking out the top-level entities and not the second-layer entities, let alone the third, fourth, fifth... modern science may have the same number of top-level discoveries, but these are underpinned by many more layers of discovery than earlier discoveries.
Second, let's stop thinking about the jump from Aristotle to modern science for a minute. Let's think about the jump from Novoselov and Geim's discovery of Graphene to today.
In their first paper, they made graphene, put it on a substrate, hooked a few wires up to it, and did low-temperature transport measurements. Worth a nobel prize. Outside of the insight that led to it, pretty simple. Not everyone could do it, but many could.
In the following years, a bunch of progressively trickier experiments were performed.
As of two years ago, our clearest path to a publishable research paper in this area was to make an enormous pristine sheet of graphene, position a layer of boron nitride on top of it, position another layer of graphene on top of that in such a way that it didn't short to the first piece, place a bunch of wires in very specific locations on this sandwich, then destroy the substrate that was all sitting on, all done so cleanly that it was smooth on every surface. This was insanely hard. This is also only a little trickier than normal for experiments in the field these days.
The low-hanging fruit has been taken, here. And it's not simply that other people took it and we're looking at sour grapes. I took some of that low-hanging fruit. There was a simple experiment to do, I did it, published it, and now I cannot do an experiment that simple again in this sub-field. My next experiment was substantially more complicated. The next experiment after that was far more complicated still.
Yes, we stand on the shoulders of giants, but we are in a progressively rarer atmosphere.
Third, and most critically, let's look at the predictions of the 'organizational inefficiency' theory. if we were to scale back our scientific establishment to 1700-s levels, do you think we'd maintain our current level of scientific progress? That seems to be the implication, here, and it seems VERY dubious to me.
Funding agencies these days fund people who get PhDs.
To get a PhD, 90% of the time you need to generate a meaningful result of some kind within a limited time horizon. Scientists who go big and create nothing do not graduate and do not get funded later.
What some of them do learn to do is to manage several projects at the same time. They diversify, working on some big ideas which may fail, but insuring a steady stream of results by also working through some lesser issues with a higher probability of success.
It's true that you can have a career stringing together nothing but small wins, but contrary to popular belief, the funding agencies (who rely on PhD scientific peer review committees) do fund many "high-risk, high-reward" projects.
In the private sector, for example, drug discovery projects have a vast failure rate but are funded nonetheless. Science funders do understand the biases toward career safety and are trying (imperfectly) to adjust for them.
I'd love to see that data & analysis! Did you post it somewhere? Can you email it to me at gmail?
I think there was a LW post years ago saying that the word "obviously" is only used to cover up the fact that something isn't obvious, and I agree with that more every year.
The evidence against the low-hanging fruit idea is that it explains only fame distribution across time, while the "attention and accretion model", which says that people gain fame in proportion to the fame they already have, and total fame in a field is constant, explains fame distribution at any given moment as well as across time. If you use "attention and accretion" to explain fame distribution in the present, you will end up also explaining its distribution across time, not leaving very much for low-hanging fruit to explain.
Of course it is possible that low-hanging fruit is a strong factor, being cancelled out by some opposing strong factor such as better knowledge and tools. In fact, I think an economic-style argument might say that people work on the highest-return problems until productivity drops below C, then work on tools until it rises just above C, then work on problems, etc. So we should expect rate of return on worked-on problems to be fairly constant over time.
I'm talking about a hypothetical analysis there. I haven't actually collected the data and put it through the grinder (at least not yet)!
Yeah, I'm trying to install mental klaxons that go off when I unreflectively write (or read) "obvious" or "obviously".
That's a fascinating result (although I'd wait for more details about the data & models involved before allocating the bulk of my probability mass to it). Does that mean our perception of fewer geniuses nowadays is merely because older geniuses grabbed most of the fame and left less of it for later geniuses? That's how it sounds to me but I may be over-interpreting.
Do we perceive there are fewer geniuses nowadays? I think we tend to pick the one thing somebody did in each generation or decade that seems most impressive, and call whoever did it an Einstein, with no idea how hard or easy it really was.
For instance, some people called Watson and Crick the great geniuses of the generation after Einstein, for figuring out the structure of DNA. Yet Watson and Crick were racing people all over the world to find the structure, because they knew anybody with the right tools would be able to figure it out within a few months. It required only basic competence.
(What's especially interesting about that case is that Watson and Crick both did things that showed genius after they were hailed as geniuses, given genius-level funding and freedom, and expected to do genius things. Were they geniuses all along (a low prior), did they develop genius in response to more-challenging conditions, or is funding and freedom more important than genius?)
Today, we've got genuine genius entrepreneurs like Sergei & Larry, Peter Thiel, and Elon Musk, yet the public thinks the great genius of that generation was Steve Jobs. Possibly because Apple spent many (dozen? hundred?) millions of dollars advertising Steve Jobs. Peter Thiel was never on a billboard.
My gut says there are fewer geniuses nowadays, although I don't really trust it on this one.
As for guts that aren't mine...Bruce G. Charlton. Gideon Rachman. Dean Keith Simonton, although he simultaneously argues that modern first-rate scientists, "[i]f anything", need "more raw brains". Cosma Shalizi, who I think is being serious there, not just florid.
I think there are certainly people who do that. There are people (not sure I can name any, but I'm sure they exist...Ray Kurzweil, maybe?) who are relentlessly upbeat about the march of scientific genius & progress, and people who just like jumping on hype bandwagons. There are also people with gloomier outlooks.
I don't intuitively think of "genius entrepreneurs" as a natural category...
That advertising (and similar hype) influences whom people think of as geniuses is a good point.