Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: PhilGoetz 21 October 2014 02:52:29AM *  5 points [-]

In order to model intelligence explosion, we need to be able to measure intelligence.

Describe a computer's power as <Memory, FLOPS>. What is the relative intelligence of these 3 computers?

  1. <M, S>
  2. <M, 2S>
  3. <2M, S>

Is 2 twice as smart as 1 because it can compute twice as many square roots in the same time? Is it smarter by a constant C, because it can predict the folding of a protein with C more residues, or can predict weather C days farther ahead?

If we want to ask where superintelligence of some predicted computational power will lie along the scale of intelligence we know from biology, we could look at evolution over the past 2 billion years, construct a table estimating how much computation evolution performed in each million years, and see how the capabilities of the organisms constructed scaled with computational power.

This would probably conclude that superintelligence will explode, because, looking only at more and more complex organisms, the computational power of evolution has decreased dramatically owing to larger generation times and smaller population sizes, yet the rate of intelligence increase has probably been increasing. And evolution is fairly brute-force as search algorithms go; smarter algorithms should have lower computational complexity, and should scale better as genome sizes increase.

Comment author: PhilGoetz 20 October 2014 06:46:03PM 2 points [-]

Therefore, they cannot identify "that computer running the code" with "me", and would cheerfully destroy themselves in the pursuit of their goals/reward.

Why do you think that? They would just use a deictic reference. That's what knowledge representation systems have done for the past 30 years.

Comment author: satt 19 October 2014 06:08:15PM 0 points [-]

when I scraped together the data, ran the big regression, and found that birth year accounted for (suppose) 30% of the variance in eminence, that wouldn't refute any of the potential explanations for why cohort correlated with eminence

I'd love to see that data & analysis! Did you post it somewhere? Can you email it to me at gmail?

I'm talking about a hypothetical analysis there. I haven't actually collected the data and put it through the grinder (at least not yet)!

I think there was a LW post years ago saying that the word "obviously" is only used to cover up the fact that something isn't obvious, and I agree with that more every year.

Yeah, I'm trying to install mental klaxons that go off when I unreflectively write (or read) "obvious" or "obviously".

The evidence against the low-hanging fruit idea is that it explains only fame distribution across time, while the "attention and accretion model", which says that people gain fame in proportion to the fame they already have, and total fame in a field is constant, explains fame distribution at any given moment as well as across time. If you use "attention and accretion" to explain fame distribution in the present, you will end up also explaining its distribution across time, not leaving very much for low-hanging fruit to explain.

That's a fascinating result (although I'd wait for more details about the data & models involved before allocating the bulk of my probability mass to it). Does that mean our perception of fewer geniuses nowadays is merely because older geniuses grabbed most of the fame and left less of it for later geniuses? That's how it sounds to me but I may be over-interpreting.

Comment author: PhilGoetz 20 October 2014 06:35:36PM *  1 point [-]

Does that mean our perception of fewer geniuses nowadays is merely because older geniuses grabbed most of the fame and left less of it for later geniuses?

Do we perceive there are fewer geniuses nowadays? I think we tend to pick the one thing somebody did in each generation or decade that seems most impressive, and call whoever did it an Einstein, with no idea how hard or easy it really was.

For instance, some people called Watson and Crick the great geniuses of the generation after Einstein, for figuring out the structure of DNA. Yet Watson and Crick were racing people all over the world to find the structure, because they knew anybody with the right tools would be able to figure it out within a few months. It required only basic competence.

(What's especially interesting about that case is that Watson and Crick both did things that showed genius after they were hailed as geniuses, given genius-level funding and freedom, and expected to do genius things. Were they geniuses all along (a low prior), did they develop genius in response to more-challenging conditions, or is funding and freedom more important than genius?)

Today, we've got genuine genius entrepreneurs like Sergei & Larry, Peter Thiel, and Elon Musk, yet the public thinks the great genius of that generation was Steve Jobs. Possibly because Apple spent many (dozen? hundred?) millions of dollars advertising Steve Jobs. Peter Thiel was never on a billboard.

Comment author: roystgnr 19 October 2014 03:46:18PM 1 point [-]

The use of science and technology isn't the same all over the world at a given time, but the availability is remarkably close, don't you think? What are the less developed countries left out on? ITAR-controlled products, trade secrets, and patents? For everything else they have access to the exact same journals.

Perhaps your side note is what's critical: are there organizational and management techniques which are available in the United States but which we've successfully kept a secret internationally? Are multi-generational trade secrets the critical part of science and technology?

Or would other countries grow much faster if they just fully used public domain technology, but there's some other factor X which is preventing them from using it? If the latter, then what is X, and wouldn't it be a better candidate explanation for disparate economic growth?

Comment author: PhilGoetz 20 October 2014 06:27:20PM *  2 points [-]

This is a reasonable observation; yes, it is not obvious why every nation can't jump straight to modern-nation productivity.

There are plenty of places in Africa where water purification is a great new technology, and plenty of places in China where closed sewage lines would be a great new technology. Why don't they use them?

The stories I hear from very-low-tech countries usually emphasize cultural resistance. One guy installed concrete toilets in Africa, and people wouldn't use them because concrete had negative connotations. People have tried plastic-water-bottle solar water purification in southeast Asia, and some concluded (according to Robin Hanson) that people wouldn't put plastic bottles of water on their roof because they didn't want the neighbors to know they didn't have purified water. Another culture wouldn't heat-sterilize water because their folk medicine was based on notions of what "hot" and "cold" do, and they believed sick people needed cold things, not hot things. There are many cases where people refused to believe there are invisible living things in water. (As Europeans also did at first.)

(Frequent hand-washing and checklists are technologies that could save many thousands of lives every year in US hospitals, but that are very difficult to get doctors to adopt.)

But lots of low-tech countries can't afford anything that they can't build themselves. How much of modern technology can be built with materials found on-site without any tools other than machetes, knives, and hammers? Mosquito netting is very valuable in some places, but impossible to manufacture in a low-tech way.

My short answer is that there are a variety of obstacles to applying any technology in a low-tech nation. But growth is only possible either by finding more resources, or by using existing resources more efficiently, and using resources more efficiently = technology.

If it were possible to have growth without technology--let's say 1% growth every 10 years--then a society with medieval technology, and no technological change, would eventually become as productive per person as today's modern countries. And that's physically impossible, just using energy calculations alone. There may be other necessary conditions, but tech improvement is absolutely necessary.

Comment author: Punoxysm 07 October 2014 11:37:30PM *  3 points [-]

The answer you just wrote could be characterized as a matrix of vocabulary words and index-of-occurrence. But that's a pretty poor way to characterize it for almost all natural language processing techniques.

First of all, something like PCA and the other methods you listed won't work on a ton of things that could be shoehorned into matrix format.

Taking an image or piece of audio and representing it using raw pixel or waveform data is horrible for most machine learning algorithms. Instead, you want to heavily transform it before you consider putting it into something like PCA.

A different problem goes for the matrix of neuronal connections in the brain: it's too large-scale, too sparse, and too heterogenous to be usefully analyzed by anything but specialized methods with a lot of preprocessing and domain knowledge going into them. You might be able to cluster different functional units of the brain, but as you tried to get to more granular units, heterogeneity in number of connnections per neuron would cause dense clusters to "absorb" sparser but legitimate clusters in almost all clustering methods. Working with a time-series of activations is an even bigger problem, since you want to isolate specific cascades of activations that correspond to a stimulus, and then look at the architecture of the activated part of the brain, characterize it, and then be able to understand things like which neurons are functionally equivalent but correspond to different parallel units in the brain (left eye vs. right eye).

If I give you a time series of neuronal activations and connections with no indication of the domain, you'd probably be able to come up with a somewhat predictive model using non-domain-specific methods, but you'd be handicapping yourself horribly.

Inferring causality is another problem - none of these predictive machine learning methods do a good job of establishing whether two factors have a causal relation, merely whether they have a predictive one (within the particular given dataset).

Comment author: PhilGoetz 18 October 2014 05:08:19PM *  0 points [-]

First, yes, I overgeneralized. Matrices don't represent natural language and logic well.

But, the kinds of problems you're talking about--music analysis, picture analysis, and anything you eventually want to put into PCA--are perfect for matrix methods. It's popular to start music and picture analysis with a discrete Fourier transform, which is a matrix operation. Or you use MPEG, which is all matrices. Or you construct feature detectors, say edge detectors or contrast detectors, using simple neural networks such as those found in primary visual cortex, and you implement them with matrices. Then you pass those into higher-order feature detectors, which also use matrices. You may break information out of the matrices and process it logically further downstream, but that will be downstream of PCA. As a general rule, PCA is used only on data that has so far existed only in matrices. Things that need to be broken out are not homogenous enough, or too structured, to use PCA on.

There's an excellent book called Neural Engineering by Chris Eliasmith in which he develops a matrix-based programming language that is supposed to perform calculations the way that the brain does. It has many examples of how to tackle "intelligent" problems with only matrices.

Comment author: ESRogs 14 October 2014 04:42:49PM 8 points [-]

yet we do not have 1000 Mozarts or 1000 Beethovens

What do you mean by this? We have plenty of composers and musicians today, and I'd bet that many modern prodigies can do the same kinds of technical tricks that Mozart could at a young age.

Comment author: PhilGoetz 18 October 2014 04:57:28PM *  1 point [-]

Good question, though doing technical tricks at a young age does not make one Mozart. I don't mean that we don't have 1000 composers as good as Mozart or Beethoven. I mean we don't have 1000 composers recognized as being that good. We may very well have 10,000 composers better than Mozart, but we're unable to recognize that many good composers.

This is conflated with questions of high versus pop art andd accidents of history. Personally, I'm open to the idea that Mozart represents a temporary decline in musical taste--a period between baroque and romantic when people ate up the kind of pleasant, predictable pop music that Mozart churned out. He wrote some great stuff, but I think the bulk of what he wrote is soulless compared to equally-prominent baroque or romantic music.

Comment author: satt 15 October 2014 03:05:18AM 6 points [-]

When I point out the low-hanging fruit effect to LWers, I do usually get a lot of agreement (and it is appreciated!) but I am starting to wish that someone would dig up some strong contrary evidence.

When the topic of apparent genius deficits and scientific stagnation comes up, people often present multiple explanations, like

  1. intrinsic difficulties in scaling scientific activity
  2. failure to identify/recognize contemporary scientific successes
  3. no more low-hanging fruit
  4. bureaucratization and institutional degradation

but tend to present only anecdotal evidence for each — myself included. And I'm not sure that can be helped; I don't know of readily available evidence which powerfully discriminates between the different explanations.

PhilGoetz has data on scientific & technological progress, but I get the impression that much of it's basically time series of counts of inventions & discoveries, which would establish only the whats and not the whys. Likewise, I think I could substantiate my January comment that cohort explains a substantial part of the variation in scientific eminence. And when I scraped together the data, ran the big regression, and found that birth year accounted for (suppose) 30% of the variance in eminence, that wouldn't refute any of the potential explanations for why cohort correlated with eminence.

A partisan of the scaling hypothesis might say, "Obviously, as science gets bigger over time, it gets less efficient; more recently born scientists just lost the birth year draw".

Someone arguing that scientific stagnation is illusory might say, "Obviously, this is a side effect of overlooking more recent scientific geniuses; scientists are working as effectively as before but we don't recognize that thanks to increasing specialization, or our own complacency, or the difficulty of picking out individual drops from the flood of brilliance, or the fact that we only recognize greatness decades after the fact".

I would say, if I were the kind of person who threw the word "obviously" around willy-nilly, "How many times do you expect general relativity to be invented? Obviously, there are only so many simple but important problems to work on, and when we turn to much harder problems, we make slower and more incremental progress".

Someone most concerned with institutional degradation might say, "Obviously, as science has become more bureaucratic and centralized, that's rendered it more careerist, risk-averse & narrow-minded and less ambitious, so of course later generations of scientists would end up being less eminent, because they're not tackling big scientific questions like they did before".

And we don't get anywhere because each explanation is broadly consistent with the observed facts, and each seems obvious to someone.

Comment author: PhilGoetz 18 October 2014 04:49:49PM *  0 points [-]

when I scraped together the data, ran the big regression, and found that birth year accounted for (suppose) 30% of the variance in eminence, that wouldn't refute any of the potential explanations for why cohort correlated with eminence

I'd love to see that data & analysis! Did you post it somewhere? Can you email it to me at gmail?

I think there was a LW post years ago saying that the word "obviously" is only used to cover up the fact that something isn't obvious, and I agree with that more every year.

The evidence against the low-hanging fruit idea is that it explains only fame distribution across time, while the "attention and accretion model", which says that people gain fame in proportion to the fame they already have, and total fame in a field is constant, explains fame distribution at any given moment as well as across time. If you use "attention and accretion" to explain fame distribution in the present, you will end up also explaining its distribution across time, not leaving very much for low-hanging fruit to explain.

Of course it is possible that low-hanging fruit is a strong factor, being cancelled out by some opposing strong factor such as better knowledge and tools. In fact, I think an economic-style argument might say that people work on the highest-return problems until productivity drops below C, then work on tools until it rises just above C, then work on problems, etc. So we should expect rate of return on worked-on problems to be fairly constant over time.

Comment author: whales 14 October 2014 07:23:33AM 5 points [-]

I measured science and technology output per scientist using four different lists of significant advances, and found that significant advances per scientist declined by 3 to 4 orders of magnitude from 1800 to 2000. During that time, the number of scientific journals has increased by 3 to 4 orders of magnitude, and a reasonable guess is that so did the number of scientists.

I'd be really interested in reading more about this.

Comment author: PhilGoetz 18 October 2014 04:13:22PM 1 point [-]

If you email philgoetz at gmail, I'll send you a draft.

During that time, the number of scientific journals has increased by 3 to 4 orders of magnitude, and a reasonable guess is that so did the number of scientists.

Er, or not. The number of publications per scientist has risen dramatically, but so has the number of authors per paper. I don't know if these cancel each other out.

Comment author: roystgnr 14 October 2014 03:43:51PM 3 points [-]

Economic growth comes (I believe) exclusively from advances in science and technology.

This alone doesn't seem sufficient to explain the distribution of economic growth between countries. Most science, and most technology more than a generation old, is now public domain. But even if we go two generations back, US GDP/capita was ~$25K, which would still put it in the top quartile of modern countries. The countries at the bottom of the economic lists are often catching up, but not uniformly.

Comment author: PhilGoetz 18 October 2014 04:09:07PM 1 point [-]

You're making some argument that you think is implied by what you've said, but that I can't see. I don't see how the US of 2 generations ago having a high GDP is inconsistent with growth being a result of science and technology, unless you imagine science and technology are the same all over the world at a given time, which would be a strange thing to imagine.

(Side note: "Technology" here include organizational and management techniques.)

Comment author: KatjaGrace 14 October 2014 03:39:25AM *  6 points [-]

If we achieved a linear relationship between input and output, we would have maybe 6 orders of magnitude more important scientific and technological advances per year. If we actually achieved "synergy", that oft-theorized state where the accumulation of knowledge grows at a rate proportional to accumulated knowledge, we would have a fast take-off scenario, just without AI.

How much should the fact that we do not have a fast take-off of organizations make us more pessimistic about one with AIs being likely?

Comment author: PhilGoetz 14 October 2014 12:52:04PM *  4 points [-]

That's the question. We should consider the overhead cost of knowledge, and the possibility that we will see a logarithmic increase in knowledge instead of a linear one (or, that we will see a linear one given an exponential explosion in resources).

Much depends on how you measure knowledge. If you count "bits of information", that's still growing exponentially. If you count "number of distinctions or predictions you can make in the world", that probably isn't.

There is a critical relationship between GDP and the efficiency of science. Until 1970, the money we put into science increased exponentially. Economic growth comes (I believe) exclusively from advances in science and technology. In 1970, we hit the ceiling; fraction of GDP spent on science had grown exponentially until then, when it suddenly flattened, so that now resources spent on science grows only as fast as GDP grows. This should cause a slower growth of GDP, causing a slower increase in scientific results, etc. IIRC there's a threshold of scientific efficiency below which (theoretically) the area under the curve giving scientific results off to infinity is finite, and another threshold of efficiency above which (theoretically) the curve rises exponentially.

View more: Next