The number of possible patterns in an information cluster is superexponential with the size of the information cluster
Firstly, you are misquoting EY's post: the possible number of patterns in a string grows exponentially with the number of bits, as expected. It is the number of 'concepts' which grows super-exponentially, where EY is defining concept very loosely as any program which classifies patterns. The super-exponential growth in concepts is combinatoric and just stems from naive specific classifiers which recognize combinations of specific patterns.
Secondly, this doesn't really relate to universal pattern recognition, which is concerned only with optimal data classifications according to a criteria such as entropy maximization.
As a simple example, consider the set of binary strings of length N. There are 2^N possible observable strings, and a super-exponential combinatoric set of naive classifiers. But consider observed data sequences of the form 10010 10010 10010 repeated ad infinitum. Any form of optimal extropy maximization will reduce this to something of the form repeat "10010" indefinitely.
In general any given sequence of observations has a single unique compressed (extropy reduced) representation, which corresponds to it's fundamental optimal 'pattern' representation.
Can you demonstrate that the patterns you're recognizing are non-arbitrary?
Depends on what you mean. It's rather trivial to construct simple universal extropy maximizers/optimizers - just survey the basic building blocks of unsupervised learning algorithms. The cortical circuit performs similar computations.
For example the 2D edge patterns that cortical tissue (and any good unsupervised learning algorithm) learns to represent when exposed to real world video are absolutely not arbitrary in the slightest. This should be obvious.
If you mean higher level thought abstractions by "the patterns you're recognizing", then the issue becomes more complex. Certainly the patterns we currently recognize at the highest level are not optimal extractions, if that's what you mean. But nor are they arbitrary. If they were arbitrary our cortex would have no purpose, would confer no selection advantage, and would not exist.
We don't have a fully general absolute pattern recognizing system;
We do have a fully general pattern recognition system. I'm not sure what you mean by "general absolute".
that would be an evolutionary hindrance even if it were something that could practically be developed.
They are trivial to construct, and require far less genetic information to specify than specific pattern recognition systems.
Specific recognition systems have the tremendous advantage that they work instantly without any optimization time. A general recognition system has to be slowly trained on the patterns of data present in the observations - this requires time and lots of computation.
Simpler short lived organisms rely more on specific recognition systems and circuitry for this reason as they allow newborn creatures to start with initial 'pre-programmed' intelligence. This actual requires considerably more genetic complexity than general learning systems.
Mammals grew larger brains with increasing reliance on general learning/recognition systems because it provides a tremendous flexibility advantage at the cost of requiring larger brains, longer gestation, longer initial development immaturity, etc. In primates and humans especially this trend is maximized. Human infant brains have very little going on initially except powerful general meta-algorithms which will eventually generate specific algorithms in response to the observed environment.
I think we don't agree on what this "complexity" is because it's not a natural category
The concept of "natural category" is probably less well defined that "complexity" itself, so it probably won't shed too much light on our discussion.
That being said, from that post he describes it as:
I've chosen the phrase "unnatural category" to describe a category whose boundary you draw in a way that sensitively depends on the exact values built into your utility function.
In that sense complexity is absolutely a natural category.
Look at Kolmogorov_complexity. It is a fundamental computable property of information, and information is the fundamental property of modern physics. So that definition of complexity is as natural as you can get, and is right up there with entropy. Unfortunately that definition itself is not perfect and is too close to entropy, but computable variants of it exist .. .. one used in a computational biology paper I was browsing recently (measuring the tendency towards increased complexity in biological systems) defined complexity as compressed information minus entropy, which may be the best fit to the intuitive concept.
Intuitively I could explain it as follows.
The information complexity of an intelligent system is a measure of the fundamental statistical pattern structure it extracts from it's environment. If the information it observes is already at maximum entropy (such as pure noise), then it is already maximally compressed, no further extraction is possible, and no learning is possible. At the other extreme if the information observed is extremely uniform (low entropy) then it can be fully described/compressed by extremely simple low complexity programs. A learning system extracts entropy from it's environment and grows in complexity in proportion.
Depends on what you mean. It's rather trivial to construct simple universal extropy maximizers/optimizers - just survey the basic building blocks of unsupervised learning algorithms. The cortical circuit performs similar computations.
For example the 2D edge patterns that cortical tissue (and any good unsupervised learning algorithm) learns to represent when exposed to real world video are absolutely not arbitrary in the slightest. This should be obvious.
It's objective that our responses exist, and they occur in response to particular things. It's not ob...
Genesis 11: 1-9
Some elementary physical quantitative properties of systems compactly describe a wide spectrum of macroscopic configurations. Take for example the concept of temperature: given a basic understanding of physics this single parameter compactly encodes a powerful conceptual mapping of state-space.
It is easy for your mind to visualize how a large change in temperature would effect everything from your toast to a planetary ecosystem. It is one of the key factors which divides habitable planets such as Earth from inhospitably cold worlds like Mars or burning infernos such as Venus. You can imagine the Earth growing hotter and visualize an entire set of complex consequences: melting ice caps, rising water levels, climate changes, eventual loss of surface water, runaway greenhouse effect and a scorched planet.
Here is an unconsidered physical parameter that could determine much of the future of civilization: the speed of thought and the derived subjective speed of light.
The speed of thought is not something we are accustomed to pondering because we all share the same underlying neurological substrate which operates at a maximum frequency of around a kilohertz, and appears to have minor and major decision update cycles at rates in the vicinity of 33hz to 3hz.1
On the other hand the communication delay has changed significantly over the last ten thousand years as we evolved from hunter-gatherer tribes to a global civilization.
For much of early human history, the normal instantaneous communication distance limit would be the audible range of about 100 feet, and long distance communication consisted of sending physical human messengers; a risky endeavor that could take months to traverse a continent.
The long distance communication delay in this era (on the order of months) was more than 10^9 times the baseline thought-frequency (which is around a millisecond). The developmental outcome in this type of regime is divergence. New ideas and slight mutations of existing beliefs are generated in local ingroups far faster than they can ever propagate to remote outgroups.
In the divergent regime cultures fragment into sub-cultures; languages split into dialects; and dialects become new languages and cultures as groups expand geographically.2
Over time a steady accumulation of technological developments increased subjective bandwidth and reduce subjective latency in the global human network: the advent of agricultural civilization concentrated human populations into smaller regions, the domestication of horses decreased long distance travel time, books allowed stored communication from the past, and the printing press provided an efficient one to many communication amplifier.
Yet despite all of this progress, even as late as the mid 19th century the pony express was considered fast long distance communication. It was not until just very recently in the 20th century that near instantaneous long distance communication became relatively cheap and widespread.3
Today the communication delay for typical point to point communication around the world is somewhere around 200 to 300 ms, corresponding to a low delay/thought-frequency ratio of 10^2. This figure is close enough to the brain's natural update cycles to permit real time communication.
It is difficult to measure, but the general modern trend seems to have now finally shifted towards convergence rather than divergence. Enough people are moving between cultures, translating between languages and communicating new ideas fast enough relevant to the speed of thought to largely counter the tendency toward divergence.
But now consider that our global computational network consists of two very different substrates: the electronic substrate which operates at near-light speed, and a neural substrate which operates at much slower chemical speeds; more than one million times slower.
At the moment the vast majority of the world's knowledge and intelligence is encoded in the larger and slower neural substrate, but the electronic substrate is growing exponentially at a vastly faster pace.
Viewed as a single global cybernetic computational network we can see there is massive discrepancy between the neural and electronic sub-components.
So what happens when we shift completely to the electronic, when we have artificial brains and AGI's that think at full electronic speeds?
The speed of light measured in atomic seconds is the same for all physical frames of reference, but it's subjective speed varies based on one's subjective speed of thought. This subjective relativity causes effective time dilation proportional to one's level of acceleration.
For an AGI or upload that has an architecture similar to the brain but encoded in the electronic substrate using high effeciency neuromorphic circuitry, thoughts could be computed in around a thousand clock cycles or less at a rate of billions of clock cycles per second.
Such a Mind would experience a million fold time dilation, or an entire subjective year every thirty seconds.
Imagine the external universe, time itself, slowing down by a factor of a million. Watching a human walk to work would be similar to us watching grass grow. Actually it would be considerably worse; five minutes would correspond to an unimaginable decade of subjective time for an acceleration level 6 hyperintelligence.
A bullet would not appear to be much faster than a commuter, and the speed of light itself, the fastest signal propagation in the universe, would be slowed down to just 300 subjective meters per second, roughly the speed of a jetliner.
Real-time communication would thus only be possible with entities in the same building and on the same local network.
It would take a subjective day or two to reach distant external internet sites. Browsing the web would not be possible in the conventional sense. It would appear the only viable strategy would be to copy most of the internet into a local cache. But even this would be impeded by the million fold subjective bandwidth slowdown.
Today's fastest gigabyte direct ethernet backbone connections would be reduced back down to mere kilobyte per second modem speeds. A cable modem connection speed would require about as much fiber bandwidth as our entire current transatlantic fiber capacity.
Acceleration level 6 corresponds to a 10^8 value for the communication delay / thoughtspeed ratio, a shift backwards roughly equivalent to the era before the advent of the telegraph. This is the historical domain of both the Roman Empire and pre civil war America.
If Moore's Law continues well into the next decade, further levels of acceleration will be possible. A combination of denser circuitry, architectural optimizations over the brain and higher clock rates could lead to acceleration level 9 hyperintelligences. Overclocked circa 2011 CPUs are already approaching 10 GHZ, and test transistors have achieved speeds into the terrahertz range in the lab.4
The brain takes about 1000 'clocks' of the base neuron frequency to compute one second worth of thought. If a future massively dense and parallel neuromorphic architecture could do the same work 10 times more effeciently and thus compute one second of thought in 100 clock cycles while running at 100 GHZ this would enable acceleration level 9.5
Acceleration level 9 stretches the limits of human imagination. It's difficult to conceive of an intelligence that experiences around 30 years in just one second, or a billion subjective years for every sidereal year.
At this dilation factor light slows to just 300 centimeters per second, a slow walking pace. More crucially, light moves just 3 centimeters per clock cycle, which would place serious size constraints on the physical implementation of a single mind. To make integrated decisions with a unified knowledge base, in other words think in how we understand the term, the core of a Mind running at these speeds would have to be crammed into the space of a modern desktop box. (although it certainly could have a larger secondary knowledge store accessible with some delay)
The small size constraint would severely limit how much power/heat one could throw at the problem, and thus these high speeds will probably require much higher circuit densities to achieve the required energy efficiency than implied by memory requirements alone.
With light itself crawling along at 300 centimeters per second it would take data packets hundreds of millions of seconds, or on the order of years, to make typical transits across the internet. These speeds are already close to physical limits; even level 9 hyperintelligences will probably not be able to surmount the speed of light delay.
The entire fiber backbone of the circa 2011 transatlantic connection would be required to achieve end 20th century dialup modem speeds.6
Even using all of that fiber it would take on the order of ten physical seconds to transfer a 10^14 byte Mind, corresponding to hundreds of thousands of subjective years.
A level 9 world is one where the subjective communication delay, approaching 10^11, is a throwback to the prehistoric era. Strong Singletons and even weaker systems such as global governments or modern markets would be unlikely or impossible at such high levels of acceleration.7
From the social and cultural perspective high levels of thought acceleration are structurally equivalent to the world expanding to billions of times it's current size.
It is similar to the earth exploding into an intergalactic or hyperdimensional civilization linked together by a vast impossibly slow lightspeed transit network.
Entire new cultures and civilizations would form and play out complex histories in the blink of an eye.
With every increase in circuit density and speed the new metaverse will vasten exponentially in virtual space and time just as it physically shrinks and quickens down into the ever smaller, faster levels of the real.
And although all of this change will be unimaginably fast for a biological human, Moore's Law will be a distant ancestral memory for level 9 intelligences, as it depends on a complex series of events in the impossibly slow physical world of matter. Even if an entire new hardware generation transition could be compressed into just 8 hours of physical time through nanotechnological miracles, that's still an unimaginable million years of subjective time at acceleration level 9.
Another interesting subjective difference: computer speed or performance will not change much from the inside perspective of a hyperintelligence running on the same hardware. Traditional computers will indefinitely maintain roughly the same subjective slow speeds for minds running on the same substrate at those same speeds. Density shrinkings will enable more and or larger minds; but only a net shift towards the latter would entail a net increase in traditional parallel CPU performance available per capita. But as discussed previously, speed of light delays severely constrain the size of large unified minds.
The radical space-time compression of the Metaverse Singularity model suggests a reappraisal of the Fermi Paradox and the long-term fate of civilizations.
The speed of light barrier gives a natural gradient to the expansion of complexity: it is inwards, not outwards.
Humanity today could mount an expedition to a nearby solar system, but the opportunity cost of such an endeavor vastly exceeds any realistic discounted returns. The incredible resources space colonization would require are much better put to use increasing our planetary intelligence through investing in further semiconductor technology.
This might never change. Indeed such a change would be a complete reversal of the general universal trend towards smaller, faster complexity.
Each transition to a new level of acceleration and density will increase the opportunity cost of expansion in proportion. Light-years are vast units of space-time for humans today, but they are unimaginably vaster for future accelerated hyperintelligences.
Facing the future it appears that looking outwards into space is looking into the past, for the future lies in innerspace, not outerspace.
Notes
1 Human neuron action potentials have a measured maximum frequency of a little less than a millisecond. This is thus one measure of rough equivalence to the clock frequency in a digital circuit, but it is something of a conservative over-estimate as neurological circuits are not synchronous at that frequency. Many circuits in the brain are semi-synchronized over longer intervals roughly corresponding to the various measured 'brain wave' frequencies, and neuron driven mechanisms such as voice have upper frequencies of the same order. Humans can react in as quickly as 150ms in some conditions, but appear to initiate actions such as saccades at a rate of 3 to 4 per second. Smaller primate brains are similar but somewhat quicker.
2 The greater monogenesis theory of all extant languages and cultures from a single distant historical proto-language is a matter of debate amongst linguistics, but the similarity in many low-level root words is far beyond chance. The restrained theory of a common root Proto-Indo-European language is near universally accepted. This map and this tree help visualize the geographical historical divergence of this original language/cultural across the supercontinent along with it's characteristic artifact: the chariot. All of this divergence occurred on a timescale of five to six millenia.
3 Homing pigeons, where available, were of course much faster than the pony express, but were rare and low-bandwidth.
4 Apparently this has been done numerous times in the last decade in different ways. Here is one example. Of course making a few transistors run in the terahertz doesn't get you much closer to making a whole CPU actually run at that speed, for a large variety of reasons.
5 None of these particular numbers will seem outlandish a decade or two from now if Moore's Law holds it's pace. However getting a brain or AGI type design to run at these fantastic speeds will likely require more significant innovations such as a move to 3D integrated circuits and major interconnect breakthroughs. There are many technological uncertainties here, but less than that involved in drexler-style nano-tech, and this is all on the current main path.
6 It looks like we currently have around 8 tbps of transatlantic bandwidth circa 2011.
7 Nick Bostrom seems to have introduced the Singleton concept to the Singularity/Futurist discourse here. He mentions artificial intelligences as one potential Singleton promoting technology but doesn't consider their speed potential with respect to the speed of light.