= 27d567bc2d48d9f40e9ccc20ea370b15)
JamesAndrix comments on Dreams of AIXI - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (145)
On what basis will they learn? You're still starting out with an initial value system and process for changing the value system, even if the value system is empty. There is no reason to think that a given preference-modifier will match humanity's. Why will they find "Because that hurts me" to be a valid point? Why will they return kindness with kindness?
You say the goal systems can't be designed in, why not?
It may be the case that we will have a wide range of semifriendly subhuman or even near human AGI's. But when we get a superhuman AGI that is smart enough to program better AGI, why can it not do that on it's own?
I am positive that 'single entity' should not have mapped to 'big distributed global brain'.
But I also think an AIXI like algorithm would be easy to parallelize and make globally distributed, and it still maximizes a single reward function.
They will have to learn by amassing a huge amount of observations and interactions, just as human infants do, and just as general agents do in AI theory (such as AIXI).
Human brains are complex, but very little of that complexity is actually precoded in the DNA. For humans values, morals, and high level goals are all learned knowledge, and have varied tremendously over time and cultures.
Well, if you raised the AI as such, it would.
Consider that a necessary precursor of of following the strategy 'returning kindness with kindness' is understanding what kindness itself actually is. If you actually map out that word, you need a pretty large vocabulary to understand it, and eventually that vocabulary rests on grounded verbs and nouns. And to understand those, they must be grounded on a vast pyramid of statistical associations acquired from sensorimotor interaction (unsupervised learning .. aka experience). You can't program in this knowledge. There's just too much of it.
From my understanding of the brain, just about every concept has (or can potentially have) associated hidden emotional context: "rightness" and "wrongness", and those concepts: good, bad, yes, no, are some of the earliest grounded concepts, and the entire moral compass is not something you add later, but is concomitant with early development and language acquisition.
Will our AI's have to use such a system as well?
I'm not certain, but it may be such a nifty, powerful trick, that we end up using it anyway. And even if there is another way to do that is still efficient, it may be that you can't really understand human languages unless you also understand the complex web of value. If nothing else, this approach certainly gives you control over the developing AI's value system. It appears for human minds the value system is immensely complex - it is intertwined at a fundamental level with the entire knowledge base - and is inherently memetic in nature.
What is an AGI? It is a computer system (hardware), some algorithms/code (which actually is always eventually better to encode directly in hardware - 1000X performance increase), and data (learned knowledge). The mind part - all the qualities of importance, comes solely from the data.
So the 'programming' of the AI is not that distinguishable from the hardware design. I think AGI's will speed this up, but not nearly as dramatically as people here think. Remember humans don't design new computers anymore anyway. Specialized simulation software does the heavy lifting - and it is already the bottleneck. An AGI would not be better than this specialized software at its task (generalized vs specialized). It will be able to improve it some almost certainly, but only to the theoretical limits, and we are probably already close enough to them that this improvement will be minor.
AGI's will have a speedup effect on moore's law, but I wouldn't be surprised if this just ends up compensating for the increased difficulty going forward as we approach quantum limits and molecular computing.
In any case, we are simulation bound already and each new generation of processors designs (through simulation) the next. The 'FOOM' has already begun - it began decades ago.
Well I'm pretty certain that AIXI like algorithms aren't going to be directly useful - perhaps not ever, only more as a sort of endpoint on the map.
But that's beside the point.
If you actually use even a more practical form of that general model - a single distributed AI with a single reward function and decision system, I can show you how terribly that scales. Your distributed AI with a million PC's is likely to be less intelligent than a single AI running on tightly integrated workstation class machine with just say 100x the performance of one of your PC nodes. The bandwidth and the latency issues are just that extreme.
If concepts like kindness are learned with language and depend on a hidden emotional context, then where are the emotions learned?
What is the AI's motivation? This is related to the is-ought problem: no input will affect the AI's preferences unless there is something already in the AI that reacts to that input that way.
If software were doing the heavy lifting, then it would require no particular cleverness to be a microprocessor design engineer.
The algorithm plays a huge role in how powerful the intelligence will be, even if it is implemented in silicon.
People might not make most of the choices in laying out chips, but we do almost all of the algorithm creation, and that is where you get really big gains. see Deep Fritz vs. Deep Blue. Better algorithms can let you cut out a billion tests and output the right answer on the first try, or find a solution you just would not have found with your old algorithm.
Software didn't invent out of order execution. It just made sure that the design actually worked.
As for the distributed AI: I was thinking of nodes that were capable of running and evaluating whole simulations, or other large chunks of work. (Though I think superintelligence itself doesn't require more than a single PC.)
In any case, why couldn't your supercomputer foom?
I think this is an open question, but certainly one approach is to follow the brain's lead and make a system that learns its ethics and high level goals dynamically, through learning.
In that type of design, the initial motivation gets imprinting queues from the parents.
Oh of course, but I was just pointing out that after a certain amount of research work in a domain, your algorithms converge on some asymptotic limit for the hardware. There is nothing even close to unlimited gains purely in software.
And the rate of hardware improvement is limited now by speed of simulation on current hardware, and AGI can't dramatically improve that.
Yes, of course. Although as a side note we are moving away from out of order execution at this point.
Because FOOM is just exponential growth, and in that case FOOM is already under way. It could 'hyper-FOOM', but the best an AGI can do is to optimize its brain algorithms down to the asymptotic limits of its hardware, and then it has to wait with everyone else until all the complex simulations complete and the next generation of chips come out.
Now, all that being said, I do believe we will see a huge burst of rapid progress after the first human AGI is built, but not because that one AGI is going to foom by itself.
The first human-level AGI's will probably be running on GPUs or something similar, and once they are proven and have economic value, there will be this huge rush to encode those algorithms directly in to hardware and thus make them hundreds of times faster.
So I think from the first real-time human-level AGI it could go quickly to 10 to 100X AGI (in speed) in just a few years, along with lesser gains in memory and other IQ measures.
This seems like a non-answer to me.
You can't just say 'learning' as if all possible minds will learn the same things from the same input, and internalize the same values from it.
There is something you have to hardcode to get it to adopt any values at all.
Well, what is that limit?
It seems to me that an imaginary perfectly efficient algorithm would read process and output data as fast as the processor could shuffle the bits around, which is probably far faster than it could exchange data with the outside world.
Even if we take that down 1000x becsaue this is an algorithm that's doing actual thinking, you're looking at an easy couple of million bytes per second. And that's superintelligently optimized structured output based on preprocessed efficient input. Because this is AGI, we don't need to count in say, raw video bandwidth, because that can be preprocessed by a system that is not generally intelligent.
So a conservatively low upper limit for my PC's intelligence is outputting a million bytes per second of compressed poetry, or viral genomes, or viral genomes that write poetry.
If the first Superhuman AGI is only superhuman by an order of magnitude or so, or must run on a vastly more powerful system, then you can bet that it's algorithms are many orders of magnitude less efficient than they could be.
No.
Why couldn't your supercomputer AGI enter into a growth phase higher than exponential?
Example: If not-too-bright but technological aliens saw us take a slow general purpose computer, and then make a chip that worked 100 times faster, but they didn't know how to put algorithms on a chip, then it would look like our technology got 1000 times better really quickly. But that's just because they didn't already know the trick. If they learned the trick, they could make some of their dedicated software systems work 1000 times faster.
"Convert algorithm to silicon." is just one procedure for speeding things up that an agent can do, or not yet know how to do. You know it's possible, and a superintelligence would figure it out, but how do you rule out a superintelligence figureing out twelve trick like that, which each provide a 1000x speedup. In it's first calendar month?
Yes, you have to hardcode 'something', but that doesn't exactly narrow down the field much. Brains have some emotional context circuitry for reinforcing some simple behaviors (primary drives, pain avoidance, etc), but in humans these are increasingly supplanted and to some extent overridden by learned beliefs in the cortex. Human values are thus highly malleable - socially programmable. So my comment was "this is one approach - hardcode very little, and have all the values acquired later during development".
Unfortunately, we need to be a little more specific than imaginary algorithms.
Computational complexity theory is the branch of computer science that deals with the computational costs of different algorithms, and specifically the most optimal possible solutions.
Universal intelligence is such a problem. AIXI is an investigation into optimal universal intelligence in terms of the upper limits of intelligence (the most intelligent possible agent), but while interesting, it shows that the most intelligent agent is unusably slow.
Taking a different route, we know that a universal intelligence can never do better in any specific domain than the best known algorithm for that domain. For example, an AGI playing chess could do no better than just pausing its AGI algorithm (pausing its mind completely) and instead running the optimal chess algorithm (assuming that the AGI is running as a simulation on general hardware instead of faster special-purpose AGI hardware).
So there is probably an optimal unbiased learning algorithm, which is the core building block of a practical AGI. We don't know for sure what that algorithm is yet, but if you survey the field, there are several interesting results. The first thing you'll see is that we have a variety of hierarchical deep learning algorithms now that are all pretty good, some appear to be slightly better for certain domains, but there is not atm a clear universal winner. Also, the mammalian cortex uses something like this. More importantly, there is alot of recent research, but no massive breakthroughs - the big improvements are coming from simple optimization and massive datasets, not fancier algorithms. This is not definite proof, but it looks like we are approaching some sort of bound for learning algorithms - at least at the lower levels.
There is not some huge space of possible improvements, thats just not how computer science works. When you discover quicksort and radix sort, you are done with serial sorting algorithms. And then you find the optimal parallel variants, and sorting is solved. There are no possible improvements past that point.
Computer science is not like moore's law at all. Its more like physics. There's only so much knowledge, and so many breakthroughs, and at this point alot of it honestly is already solved.
So its just pure naivety to think that AGI will lead to some radical recursive breakthrough in software. poppycock. Its reasonably likely humans will have narrowed in on the optimal learning algorithms by the time AGI comes around. Further improvements will be small optimizations for particular hardware architectures - but thats really not much different at all then hardware design itself, and eventually you want to just burn the universal learning algorithms into the hardware (as the brain does).
Hardware is quite different, and there is a huge train of future improvements there. But AGI's impact there will be limited by computer speeds! Because you need regular computers running compilers and simulators to build new programs and new hardware. So AGI can speed Moore's Law up some, but not dramatically - an AGI that thought 1000x faster than a human would just spend 1000x longer waiting for its code to compile.
I am a software engineer, and I spend probably about 30-50% of my day waiting on computers (compiling, transferring, etc). And I only think at human speeds.
AGI's will soon have a massive speed advantage, but ironically they will probably leverage that to become best selling authors, do theoretical physics and math, and non-engineering work in general where you don't need alot of computation.
Say you had an AGI that thought 10x faster. It would read and quickly learn everything about its own AGI design, software, etc etc. It would get a good idea of how much optimization slack there was in its design and come up with a bunch of ideas. It could even write the code really fast. But unfortunately it would still have to compile it and test it (adding extra complexity in that this is its brain we are talking about).
Anyway, it would only be able to get small gains from optimizing its software - unless you assume the human programmers were idiots. Maybe a 2x speed gain or something - we are just throwing numbers out, but we have a huge experience with real-time software on fixed hardware in say the video game industry (and other industries) and this asymptotic wall is real, and complexity theory is solid.
Big gains necessarily must come from hardware improvements. This is just how software works - we find optimal algorithms and use them, and further improvement without increasing the hardware hits an asymptotic wall. You spend a few years and you get something 3x better, spend 100 more and you get another 50%, and spend 1000 more and get another 30% and so on.
EDIT: After saying all this, I do want to reiterate that I think there could be a quick (even FOOMish) transition from the first AGIs to AGI's that are 100-1000x or so faster thinking, but the constraint on progress will quickly be the speed of regular computers running all the software you need to do anything in the modern era. Specialized software already does much of the heavy lifting in engineering, and will do even more of it by the time AGI arrives.
Hardcode very little?
What is the information content of what an infant feels when it is fed after being hungry?
I'm not trying to narrow the feild, the feild is always narrowed to whatever learning system an agent actually uses. In humans, the system that learns new values is not generic
Using a 'generic' value learning system will give you an entity that learns morality in an alien way. I cannot begin to guess what it would learn to want.
I'd like to table the intelligence explosion portion of this discussion, I think we agree that an AI or group of AI's could quickly grow powerful enough that they could take over, if that's what they decided to do. So establishing their values is important regardless of precisely how powerful they are.
Yes. The information in the genome, and the brain structure coding subset in particular, is a tiny tiny portion of the information in an adult brain.
An infant brain is mainly an empty canvas (randomized synaptic connections from which learning will later literally carve out a mind) combined with some much simpler, much older basic drives and a simpler control system - the old brain - that descends back to the era of reptiles or earlier.
That depends on what you mean by 'values'. If you mean linguistic concepts such as values, morality, kindness, non-cannibalism, etc etc, then yes, these are learned by the cortex, and the cortex is generic. There is a vast weight of evidence for almost overly generic learning in the cortex.
Not at all. To learn alien morality, it would have to either invent alien morality from scratch, or be taught alien morality from aliens. Morality is a set of complex memetic linguistic patterns that have evolved over long periods of time. Morality is not coded in the genome and it does not spontaneously generate.
Thats not to say that there are no genetic tweaks to the space of human morality - but any such understanding based on genetic factors must also factor in complex cultural adaptations.
For example, the Aztecs believed human sacrifice was noble and good. Many Spaniards truly believed that the Aztecs were not only inhuman, but actually worse than human - actively evil, and truly believed that they were righteous in converting, conquering, or eliminating them.
This mindspace is not coded in the genome.
Agreed.
I'm not saying that all or even most of the information content of adult morality is in the genome. I'm saying that the memetic stimulus that creates it evolved with hooks specific to how humans adjust their values.
If the emotions and basic drives are different, the values learned will be different. If the compressed description of the basic drives is just 1kb, there are ~2^1024 different possible initial minds with drives that complex, most of them wildly alien.
How would you know what the AI would find beautiful? Will you get all aspects of it's sexuality right?
If the AI isn't comforted by physical contact, that's at least few bytes of the drive description that's different than the description that matches our drives. That difference throws out a huge chunk of how our morality has evolved to instill itself.
We might still be able to get an alien mind to adopt all the complex values we have, but we would have to translate the actions we would normally take into actions that match alien emotions. This is a hugely complex task that we have no prior experience with.
Right, so we agree on that then.
If I was going to simplify - our emotional systems and the main associated neurotransmitter feedback loops are the genetic harnesses that constrain the otherwise overly general cortex and its far more complex, dynamic memetic programs.
We have these simple reinforcement learning systems to avoid pain-causing stimuli, pleasure-reward, and so on - these are really old conserved systems from the thalamus that have maintained some level of control and shaping of the cortex as it has rapidly expanded and taken over.
You can actually disable a surprising large number of these older circuits (through various disorders, drugs, injuries) and still have an intact system: - physical pain/pleasure, hunger, yes even sexuality.
And then there are some more complex circuits that indirectly reward/influence social behaviour. They are hooks though, they don't have enough complexity to code for anything as complex as language concepts. They are gross, inaccurate statistical manipulators that encourage certain behaviours apriori
If these 'things' could talk, they would be constantly telling us to: (live in groups, groups are good, socializing is good, share information, have sex, don't have sex with your family, smiles are good, laughter is good, babies are cute, protect babies, it's good when people like you, etc etc.)
Another basic drive appears to be that for learning itself, and its interesting how far that alone could take you. The learning drive is crucial. Indeed the default 'universal intelligence' (something like AIXI) may just have the learning drive taken to the horizon. Of course, that default may not necessarily be good for us, and moreover it may not even be the most efficient.
However, something to ponder is that the idea of "taking the learning drive" to the horizon (maximize knowledge) is surprisingly close to the main cosmic goal of most transhumanists, extropians, singularitans, etc etc. Something to consider: perhaps there is some universal tendency towards a universal intelligence (and single universal goal).
Looking at it this way, scientists and academic types have a stronger than usual learning drive, closely correlated with higher-than-average intelligence. The long standing ascetic and monastic traditions in human cultures show how memetics can sometimes override the genetic drives completely, resulting in beings who have scarified all genetic fitness for memetic fitness. Most scientists don't go to that extreme, but it is a different mindset - and the drives are different.
Sure, but we don't need all the emotions and basic drives. Even if we take direct inspiration from the human brain, some are actually easy to remove - as mentioned earlier. Sexuality (as a drive) is surprisingly easy to remove (although certainly considered immoral to inflect on humans! we seem far less concerned with creating asexual AIs) along with most of the rest.
The most important is the learning drive. Some of the other more complex social drives we may want to keep, and the emotional reinforcement learning systems in general may actually just be nifty solutions to very challenging engineering problems - in which case we will keep some of them as well.
I don't find your 2^1024 analysis useful - the space of possible drives/brains created by the genome is mainly empty - almost all designs are duds, stillbirths.
We aren't going to be randomly picking random drives from a lottery. We will either be intentionally taking them from the brain, or intentionally creating new systems.
There is probably a name for this as a 'disorder', but I had a deep revulsion of physical contact as a child. I grew out of this to a degree later. I don't see the connection to morality.
Part of the problem here is morality is a complex term.
The drives and the older simpler control systems in the brain do not operate at the level of complex linguistic concepts - that came much much later. They can influence our decisions and sense of right/wrongness for simple decisions especially, but they have increasingly less influence as you spend more time considering the problem and developing a more complex system of ethics.
alien mind? Who is going to create alien minds? There is the idea of running some massive parallel universe sim to evolve intelligence from scratch, but thats just silly from a computational point of view.
The most likely contender at this point is reverse engineering the brain, and to the extent that human morality has some genetic tweaked-tendencies, we can get those by reverse engineering the relevant circuits.
But remember the genetically preserved emotional circuits are influencers on behavior, but minor, and are not complex enough to cope with abstract linguistic concepts.
Again again, there is nothing in the genome that tells you that slavery is wrong, or that human sacrifice is wrong, or that computers can have rights.
Those concepts operate an entire new plane which the genome does not participate in.