Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: jacob_cannell 02 July 2015 12:09:25AM *  4 points [-]

How does this "seed" find the correct high-level sensory features to plug into? How can it wire complex high-level behavioral programs (such as courtship behaviors) to low-level motor programs learned by unsupervised learning?

This particular idea is not well developed yet in my mind, and I haven't really even searched the literature yet. So keep that in mind.

Leave courtship aside, let us focus on attraction - specifically evolution needs to encode detectors which can reliably identify high quality mates of the opposite sex apart from all kinds of other objects. The problem is that a good high quality face recognizer is too complex to specify in the genome - it requires many billions of synapses, so it needs to be learned. However, the genome can encode an initial crappy face detector. It can also encode scent/pheromone detectors, and it can encode general 'complexity' and or symmetry detectors that sit on top, so even if it doesn't initially know what it is seeing, it can tell when something is about yeh complex/symmetric/interesting. It can encode the equivalent of : if you see an interesting face sized object which appears for many minutes at a time and moves at this speed, and you hear complex speech like sounds, and smell human scents, it's probably a human face.

Then the problem is reduced in scope. The cortical map will grow a good face/person model/detector on it's own, and then after this model is ready certain hormones in adolescence activate innate routines that learn where the face/person model patch is and help other modules plug into it. This whole process can also be improved by the use of a weak top down prior described above.

That being said, some systems - such as Atari's DRL agent - can be considered simple early versions of ULMs.

Not so fast.

Actually on consideration I think you are right and I did get ahead of myself there. The Atari agent doesn't really have a general memory subsystem. It has an episode replay system, but not general memory. Deepmind is working on general memory - they have the NTM paper and what not, but the Atari agent came before that.

I largely agree with your assessment of the Atari DRL agent.

Despite the name, no machine learning system, "deep" or otherwise, has been demonstrated to be able to efficiently learn any provably deep function (in the sense of boolean circuit depth-complexity), such as the parity function which any human of average intelligence could learn from a small number of examples.

I highly doubt that - but it all depends on what your sampling class for 'human' is. An average human drawn from the roughly 10 billion alive today? Or an average human drawn from the roughly 100 billion who have ever lived? (most of which would have no idea what a parity function is).

When you imagine a human learning the parity function from a small number of examples, what you really imagine is a human who has already learned the parity function, and thus internally has 'parity function' as one of perhaps a thousand types of functions they have learned, such that if you give them some data, it is one of the obvious things they may try.

Training a machine on a parity data set from scratch and expecting it to learn the parity function is equivalent to it inventing the parity function - and perhaps inventing mathematics as well. It should be compared to raising an infant without any knowledge of mathematics or anything related, and then training them on the raw data.

Comment author: V_V 02 July 2015 02:14:37PM *  1 point [-]

However, the genome can encode an initial crappy face detector.

It's not that crappy given that newborns can not only recognize faces with significant accuracy, but also recognize facial expressions.

The cortical map will grow a good face/person model/detector on it's own, and then after this model is ready certain hormones in adolescence activate innate routines that learn where the face/person model patch is and help other modules plug into it.

Having two separate face recognition modules, one genetically specified and another learned seems redundant, and still it's not obvious to me how a genetically-specified sexual attraction program could find how to plug into a completely learned system, which would necessarily have some degree of randomness.

It seems more likely that there is a single face recognition module which is genetically specified and then it becomes fine tuned by learning.

I highly doubt that - but it all depends on what your sampling class for 'human' is. An average human drawn from the roughly 10 billion alive today? Or an average human drawn from the roughly 100 billion who have ever lived? (most of which would have no idea what a parity function is).

Show a neolithic human a bunch of pebbles, some black and some white, laid out in a line. Ask them to add a black or white pebble to the line, and reward them if the number of black pebbles is even. Repeat multiple times.

Even without a concept of "even number", wouldn't this neolithic human be able to figure out an algorithm to compute the right answer? They just need to scan the line, flipping a mental switch for each black pebble they encounter, and then add a black pebble if and only if the switch is not in the initial position.

Maybe I'm overgeneralizing, but it seems unlikely to me that people able to invent complex hunting strategies, to build weapons, tools, traps, clothing, huts, to participate in tribe politics, etc. wouldn't be able to figure something like that.

Comment author: jacob_cannell 01 July 2015 08:48:30PM *  2 points [-]

But it doesn't imply the software architectures have to be similar. For example I see no reason to assume any ULM should be anything like a neural net.

Sure - any general model can simulate any other. Neural networks have strong practical advantages. Their operator base is based on fmads, which is a good match for modern computers. They allow explicit search of program space in terms of the execution graph, which is extremely powerful because it allows one to a priori exclude all programs which don't halt - you can constrain the search to focus on programs with exact known computational requirements.

Neural nets make deep factoring easy, and deep factoring is the single most important huge gain in any general optimization/learning system: it allows for exponential (albeit limited) speedup.

And another thing: teaching an AIs values by placing it in a human environment and counting on reinforcement learning can fail spectacularly if the AIs intelligence grows much faster than that of a human child.

Yes. There are pitfalls, and in general much more research to do on value learning before we get to useful AGI, let alone safe AGI.

A human brain is never going to learn to rearrange its low level circuitry to efficiently perform operations like numerical calculation.

This is arguably a misconception. The brain has a 100 hz clock rate at most. For general operations that involve memory, it's more like 10hz. Most people can do basic arithmetic in less than a second, which roughly maps to a dozen clock cycles or so, maybe less. That actually is comparable to many computers - for example on the current maxwell GPU architecture (nvidia's latest and greatest), even the simpler instructions have a latency of about 6 cycles.

Now, obviously the arithmetic ops that most humans can do in less than a second is very limited - it's like a minimal 3 bit machine. But some atypical humans can do larger scale arithmetic at the same speed.

Point is, you need to compare everything adjusted for the 6 order of magnitude speed difference.

Comment author: V_V 02 July 2015 01:08:13PM *  0 points [-]

This is arguably a misconception. The brain has a 100 hz clock rate at most. For general operations that involve memory, it's more like 10hz.

Mechanical calculators were slower than that, and still they were very much better at numeric computation than most humans, which made them incredibly useful.

Now, obviously the arithmetic ops that most humans can do in less than a second is very limited - it's like a minimal 3 bit machine. But some atypical humans can do larger scale arithmetic at the same speed.

Indeed these are very rare people. The vast majority of people, even if they worked for decades in accounting, can't learn to do numeric computation as fast and accurately as a mechanical calculator does.

Comment author: V_V 01 July 2015 10:41:05PM *  1 point [-]

Nice essay.

Do you think that transpararent machine learning could be practically achievable, or could it be the case that most models that we may want our machine learning systems to learn can be only represented by complex, unintellegible specifications?
Intuitively, the space of opaque models, be them neural networks, large decision tree forests, or incomprehensible spaghetti-code computer programs, seems bigger than the space of transparent models.

For instance, what would a transparent visual recognition model look like?

The most obvious choice would be Bayesian graphical model, with a prior over objects that could be in an image, a stochastic model over their properties (including stuff like body pose for animals), a prior over lights positions and properties, a prior over the backgrounds, a prior over camera poses and optical properties, a stochastic physics model of the interactions between light and the object of interest, background and camera, and so on.
It seem to me that it would be a very complex model, with lots of parameters, and likely not supporting efficient inference, much less efficient learning.

Traditional computer vision approaches tried to do this more or less, with some clever approximations and lots of engineering, and they were soundly beaten by opaque systems like ConvNets.

State of the art systems like ConvNets, on the other hand, learn shortcuts and heuristics, such as recognizing distinctive textures, which works very well in most cases, with some occasional glaring mistakes.
Perhaps any visual system capable of that level of performance must necessarily be a huge collection of heuristics of this type, maybe with more sophistication to avoid classifying a leopard print sofa as a leopard ( * ), but still fundamentally based on this architecture.

( * it's not like humans are immune to this failure mode anyway: people see a face on Mars, ghosts in blurry pictures, Jesus on a toast, Allah's name on a fish, etc. Pareidolia is certainly a thing.)

Comment author: jacob_cannell 30 June 2015 08:00:31PM *  2 points [-]

There are very different structures that are conceptually equivalent to a UTM (cellular automata, lambda calculus, recursive functions, Wang carpets etc.) In the same manner there can be AI architectures very different from the brain which are ULM-equivalent in a relevant sense.

Of course - but all of your examples are not just conceptually equivalent - they are functionally equivalent (they can emulate each other). They are all computational foundations for constructing UTMs - although not all foundations are truly practical and efficient. Likewise there are many routes to implementing a ULM - biology is one example, modern digital computers is another.

Frankly, this sounds questionable. For example, do you suggest sexual attraction is a cultrual/memetic construct?

Well I said "most everything", and I stressed several times in the article that much of the innate complexity budget is spent on encoding the value/reward system and the learning machinery (which are closely intertwined).

Sexual attraction is an interesting example, because it develops later in adolescence and depends heavily on complex learned sensory models. Current rough hypothesis: evolution encodes sexual attraction as a highly compressed initial 'seed' which unfolds over time through learning. It identifies/finds and then plugs into the relevant learned sensory concept representations which code for attractive members of the opposite sex. The compression effect explains the huge variety in human sexual preferences. Investigating/explaining this in more detail would take it's own post - its a complex interesting topic.

One key idea - which I proposed five years ago is that the AI should not know it is in a sim.

How do you suggest preventing it from discovering on its own that it is in a sim?

I should rephrase - it isn't necessarily a problem if the AI suspects its in a sim. Rather the key is that knowing one is in a sim and then knowing how to escape should be difficult enough to allow for sufficient time to evaluate the agent's morality, worth/utility to society, and potential future impact. In other words, the sandbox sim should be a test for both intelligence and morality.

Suspecting or knowing one is in a sim is easy. For example - the gnostics discovered the sim hypothesis long before Bostrom, but without understanding computers and computation they had zero idea how to construct or escape sims - it was just mysticism. In fact, the very term 'gnostic' means "one who knows" - and this was their self-identification; they believed they had discovered the great mystery of the universe (and claimed the teaching came from Jesus, although Plato had arguably hit upon an earlier version of the idea, and the term demiurge in particular comes from Plato).

It seems to me that the fact we have no conscious awareness of the workings of our brain and no way to consciously influence them suggests that the brain is at best an approximation of a ULM.

We certainly have some awareness of the workings of our brain - to varying degrees. For example you are probably aware of how you perform long multiplication, such that you could communicate the algorithm and steps. Introspection and verbalization of introspective insights are specific complex computations that require circuitry - they are not somehow innate to a ULM, because nothing is.

Current ANN engines can already train and run models with around 10 million neurons and 10 billion (compressed/shared) synapses on a single GPU, which suggests that the goal could soon be within the reach of a large organization.

This assume current ANN agents are already ULMs which I seriously doubt.

Sorry should have clarified - we will probably soon have the computational power to semi-affordably simulate ANNs with billions of neurons. That doesn't necessarily have anything to do with whether current ANN systems are ULMs. That being said, some systems - such as Atari's DRL agent - can be considered simple early versions of ULMs.

There is probably still much research and engineering work to do in going from simple basic ULMs up to brain-competitive systems. But research sometimes moves quickly - punctuated equilibrium and all that.

Here is a useful analogy: a simple abstract turing machine is to a modern GPU as a simple abstract ULM is to the brain. There is a huge engineering gap between the simplest early version of an idea, and a subsequent scaled up complex practical efficient version.

Comment author: V_V 01 July 2015 10:03:27PM *  2 points [-]

Current rough hypothesis: evolution encodes sexual attraction as a highly compressed initial 'seed' which unfolds over time through learning. It identifies/finds and then plugs into the relevant learned sensory concept representations which code for attractive members of the opposite sex.

How does this "seed" find the correct high-level sensory features to plug into? How can it wire complex high-level behavioral programs (such as courtship behaviors) to low-level motor programs learned by unsupervised learning?
This seems unlikely.

For example you are probably aware of how you perform long multiplication, such that you could communicate the algorithm and steps.

But long multiplication is something that you were taught in school, which most humans wouldn't be able to discover independently. And you are certainly not aware of how your brain perform visual recognition, the little you know was discovered through experiments, not introspection.

That being said, some systems - such as Atari's DRL agent - can be considered simple early versions of ULMs.

Not so fast.

The Atari DRL agent learns a good mapping between short windows of frames and button presses. It has some generalization capability which enables it to achieve human-level or sometimes even super human-level performances on games that are based on eye-hand coordination (after all it's not burdened by the intrinsic delays that occur in the human body), but it has no reasoning ability and fails miserably at any game which requires planning ahead more than a few frames.

Despite the name, no machine learning system, "deep" or otherwise, has been demonstrated to be able to efficiently learn any provably deep function (in the sense of boolean circuit depth-complexity), such as the parity function which any human of average intelligence could learn from a small number of examples.

I see no particular reason to believe that this could be solved by just throwing more computational power at the problem: you can't fight exponentials that way.

Comment author: lululu 30 June 2015 05:48:12PM 1 point [-]

Given the speed of AI development in other countries, do we know if any of the work on friendly AI is being translated or implemented outside of the US? Or what the level of awareness of AI friendliness issues among AI researchers in non-English speaking countries?

(I realize that IQ isn't an effective test of AI, but this is the article that prompted me wondering: http://www.businessinsider.com/chinese-ai-beat-humans-in-an-iq-test-2015-6. )

Comment author: V_V 01 July 2015 01:47:06PM 0 points [-]

Anybody who can contribute to AI research can read English.

Comment author: Wei_Dai 01 July 2015 06:25:31AM 2 points [-]

Nobody desires extinction, and nobody is better off if extinction comes form their own AI project rather than the AI project of somebody else, hence there is no tragedy of the commons scenario.

Extinction is much more costly to society as a whole than to any individual (especially if we count future unborn people). For example a purely selfish individual might value the cost of extinction the same as their own death (which is on average around $10 million as estimated by how much you have to pay people to compensate for increasing their risk of death). For society as a whole this cost is at least quadrillions of dollars if not astronomically more. So selfish individuals would be willing to take much bigger extinction risks than is socially optimal, if doing so provides them with private benefits. This is a tragedy of the commons scenario.

In the slow takeoff scenario, I think a similar tragedy of the commons dynamic is likely to play out. If humanity as a whole could coordinate and wait until we fully solve the AI control / value alignment problem before creating autonomous AIs, then humane values could eventually control all or most of the universe. But instead we're likely to create such AIs as soon as we can extract private benefits (fame, prestige, profit, etc.) from creating them. Once we do, they'll take over larger and larger share of the economy and eventually the universe. (Nobody currently owns the universe, so again it's a classic commons.)

Comment author: V_V 01 July 2015 01:45:42PM 0 points [-]

For example a purely selfish individual might value the cost of extinction the same as their own death (which is on average around $10 million as estimated by how much you have to pay people to compensate for increasing their risk of death). For society as a whole this cost is at least quadrillions of dollars if not astronomically more. So selfish individuals would be willing to take much bigger extinction risks than is socially optimal, if doing so provides them with private benefits. This is a tragedy of the commons scenario.

But a single purely selfish individual is unlikely to create a competitive AI project. For a medium-large organization made of people who care at least of their own life and the life of their kin the cost of extinction will be so high that it will offset any benefits that they may hope to obtain.

Comment author: PhoenixComplex7 30 June 2015 03:41:43AM *  0 points [-]

An A.I. arms race might cause researchers or sponsors to commit a number of inadvisable actions. These sorts of political concepts are discussed well in Bostrom's book, Superintelligence: Path, Dangers, Strategies, but can be summed up as follows:

Due to the fact that moderate and fast takeoffs are more likely than slow ones, any project that achieves it's goals is likely to gain a decisive strategic advantage over other projects, meaning they lose.

Thus, if a given project is not in the lead, it might start lessening it's safety protocol in favor of speed (not to mention standard cloak and dagger actions, or even militaristic scenarios). Is not good, gets extinction.

Comment author: V_V 30 June 2015 09:22:47AM 1 point [-]

Due to the fact that moderate and fast takeoffs are more likely than slow ones,

That's a big assumption.

Thus, if a given project is not in the lead, it might start lessening it's safety protocol in favor of speed (not to mention standard cloak and dagger actions, or even militaristic scenarios). Is not good, gets extinction.

Nobody desires extinction, and nobody is better off if extinction comes form their own AI project rather than the AI project of somebody else, hence there is no tragedy of the commons scenario.
People are not going to make an AI capable of causing major disasters without being reasonable sure that they can control it.

Comment author: Wei_Dai 27 June 2015 02:28:24AM *  7 points [-]

Thank you, I saw this earlier but posting it here makes it easier for me to comment. :)

In order for an AI to learn a utility function from humans that is safe to maximize over, it needs to have no errors over its whole domain (if there is a single error in the utility function arbitrarily far away from the training distribution, the AI could eventually seek it out when it gets powerful enough to be able to reach that point in state space). Not only that, but it has to correct the errors that are in the training data. So from current learning algorithms whose average case error can suffer badly when the input distribution is slightly shifted, we have to get to one with a worst case error of zero and negative average case error. Does this seem like a fair or useful way to state the severity of the problem in ML terms?

(BTW, I'm aware of work by Paul Christiano to try to design FAI that can tolerate some errors made by the learning algorithm, but I'm not sure that the cost in terms of higher complexity and lower efficiency is worth it, since it may not be much easier to get the kind of theoretical guarantees that his designs need.)

Another issue that makes me pessimistic about the long run outcome is the seeming inevitability of AI arms races and resulting incentives to skimp on ethics and safety in order to beat the competition to market or publication. I haven't seen much discussion of this by people who are "in" AI / ML. What's your view on this, and do you think there should be greater awareness/discussion of this issue?

Do you have a reference for "weakly supervised learning"? I did some searches but couldn't find anything that seemed especially relevant to the way you're using it.

Comment author: V_V 28 June 2015 12:05:00PM *  0 points [-]

Another issue that makes me pessimistic about the long run outcome is the seeming inevitability of AI arms races and resulting incentives to skimp on ethics and safety in order to beat the competition to market or publication.

Isn't the arms race a safeguard? If multiple AIs of similar intelligence are competing it is difficult for any one of them to completely outsmart all the others and take over the world.

Comment author: jacob_cannell 25 June 2015 07:19:14PM 9 points [-]

in all these experiments the original task-specific regions are still present and functional, therefore maybe the brain can partially use these regions by learning how to route the signals to them.

No - these studies involve direct measurements (electrodes for the ferret rewiring, MRI for echolocation). They know the rewired auditory cortex is doing vision, etc.

But then why doesn't universal learning just co-opt some other brain region to perform the task of the damaged one?

It can, and this does happen all the time. Humans can recover from serious brain damage (stroke, injury, etc). It takes time to retrain and reroute circuitry - similar to relearning everything that was lost all over again.

And anyway why is the specialization pattern consistent across individuals and even species? If you train an artificial neural network multiple times on the same dataset

Current ANN's assume a fixed module layout, so they aren't really comparable in module-task assignment.

Much of the specialization pattern could just be geography - V1 becomes visual because it is closest to the visual input. A1 becomes auditory because it is closest to the auditory input. etc.

This should be the default hypothesis, but there also could be some element of prior loading, perhaps from pattern generators in the brainstem. (I have read a theory that there is a pattern generator for faces that pretrains the visual cortex a little bit in the womb, so that it starts with a vague primitive face detector).

After all, in a computer you can swap block or pages of memory around and as long as pointers (or page tables) are updated the behavior does not change, up to some performance issues due to cache misses. If the brain worked that way we should expect cortical regions to be allocated to different tasks in a more or less random pattern varying between individuals.

I said the BG is kind-of-like the CPU, the cortex is kind-of-like a big FPGA, but that is an anlogy. The are huge differences between slow bio-circuitry and fast von neumman machines.

Firstly the brain doesn't really have a concept of 'swapping memory'. The closest thing to that is retraining, where the hippocampus can train info into the cortex. It's a slow complex process that is nothing like swapping memory.

Finally the brain is much more optimized at the wiring/latency level. Functionality goes in certain places because that is where it is best for that functionality - it isn't permutation symmetric in the slightest. Every location has latency/wiring tradeoffs. In a von neumman memory we just abstract that all away. Not in the brain. There is an actual optimal location for every concept/function etc.

a newborn horse is able to walk, run and follow their mother within a few hours from birth.

That is fast for mammals - I know first hand that it can take days for deer. Nonetheless, as we discussed, the brainstem provides a library of innate complex motor circuitry in particular, which various mammals can rely on to varying degrees, depending on how important complex early motor behavior is.

Targetprop is still highly speculative. It has not shown to work well in artificial neural networks and the evidence of biological plausibility is handwavy.

I agree that there is still more work to be done understanding the brain's learning machinery. Targetprop is useful/exciting in ML, but it isn't the full picture yet.

[Atari] That's actually extremely impressive - superhuman learning speed.

Humans get tired after continuously playing for a few hours, but in terms of overall playtime they learn faster.

Not at all. The Atari agent becomes semi-superhuman by day 3 of it's life. When humans start playing atari, they already have trained vision and motor systems, and Atari is designed for these systems. Even then your statement is wrong - in that I don't think any children achieve playtester levels of skill in just even a few days.

Comment author: V_V 27 June 2015 07:31:23PM *  2 points [-]

Finally the brain is much more optimized at the wiring/latency level. Functionality goes in certain places because that is where it is best for that functionality - it isn't permutation symmetric in the slightest. Every location has latency/wiring tradeoffs. In a von neumman memory we just abstract that all away. Not in the brain. There is an actual optimal location for every concept/function etc.

Well, the eyes are at the front of the head, but the optic nerves connect to the brain at the back, and they also cross at the optic chiasm. Axons also cross contralaterally in the spinal cord and if I recall correctly there are various nerves that also don't take the shortest path.
This seems to me as evidence that the nervous system is not strongly optimized for latency.

Comment author: Risto_Saarelma 27 June 2015 01:03:57PM *  5 points [-]

There's also the whole Lesswrong-is-dying thing that might be contribute to the vibe you're getting. I've been reading the forum for years and it hasn't felt very healthy for a while now. A lot of the impressive people from earlier have moved on, we don't seem to be getting that many new impressive people coming in and hanging out a lot on the forum turns out not to make you that much more impressive. What's left is turning increasingly into a weird sort of cargo cult of a forum for impressive people.

Comment author: V_V 27 June 2015 02:13:56PM *  4 points [-]

Actually, I think that LessWrong used to be worse when the "impressive people" were posting about cryonics, FAI, many-world interpretation of quantum mechanics, and so on.

View more: Next