The challenges of bringing up AIs

Stuart_Armstrong

At the current AGI-12 conference, some designers have been proponents of keeping AGI's safe by bringing them up in human environments, providing them with interactions and feedback in a similar way to how we bring up human children. Obviously that approach would fail for a fully smart AGI with its own values - it would pretend to follow our values for as long as it needed, and then defect. However, some people have confidence if we started with a limited, dumb AGI, then we could successfully inculcate our values in this way (a more sophisticated position would be that though this method would likely fail, it's more likely to succeed than a top-down friendliness project!).

The major criticism of this approach is that it anthropomorphises the AGI - we have a theory of children's minds, constructed by evolution, culture, and our own child-rearing experience. And then we project this on the alien mind of the AGI, assuming that if the AGI presents behaviours similar to a well-behaved child, then it will become a moral AGI. The problem is that we don't know how alien the AGI's mind will be, and if our reinforcement is actually reinforcing the right thing. Specifically, we need to be able to find some way of distinguishing between:

An AGI being trained to be friendly.
An AGI being trained to lie and conceal.
An AGI that will behave completely differently once out of the training/testing/trust-building environment.
An AGI that forms the wrong categories and generalisations (what counts as "human" or "suffering", for instance), because it lacks human-shared implicit knowledge that was "too obvious" for us to even think of training it on.

An AGI being trained to be friendly.
An AGI being trained to lie and conceal.
An AGI that will behave completely differently once out of the training/testing/trust-building environment.
An AGI that forms the wrong categories and generalisations (what counts as "human" or "suffering", for instance), because it lacks human-shared implicit knowledge that was "too obvious" for us to even think of training it on.

Stuart: The majority of people proposing the "bringing up baby AGI" approach to encouraging AGI ethics, are NOT making the kind of naive cognitive error you describe here. This approach to AGI ethics is not founded on naive anthropomorphism. Rather, it is based on the feeling of having a mix of intuitive and rigorous understanding of the AGI architectures in question, the ones that will be taught ethics.

For instance, my intuition is that if we taught an OpenCog system to be loving and ethical, then it would very likely be so, according to broad human standards. This intuition is NOT based on naively anthropomorphizing OpenCog systems, but rather based on my understanding of the actual OpenCog architecture (which has many significant differences from the human cognitive architecture).

No one, so far as I know, claims to have an airtight PROOF that this kind of approach to AGI ethics will work. However, the intuition that it will work is based largely on understanding of the specifics of the AGI architectures in question, not just on anthropomorphism.

If you want to counter-argue against this approach, you should argue about it in the context of the specific AGI architectures in question. Or else you should present some kind of principled counter-argument. Just claiming "anthropomorphism" isn't very convincing.

An AGI that is not either deeply neuromorphic or possessing a well-defined and formally stable utility function sounds like... frankly one of the worst ideas I've ever heard. I'm having difficulty imagining a way you could demonstrate the safety of such a system, or trust it enough at any point to give it enough resources to learn. Considering that the fate of intelligent life in our future light cone may hang in the balance, standards of safety must obviously be very high! Intuition is, I'm sorry, simply not an acceptable criteria on which to wager at ... (read more)

9Dr_Manhattan13y

Ben, your response is logical (if not correct), but the fact that many AI researchers advocate the "upbringing approach" (for other architectures) makes me very suspicious that they're anthropomorphising after all.

14Stuart_Armstrong13y

Thanks for your answer, Ben! First of all, all of these methods involve integrating the AGI in human society. So the AGI is forming its values, at least in part, through doing something (possibly talking) and getting a response from some human. That human will be interpreting the AGI's answers, and selecting the right response, using their own theory of the AGI's mind - nearly certainly an anthopomorphisation! Even if that human develops experience dealing with the AGI, their understanding will be limited (as our understanding of other humans is limited, except worse than that). So the AGI programmer is taking a problem that they can't solve through direct coding, and putting the AGI through interactions so that it will acquire the values that the programmer can't specify directly, in settings where the other interactors will be prone to anthropomorphisation. ie: "I can't solve this problem formally, but I do understand it's structure enough to be reasonably sure that anthropomorphic interactions will solve it". If that's the claim, I would expect the programmer to be very schooled in the properties and perils of anthropomorphisation, and to cast their arguments, as much as possible, in formal logic or code form. For instance, if we want the AGI to "love" us: what kind of behaviour would we expect that this entailed, and why would this code acquire that behaviour from these interactions? If you couldn't use the word love, or any close synonyms, could you still describe the process and show that it will perform well? If you can't describe love without saying "love", then you are counting on a shared non-formalised human understanding of what love is, and hoping that the AGI will stumble upon the same understanding - you don't know the contours of the definition, and the potential pitfalls, but you're counting on the AGI to avoid them. Those four types of behaviours that I mentioned there, and that we need to separate - don't just decry the use of anthropomorphis

16

The challenges of bringing up AIs

16

16

16

The challenges of bringing up AIs

16

16