Superintelligence Reading Group 3: AI and Uploads

KatjaGrace

This is part of a weekly reading group on Nick Bostrom's book, Superintelligence. For more information about the group, and an index of posts so far see the announcement post. For the schedule of future topics, see MIRI's reading guide.

Welcome. This week we discuss the third section in the reading guide, AI & Whole Brain Emulation. This is about two possible routes to the development of superintelligence: the route of developing intelligent algorithms by hand, and the route of replicating a human brain in great detail.

This post summarizes the section, and offers a few relevant notes, and ideas for further investigation. My own thoughts and questions for discussion are in the comments.

There is no need to proceed in order through this post. Feel free to jump straight to the discussion. Where applicable, page numbers indicate the rough part of the chapter that is most related (not necessarily that the chapter is being cited for the specific claim).

Reading: “Artificial intelligence” and “Whole brain emulation” from Chapter 2 (p22-36)

Summary

Intro

Superintelligence is defined as 'any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest'
There are several plausible routes to the arrival of a superintelligence: artificial intelligence, whole brain emulation, biological cognition, brain-computer interfaces, and networks and organizations.
Multiple possible paths to superintelligence makes it more likely that we will get there somehow.

A human-level artificial intelligence would probably have learning, uncertainty, and concept formation as central features.
Evolution produced human-level intelligence. This means it is possible, but it is unclear how much it says about the effort required.
Humans could perhaps develop human-level artificial intelligence by just replicating a similar evolutionary process virtually. This appears at after a quick calculation to be too expensive to be feasible for a century, however it might be made more efficient.
Human-level AI might be developed by copying the human brain to various degrees. If the copying is very close, the resulting agent would be a 'whole brain emulation', which we'll discuss shortly. If the copying is only of a few key insights about brains, the resulting AI might be very unlike humans.
AI might iteratively improve itself from a meagre beginning. We'll examine this idea later. Some definitions for discussing this:
1. 'Seed AI': a modest AI which can bootstrap into an impressive AI by improving its own architecture.
2. 'Recursive self-improvement': the envisaged process of AI (perhaps a seed AI) iteratively improving itself.
3. 'Intelligence explosion': a hypothesized event in which an AI rapidly improves from 'relatively modest' to superhuman level (usually imagined to be as a result of recursive self-improvement).
The possibility of an intelligence explosion suggests we might have modest AI, then suddenly and surprisingly have super-human AI.
An AI mind might generally be very different from a human mind.

Whole brain emulation

Whole brain emulation (WBE or 'uploading') involves scanning a human brain in a lot of detail, then making a computer model of the relevant structures in the brain.
Three steps are needed for uploading: sufficiently detailed scanning, ability to process the scans into a model of the brain, and enough hardware to run the model. These correspond to three required technologies: scanning, translation (or interpreting images into models), and simulation (or hardware). These technologies appear attainable through incremental progress, by very roughly mid-century.
This process might produce something much like the original person, in terms of mental characteristics. However the copies could also have lower fidelity. For instance, they might be humanlike instead of copies of specific humans, or they may only be humanlike in being able to do some tasks humans do, while being alien in other regards.

Notes

What routes to human-level AI do people think are most likely?
Bostrom and Müller's survey asked participants to compare various methods for producing synthetic and biologically inspired AI. They asked, 'in your opinion, what are the research approaches that might contribute the most to the development of such HLMI?” Selection was from a list, more than one selection possible. They report that the responses were very similar for the different groups surveyed, except that whole brain emulation got 0% in the TOP100 group (100 most cited authors in AI) but 46% in the AGI group (participants at Artificial General Intelligence conferences). Note that they are only asking about synthetic AI and brain emulations, not the other paths to superintelligence we will discuss next week.
How different might AI minds be?
Omohundro suggests advanced AIs will tend to have important instrumental goals in common, such as the desire to accumulate resources and the desire to not be killed.
Anthropic reasoning
‘We must avoid the error of inferring, from the fact that intelligent life evolved on Earth, that the evolutionary processes involved had a reasonably high prior probability of producing intelligence’ (p27)

Whether such inferences are valid is a topic of contention. For a book-length overview of the question, see Bostrom’s Anthropic Bias. I’ve written shorter (Ch 2) and even shorter summaries, which links to other relevant material. The Doomsday Argument and Sleeping Beauty Problem are closely related.
More detail on the brain emulation scheme
Whole Brain Emulation: A Roadmap is an extensive source on this, written in 2008. If that's a bit too much detail, Anders Sandberg (an author of the Roadmap) summarises in an entertaining (and much shorter) talk. More recently, Anders tried to predict when whole brain emulation would be feasible with a statistical model. Randal Koene and Ken Hayworth both recently spoke to Luke Muehlhauser about the Roadmap and what research projects would help with brain emulation now.
Levels of detail
As you may predict, the feasibility of brain emulation is not universally agreed upon. One contentious point is the degree of detail needed to emulate a human brain. For instance, you might just need the connections between neurons and some basic neuron models, or you might need to model the states of different membranes, or the concentrations of neurotransmitters. The Whole Brain Emulation Roadmap lists some possible levels of detail in figure 2 (the yellow ones were considered most plausible). Physicist Richard Jones argues that simulation of the molecular level would be needed, and that the project is infeasible.
Other problems with whole brain emulation
Sandberg considers many potential impediments here.
Order matters for brain emulation technologies (scanning, hardware, and modeling)
Bostrom points out that this order matters for how much warning we receive that brain emulations are about to arrive (p35). Order might also matter a lot to the social implications of brain emulations. Robin Hanson discusses this briefly here, and in this talk (starting at 30:50) and this paper discusses the issue.
What would happen after brain emulations were developed?
We will look more at this in Chapter 11 (weeks 17-19) as well as perhaps earlier, including what a brain emulation society might look like, how brain emulations might lead to superintelligence, and whether any of this is good.
Scanning (p30-36)
‘With a scanning tunneling microscope it is possible to ‘see’ individual atoms, which is a far higher resolution than needed...microscopy technology would need not just sufficient resolution but also sufficient throughput.’

Here are some atoms, neurons, and neuronal activity in a living larval zebrafish, and videos of various neural events.

Array tomography of mouse somatosensory cortex from Smithlab.

A molecule made from eight cesium and eight
iodine atoms (from here).
Efforts to map connections between neurons
Here is a 5m video about recent efforts, with many nice pictures. If you enjoy coloring in, you can take part in a gamified project to help map the brain's neural connections! Or you can just look at the pictures they made.
The C. elegans connectome (p34-35)
As Bostrom mentions, we already know how all of C. elegans’ neurons are connected. Here's a picture of it (via Sebastian Seung):

In-depth investigations

If you are particularly interested in these topics, and want to do further research, these are a few plausible directions, some taken from Luke Muehlhauser's list:

Produce a better - or merely somewhat independent - estimate of how much computing power it would take to rerun evolution artificially. (p25-6)
How powerful is evolution for finding things like human-level intelligence? (You'll probably need a better metric than 'power'). What are its strengths and weaknesses compared to human researchers?
Conduct a more thorough investigation into the approaches to AI that are likely to lead to human-level intelligence, for instance by interviewing AI researchers in more depth about their opinions on the question.
Measure relevant progress in neuroscience, so that trends can be extrapolated to neuroscience-inspired AI. Finding good metrics seems to be hard here.
e.g. How is microscopy progressing? It’s harder to get a relevant measure than you might think, because (as noted p31-33) high enough resolution is already feasible, yet throughput is low and there are other complications.
Randal Koene suggests a number of technical research projects that would forward whole brain emulation (fifth question).

If you are interested in anything like this, you might want to mention it in the comments, and see whether other people have useful thoughts.

How to proceed

This has been a collection of notes on the chapter. The most important part of the reading group though is discussion, which is in the comments section. I pose some questions for you there, and I invite you to add your own. Please remember that this group contains a variety of levels of expertise: if a line of discussion seems too basic or too incomprehensible, look around for one that suits you better!

Next week, we will talk about other paths to the development of superintelligence: biological cognition, brain-computer interfaces, and organizations. To prepare, read Biological Cognition and the rest of Chapter 2. The discussion will go live at 6pm Pacific time next Monday 6 October. Sign up to be notified here.

This post summarizes the section, and offers a few relevant notes, and ideas for further investigation. My own thoughts and questions for discussion are in the comments.

Reading: “Artificial intelligence” and “Whole brain emulation” from Chapter 2 (p22-36)

Summary

Intro

Superintelligence is defined as 'any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest'
There are several plausible routes to the arrival of a superintelligence: artificial intelligence, whole brain emulation, biological cognition, brain-computer interfaces, and networks and organizations.
Multiple possible paths to superintelligence makes it more likely that we will get there somehow.

A human-level artificial intelligence would probably have learning, uncertainty, and concept formation as central features.
Evolution produced human-level intelligence. This means it is possible, but it is unclear how much it says about the effort required.
Humans could perhaps develop human-level artificial intelligence by just replicating a similar evolutionary process virtually. This appears at after a quick calculation to be too expensive to be feasible for a century, however it might be made more efficient.
Human-level AI might be developed by copying the human brain to various degrees. If the copying is very close, the resulting agent would be a 'whole brain emulation', which we'll discuss shortly. If the copying is only of a few key insights about brains, the resulting AI might be very unlike humans.
AI might iteratively improve itself from a meagre beginning. We'll examine this idea later. Some definitions for discussing this:
1. 'Seed AI': a modest AI which can bootstrap into an impressive AI by improving its own architecture.
2. 'Recursive self-improvement': the envisaged process of AI (perhaps a seed AI) iteratively improving itself.
3. 'Intelligence explosion': a hypothesized event in which an AI rapidly improves from 'relatively modest' to superhuman level (usually imagined to be as a result of recursive self-improvement).
The possibility of an intelligence explosion suggests we might have modest AI, then suddenly and surprisingly have super-human AI.
An AI mind might generally be very different from a human mind.

Whole brain emulation

Whole brain emulation (WBE or 'uploading') involves scanning a human brain in a lot of detail, then making a computer model of the relevant structures in the brain.
Three steps are needed for uploading: sufficiently detailed scanning, ability to process the scans into a model of the brain, and enough hardware to run the model. These correspond to three required technologies: scanning, translation (or interpreting images into models), and simulation (or hardware). These technologies appear attainable through incremental progress, by very roughly mid-century.
This process might produce something much like the original person, in terms of mental characteristics. However the copies could also have lower fidelity. For instance, they might be humanlike instead of copies of specific humans, or they may only be humanlike in being able to do some tasks humans do, while being alien in other regards.

Notes

What routes to human-level AI do people think are most likely?
Bostrom and Müller's survey asked participants to compare various methods for producing synthetic and biologically inspired AI. They asked, 'in your opinion, what are the research approaches that might contribute the most to the development of such HLMI?” Selection was from a list, more than one selection possible. They report that the responses were very similar for the different groups surveyed, except that whole brain emulation got 0% in the TOP100 group (100 most cited authors in AI) but 46% in the AGI group (participants at Artificial General Intelligence conferences). Note that they are only asking about synthetic AI and brain emulations, not the other paths to superintelligence we will discuss next week.
How different might AI minds be?
Omohundro suggests advanced AIs will tend to have important instrumental goals in common, such as the desire to accumulate resources and the desire to not be killed.
Anthropic reasoning
‘We must avoid the error of inferring, from the fact that intelligent life evolved on Earth, that the evolutionary processes involved had a reasonably high prior probability of producing intelligence’ (p27)

Whether such inferences are valid is a topic of contention. For a book-length overview of the question, see Bostrom’s Anthropic Bias. I’ve written shorter (Ch 2) and even shorter summaries, which links to other relevant material. The Doomsday Argument and Sleeping Beauty Problem are closely related.
More detail on the brain emulation scheme
Whole Brain Emulation: A Roadmap is an extensive source on this, written in 2008. If that's a bit too much detail, Anders Sandberg (an author of the Roadmap) summarises in an entertaining (and much shorter) talk. More recently, Anders tried to predict when whole brain emulation would be feasible with a statistical model. Randal Koene and Ken Hayworth both recently spoke to Luke Muehlhauser about the Roadmap and what research projects would help with brain emulation now.
Levels of detail
As you may predict, the feasibility of brain emulation is not universally agreed upon. One contentious point is the degree of detail needed to emulate a human brain. For instance, you might just need the connections between neurons and some basic neuron models, or you might need to model the states of different membranes, or the concentrations of neurotransmitters. The Whole Brain Emulation Roadmap lists some possible levels of detail in figure 2 (the yellow ones were considered most plausible). Physicist Richard Jones argues that simulation of the molecular level would be needed, and that the project is infeasible.
Other problems with whole brain emulation
Sandberg considers many potential impediments here.
Order matters for brain emulation technologies (scanning, hardware, and modeling)
Bostrom points out that this order matters for how much warning we receive that brain emulations are about to arrive (p35). Order might also matter a lot to the social implications of brain emulations. Robin Hanson discusses this briefly here, and in this talk (starting at 30:50) and this paper discusses the issue.
What would happen after brain emulations were developed?
We will look more at this in Chapter 11 (weeks 17-19) as well as perhaps earlier, including what a brain emulation society might look like, how brain emulations might lead to superintelligence, and whether any of this is good.
Scanning (p30-36)
‘With a scanning tunneling microscope it is possible to ‘see’ individual atoms, which is a far higher resolution than needed...microscopy technology would need not just sufficient resolution but also sufficient throughput.’

Here are some atoms, neurons, and neuronal activity in a living larval zebrafish, and videos of various neural events.

Array tomography of mouse somatosensory cortex from Smithlab.

A molecule made from eight cesium and eight
iodine atoms (from here).
Efforts to map connections between neurons
Here is a 5m video about recent efforts, with many nice pictures. If you enjoy coloring in, you can take part in a gamified project to help map the brain's neural connections! Or you can just look at the pictures they made.
The C. elegans connectome (p34-35)
As Bostrom mentions, we already know how all of C. elegans’ neurons are connected. Here's a picture of it (via Sebastian Seung):

In-depth investigations

If you are particularly interested in these topics, and want to do further research, these are a few plausible directions, some taken from Luke Muehlhauser's list:

Produce a better - or merely somewhat independent - estimate of how much computing power it would take to rerun evolution artificially. (p25-6)
How powerful is evolution for finding things like human-level intelligence? (You'll probably need a better metric than 'power'). What are its strengths and weaknesses compared to human researchers?
Conduct a more thorough investigation into the approaches to AI that are likely to lead to human-level intelligence, for instance by interviewing AI researchers in more depth about their opinions on the question.
Measure relevant progress in neuroscience, so that trends can be extrapolated to neuroscience-inspired AI. Finding good metrics seems to be hard here.
e.g. How is microscopy progressing? It’s harder to get a relevant measure than you might think, because (as noted p31-33) high enough resolution is already feasible, yet throughput is low and there are other complications.
Randal Koene suggests a number of technical research projects that would forward whole brain emulation (fifth question).

If you are interested in anything like this, you might want to mention it in the comments, and see whether other people have useful thoughts.

How to proceed

Why ‘WB’ in “WBE” is not well-defined and why WBE is a worthwhile research paradigm, despite its nearly fatal ambiguities.

Our community (in which I include cognitive neurobiologists, AI researchers, philosophers of mind, research neurologists, behavioral and neuro-zoologists and ethologists, and anyone here) has, for some years, included theorists who present various versions of “extended mind” theories.

Without taking any stances about those theories (and I do have a unique take on those) in this post, I’ll outline some concerns about extended brain issues.

I have to compress this so it fits, but I think my concerns will be pretty self-evident. I should say that this week has been a schedule crunch for me, so I apologize if some of what I adduce in the service this post's topic, is mentioned in one of the papers linked in the existing comments, or the moderators summary. I do read all of those links, eventually, and they are very helpful value-added resources... at least as valuable to me as what I have read thus far in Bostrom's book.

I’ll be making a video lecture with diagrams, some additional key ideas, and a more in-depth discussion of several issues touched on in this post, for the YouTube channel I am developing on my own. (The channel will be live in a few weeks.)

(I think a video with some graphics - even line drawings on a whiteboard - might be helpful to schematically depict some physiological structures and connecting causal feed lines that I'll mention, for readers whose pictorial imagination of some of the intracranial and extracranial "extended brain" structures I’ll discuss, doesn't readily come to mind.)

My overarching global point is this: There is no theory-neutral, intelligible definition of the term, ‘whole brain’.

Let me walk through this in enough detail to make the point compelling – and it is compelling, not just some technical quibble -- without burying any of us in physiological minutia.

First, let us pretend that we can limit our focus to just what is in the skull – a posit which I will criticize and withdraw in a moment.

Does “whole brain” include glia: such as oligodendrocytes and astrocytes? Glia (properly: “neuroglia”) actually outnumber “standard” neurons by 3 to 1, in the brain. See this reference: http://www.ncbi.nlm.nih.gov/books/NBK10869/

One of the more dramatic areas of continuing advance in our understanding of the functionally relevant neurobiology of the brain and CNS has, for nearly two decades now, involved neuroglia.

Article after article of impeccable science has expanded the consensus view of neuroglial function, to comprise much more than housekeeping, i.e., more than merely helping to implement aspects of the blood-brain barrier, or assisting phagocytosis after unfortunate neurons undergo apoptosis, and so on, as believed prior to about 20 years ago.

Some things now commonly known: Glia help regulate and recycle ATP and make it available to adjacent neurons and their mitochondria (as well as tracking local ATP reserve); they ballast, balance and reclaim the glutamate pool; they participate in neurosteroid signal pathways (into and out of the brain), and may help ballast local ion levels

The list is continually expanding, and includes functions relevant to learning, neuromodulation, reacting to stress and allostatic load, interacting with some inflammatory signaling pathways, and neuroplasticity.

Of course, the functions and contribution of myelin (Schwann cells) to normal, healthy neuronal behavior, are well known, as are numerous decimating diseases of myelin maintenance. Some reputable researchers think glia are not only under-emphasized, but might be crucial for unraveling some stubborn, still-unanswered questions relevant to cognition.

For example researchers are investigating mechanisms by which they may play a role as auxiliary signaling systems throughout the brain. Consider the phase-coupled “communication” of distantly separated ensembles of coherently oscillating neurons. Recall that these transiently – within each ensemble – organize into cooperative intra-ensemble oscillators, and a well-documented phenomenon commonly proposed as a way the brain achieves temporary “binding” across sensory modalities, as it constructs a unified object percept, is the simultaneous -- yet widely separated across the brain -- phase-locked and frequency coupled oscillations of multiple, separate, oscillating ensembles.

This (otherwise) spooky action-at-a-distance of comcommittantly oscillating distal ensembles might be undergirded by neuroglia.

In short, neuroglia (even without the more speculative roles being considered, like the very last one I mentioned) are not just “physiological” support for “real neurons” that do the “real information processing.” Rather, neuroglia strongly appear to participate on multiple levels in learning, information processing, mood, plasticity, and other functions on what seems to be a growing list.

So, it seems to me that you have to emulate neuroglia, in WBE strategies (or else bear the burden of explaining why you are leaving them out, like you would have to bear the burden of explaining why, e.g., you decided to omit pyramidal neurons from your WBE strategy.

Next, we must consider the hypothalamic-pituitary stalk. Here we have an interesting interface between “standard neuron and neurotransmitter behavior”, and virtually every other system in the body: immune, adrenal, gonadal… digestive, hepatic.

The hypothalmic-pituitary system is not just an outgoing gateway, regulating sleep, hunger, temperature, but is part of numerous feedback loops that come back and effect neuronal function. Consider the HPA system, so studied in depression and mood disorders. Amydiala and cingulate centers (and the cortex, which promotes and demotes traffic to areas of the hypothalamus based on salience) trigger corticotropin releasing factor, which goes out eventually triggering adrenalin and noradrenalin, blood sugar changes... the whole course of alerting responses, which are picked up globally by the brain and CNS. Thalamic filters change routing priorities (the thalamus is the exchange switchyard for most sensory traffic), NE and DA levels change (therefore, so does LTP, i.e. learning, which is responsive to alerting neurotransmitter and neuromodulator levels.)

My point so far is: does “whole brain” emulation mean we emulate this very subtle and complex neurohomonal “gateway” system (to the other systems of the body?)

Lastly, just two more, because this is running long.

Muscles as regulators of brain function. It is now known that voluntary muscle activity produces BDNF, the neurosteroid that everyone had heard about, if they have heard about any of them. BDNF is produced locally in muscle tissue and can be released into general circulation. Circulating BDNF and GDNF enter the brain (which also makes its own BDNF too), and docks with (intracranial) neurons' nuclear receptors, changing gene expression, ultimately regulating long term plasticity, synaptic remodeling, and general neuronal health.

BDNF is hugely important as a neuromodulator and neurosteroid. The final common pathway of virtually all antidepressant effectiveness on mood, is predomantly through BDNF.

And MOOD is very, very cognitive. It is not "just" background emotion. It effects memory, judgement, sensory filtering and salience-assignment, and attentional target selection. Mood, is a potent kind of global neuromodulator. (In fact, it is almost arbitrary, whether one wants to consider mood disorders to be cognitive disorders, or affective disorders.) It effects information processing, both thermodynamic (sub-semantic) and "semantically coupled" (useful to us -- experienced in the first person as contents of our thoughts, or to a system programmer who ignores thermodynamic but carefully choreographs the flow of) information that carries his software's semantic content (that which bears intentional or representational content, as in a mathematical model of a jet turbine, or whatever it may be his program is designed to do. )

So, if you want to model the brains information processing,do you model muscle production of BDNF too? If not, your model is incomplete, even if you give the brain a virtual body. You have to model that virtual body's feedback loops into the CNS.

This is only scratching the surface. The more I have studied physiology, the more I have learned about what an intricately coupled system the body is, CNS included. I could add some even more surprising results I have seen recently. I have to stop now, and continue in next message (about why, despite this, it is worth considering anyway.) But my point about "Whole Brain" being not well- defined, in that there is no theory neutral place to draw the boundary, without making arbitrary judgments about what kind of information can be neglected and what must be considered crucial, should be clear from these examples. why we should do it anyway...Continued in a future message…

17

Superintelligence Reading Group 3: AI and Uploads

17

Summary

Notes

In-depth investigations

How to proceed

17

17

Superintelligence Reading Group 3: AI and Uploads

17

Summary

Notes

In-depth investigations

How to proceed

17

So, it seems to me that you have to emulate neuroglia, in WBE strategies (or else bear the burden of explaining why you are leaving them out, like you would have to bear the burden of explaining why, e.g., you decided to omit pyramidal neurons from your WBE strategy.

My point so far is: does “whole brain” emulation mean we emulate this very subtle and complex neurohomonal “gateway” system (to the other systems of the body?)