Whether current or near-term AI systems could be conscious is a topic of scientific interest and increasing public concern. This report argues for, and exemplifies, a rigorous and empirically grounded approach to AI consciousness: assessing existing AI systems in detail, in light of our best-supported neuroscientific theories of consciousness. We survey several prominent scientific theories of consciousness, including recurrent processing theory, global workspace theory, higher-order theories, predictive processing, and attention schema theory. From these theories we derive "indicator properties" of consciousness, elucidated in computational terms that allow us to assess AI systems for these properties. We use these indicator properties to assess several recent AI systems, and we discuss how future systems might implement them. Our analysis suggests that no current AI systems are conscious, but also suggests that there are no obvious technical barriers to building AI systems which satisfy these indicators.
Youtube description
This event took place on Tuesday September 5, 2023 and was hosted by the NYU Mind, Ethics, and Policy Program.
About the event
This panel discussion featured four authors from the recently released and widely discussed AI consciousness report. This report argues for, and exemplifies, a rigorous and empirically grounded approach to AI consciousness: assessing existing AI systems in detail, in light of the best-supported neuroscientific theories of consciousness. The paper surveys several prominent scientific theories of consciousness, including recurrent processing theory, global workspace theory, higher-order theories, predictive processing, and attention schema theory. From these theories the authors derive "indicator properties" of consciousness, elucidated in computational terms that allow them to assess AI systems for these properties. They use these indicator properties to assess several recent AI systems, and discuss how future systems might implement them. In this event, the authors summarized the report, offered perspectives from philosophy, cognitive science, and computer science, and responded to questions and comments.
About the panelists
Patrick Butlin is a philosopher of mind and cognitive science and a Research Fellow at the Future of Humanity Institute at the University of Oxford. His current research is on consciousness, agency and other mental capacities and attributes in AI.
Robert Long is a Research Affiliate at the Center for AI Safety. He recently completed his PhD in philosophy at New York University, during which he also worked as a Research Fellow at the Future of Humanity Institute. He works on issues related to possible AI consciousness and sentience.
Yoshua Bengio is recognized worldwide as one of the leading experts in artificial intelligence, known for his conceptual and engineering breakthroughs in artificial neural networks and deep learning. He is a Full Professor in the Department of Computer Science and Operations Research at Université de Montréal and the Founder and Scientific Director of Mila – Quebec AI Institute, one of the world’s largest academic institutes in deep learning. He is also the Scientific Director of IVADO. His scientific contributions have earned him numerous awards. He is the 2018 laureate of the A.M. Turing Award, “the Nobel Prize of Computing,” alongside Geoffrey Hinton and Yann LeCun for their important contributions and advances in deep learning. In 2022, he was appointed Knight of the Legion of Honor by France and named co-laureate of Spain’s Princess of Asturias Award for technical and scientific research. Later that year, Professor Bengio became the most cited computer scientist in the world in terms of h-index. He is a Fellow of the Royal Society of London and of Canada, an Officer of the Order of Canada and a Canadian CIFAR AI Chair.
Grace Lindsay is currently an Assistant Professor of Psychology and Data Science at New York University. After a BS in neuroscience from the University of Pittsburgh and a year at the Bernstein Center for Computational Neuroscience in Freiburg, Germany, Grace got her PhD at the Center for Theoretical Neuroscience at Columbia University in the lab of Ken Miller. Following that, she was a Sainsbury Wellcome Centre/Gatsby Computational Neuroscience Unit Research Fellow at University College London.
About the host
The NYU Mind, Ethics, and Policy Program is dedicated to advancing understanding of the consciousness, sentience, sapience, and moral, legal, and political status of nonhumans, including animals and artificial intelligences. We believe that all beings with the capacity to suffer deserve to be treated with respect and compassion, and that our policies and practices should reflect this. Our goal is to promote research and scholarship that will help to shape a more just and humane world for all.
* * As it happens, this program description was co-authored by GPT-3, a large language model! Thank you to our co-sponsors for your generous support of this event:
NYU Center for Mind, Brain, and Consciousness
NYU Center for Bioethics
Transcript
JEFF
Hi everybody. My name is Jeff Sebo. Welcome to this event. Thank you for being here during what I know is a very busy week and day for everybody. This is the first day of class at NYU, and the fact that we have an event today with this many people interested, I think is a testament to how interesting the topic and the report that will be the basis for our conversation are.
So this is an event hosted by the NYU Mind Ethics and Policy Program, and we are also grateful to our co-sponsors, the NYU Center for Bioethics and the NYU Center for Mind, Brain and Consciousness. Thank you for co-sponsoring this event. The NYU Mind Ethics and Policy Program examines the nature and intrinsic value of non-human minds, including biological and nonbiological minds, with special focus on invertebrates and artificial intelligence systems. We are interested in examining what kinds of beings can be conscious and sentient and sapient, and what kinds of moral and legal and political status they should have.
And so we were very interested when this team released this report about a week ago, a report on what leading scientific theories of consciousness have to say about possible AI consciousness. And so we thought it would be useful, especially since the report, as soon as it was released, was generating a lot of discussion. It received coverage in Nature and Science and a lot of conversation on the Internet. We thought it would be valuable to bring some of the authors of the report together. There were 19 authors from different fields and different institutes, and we thought we could bring four of them together, including the two lead authors and representatives of other fields as well, to talk about the report, summarize some of the main points, and then offer their own individual perspectives about some of the implications.
So I will now briefly introduce the speakers, and then we can hear some remarks from them, and then we can open up the discussion and hear questions and comments from you and have a conversation. So here are the speakers.
Patrick Butlin is a philosopher of mind and cognitive science and a Research Fellow at the Future of Humanity Institute at the University of Oxford. His current research is on consciousness, agency and other mental capacities and attributes in AI. Robert Long is a research affiliate at the center for AI Safety. He recently completed his PhD in philosophy at New York University, during which he also worked as a research fellow at the Future of Humanity Institute. He works on issues related to possible AI consciousness and sentience. Yoshua Bengio is recognized worldwide as one of the leading experts in artificial intelligence, known for his conceptual and engineering breakthroughs in artificial neural networks and deep learning. He is a full professor in the Department of Computer Science and Operations Research at the University de Montreal, and the founder and Scientific Director of Mila Quebec AI Institute one of the world's largest academic institutes in deep learning. He is also the winner of many awards and has many other accomplishments that you can easily find online. And finally, Grace Lindsay is currently an assistant professor of Psychology and Data Science at New York University. After a BS in neuroscience from the University of Pittsburgh and a year at the Bernstein Center for Computational Neuroscience in Freiburg, Germany, Grace got her PhD at the center for Theoretical Neuroscience at Columbia University in the lab of Ken Miller. Following that, she was a Sainsbury Wellcome Center Gatsby Computational Neuroscience Unit Research Fellow at University College London.
So thank you all so much for, first of all, writing this very interesting report, and second of all, being here to talk with us about it. I know a lot of people here are interested in hearing from you and discussing this with you. Just a note to everybody that the format and structure of this event will be that we will hear brief remarks from each of the four authors and speakers. Patrick and Rob will speak together about a summary of the report, followed by a philosophical perspective. And then we can hear cognitive science and computer science perspectives from Grayson and Yoshua, respectively. And all along the way, people in the audience can submit questions and comments, objections, whatever you like in the Q/A tab on zoom. And then when we reach the discussion portion of this event, I can read questions, comments, objections that have been entered and upvoted from the Q/A tab. So please go ahead and open that up and add your questions and comments whenever you like throughout the session. Okay, so with that in mind, without further ado, we can start the show. So, Patrick and Rob, whenever you are ready, feel free to share your screen and tell us about your report.
ROBERT
All right, so here we go, Jeff. And if you can, let me know that everything is in working order.
JEFF
Everything is in working order. Yeah, go for it.
ROBERT
Excellent. So let's begin.
PATRICK
Yeah.
ROBERT
First, I just want to say thank you so much to the audience for turning out. We're extremely excited to discuss these issues with you. A big part of why we wrote this report is to discuss these topics more widely and hear from different people. And thanks so much to you, Jeff, and to the NYU Mind Ethics and Policy Program for hosting this. So, my name is Robert Long, and along with Patrick Butlin, I'm one of the lead authors on this report. As Jeff mentioned, there are also 17 esteemed colleagues from various disciplines, and I'm going to briefly say what we did in the report and why we did it and why it matters. And, I mean, Patrick will be covering some of that.
So you've all probably read about the case of Blake Lemoyne, a Google engineer who was fired from Google after he became convinced that an AI system he was interacting with was conscious. You've probably seen the increased deployment of chatbots, including romantic chatbots, that interact with people in very human-like, compelling ways. And you may have seen a tweet from OpenAI's chief scientist Ilya Sutzkiver saying that it may be that today's large neural networks are slightly conscious. So AI consciousness is obviously a topic of increasing public concern and public interest, and it's also a perennially fascinating scientific question. Speaking for myself, as AI consciousness has become a bigger topic recently, I've often been frustrated by some of the characteristics that conversation about it has. I think when people talk about AI consciousness, there's often a lack of specific scientific evidence. There's very little detailed analysis of existing systems. The conversation is often kind of emotive and highly charged and often in dichotomies. Like how could AI systems possibly be conscious? We know that that's absurd. Or on the other hand, people being certain that they definitely are conscious. And what we wanted to provide in this report is kind of a better footing for these kinds of conversations. We want to bring a more nuanced and a more evidence based approach to the report. So in addition to the actual findings that we have, we really want to spur a conversation and spur further research along these lines.
So one aspect I'm going to talk about briefly is also conceptual clarity, something that philosophers love about what exactly we're talking about when we talk about AI consciousness. And before I do that again, just an advertisement for the paper. We put it up on Archive about two weeks ago, I think, and you can find it there.
And there is a list of this really wonderful team that helped us out. We have people from neuroscience, AI and philosophy. So there it is. So what are we talking about in this report? It's called consciousness and AI. Insights from the science of consciousness. It's very important both in this talk and we do this in the paper to pin down what we're talking about. So what I'll be talking about and what we discuss in the paper is what philosophers call phenomenal consciousness. And this means subjective experience. Thomas Nagel famously said that consciousness is what it is like to be an entity. So, for example, if you've tuned in, you're now currently listening to my voice, you're seeing my face and this wall on the screen, and there's a subjective aspect to your experience. There's something that it's like for you to be seeing a white wall or hearing my voice. And what we're asking about in this report is whether AI systems now or in the near future could have subjective experiences in this way, if they could be conscious in this sense.
And it's very important in conversations about this to be clear about what we're not talking about. We're not talking about whether AI systems are intelligent in a certain way. We're certainly not talking about AGI or artificial general intelligence. We're not talking about language, understanding or rationality. And also we're not talking about whether AI systems experience the world in exactly the way that humans do. So Thomas Nagel, when he introduced this phrase, famously asked what it's like to be a bat? It's very plausible that bats might be conscious, that they might have subjective experiences of the world. But if they are, the way that they experience the world is presumably very different than the way we experience the world. And that also comes along with the fact that the way they think about the world is presumably very different and they don't have the full suite of human cognitive capacities. So when we're talking about AI, we're wondering about consciousness in this sense and not necessarily any further assumptions about human like intelligence or experiences.
So how do we go about this in this report? Here's what I think are kind of the main foundations of what we do. First of all, we want the conversation to be empirical. So that means that we're looking at scientific theories of consciousness. So there's a broad field of consciousness science where scientists examine the human brain and animal brains and ask what sort of processes are associated with having conscious experiences. And this is somewhat distinct from philosophical questions about consciousness. We're going to look at these empirical theories and see what they can tell us about potential AI consciousness. Another key aspect is that we take this working assumption of this very broad thesis about the nature of consciousness, which we call computational functionalism. And very broadly, this is saying that what matters for consciousness is the computations that a system performs and not necessarily what it's made of. So this is compatible with consciousness coming from computations done by biological neurons or computations happening in silicon, transistors and chips. Patrick's going to talk a little bit more about exactly why we made this assumption and what it amounts to. And then we try to get precise. We try to derive indicator properties from the science of consciousness that we can use to examine different AI systems, and then we apply these to a broad class of AI systems from several different kinds of research programs, several different kinds of systems. And lastly, we're not aiming for absolute certainty. It's my opinion, and I think most of my co-authors, if you're asking for absolute certainty about AI consciousness, you are going to be disappointed because there still is a lot we don't know about consciousness. That's in spite of the fact that we do know some things, which is what we try to show in this report.
So it's a long report. So I would like to just talk about one specific example of this methodology and how it works in the paper. So one of the most prominent scientific theories of consciousness is called the global Workspace theory. And at a broad level, the global workspace theory says that consciousness is about global broadcast throughout a system. So it has this picture of the mind where there are these specialized independent modules that are responsible for different kinds of information processing. So for example, as you can see in that diagram, one such module might be Perceptual processing. So Vision is responsible for visual information processing. Or maybe there's a module dedicated to making decisions and choosing actions. The global workspace theory says that there's also a global workspace that will select the output from one of these usually independent modules and then broadcast that to all of the modules. This is a way of coordinating all of the processing that they are each doing individually. So consciousness is related to global broadcasts of information throughout a system, and a system having a certain architectural or information processing property that's related to the computational functionalist assumption that we have. And importantly, as we'll see, there needs to be a certain kind of feedback loop between modules and a global workspace. The global workspace taking input from the modules and then also broadcasting it back to them.
So from that we get some indicator properties to say what kind of features does an AI system need to have if it's going to satisfy, at least according to global workspace theory, the indicators that might be associated with consciousness. So one of the top questions you might have is, well, what do we say about large language models? So briefly, large language models, you're probably all familiar with them, these are some of the most prominent systems today. The GPT systems are an example. These are often built with what's called the transformer architecture. And now is certainly not a time for a full explanation of what that amounts to. But one key aspect for our analysis is that the transformer architecture is feed forward and that is to say when the system is taking in an input, say, a certain amount of text and it's going to produce an output, which is the text that it will continue to produce. For example, when you're chatting with it as it's processing that information, the information just goes from the input layer to the output layer kind of continuously. And when we look at this architecture and ask whether the way that information is flowing through, it kind of fits with the picture from global Workspace theory. Our provisional analysis in the paper is that as far as we can tell, there's nothing like a global workspace that is both receiving and broadcasting information to and from anything like individual modules. To put it another way, there don't seem to be the right kind of feedback loops in the transformer architecture for it to be satisfying these particular indicators of consciousness. So our answer in the paper is probably not according to global workspace theory, they do not satisfy these indicators.
So that's the kind of thing we do in the paper. Take a scientific theory, ask more precisely what kind of architectural and informational processing features it says are associated with consciousness. Look at how an AI system works and see how much it matches that. As I said, we do this with a lot of theories. We have, like, five very prominent scientific theories of consciousness that we look at. One thing that's worth remarking on is, why do we look at five different theories? Why not just, for example, go with global work theory? And that's kind of related to our points about nuance and uncertainty. We do think that the science of consciousness is making progress on discovering the processes associated with consciousness, but we think it's premature to commit to one particular theory. There's nothing like a consensus in consciousness science about which one of these stories is the right one for consciousness. So we kind of want to cast a broad net and say, here are a variety of things that might be associated with consciousness according to different theories, and let's look at all of them.
Jeff, I saw that my Internet said it was unstable. Did I skip at all? Are we good?
JEFF
Briefly. But I heard everything you were saying.
PATRICK
Cool.
ROBERT
And yeah, as mentioned, one thing that we think it's important to do is look beyond just large language models. They have understandably received a lot of the attention. They're very compelling. Their capabilities are extremely impressive and most saliently. They sometimes say that they're conscious, so it's understandable that they're one of the main systems that people wonder about. But at least speaking for myself, I think when you look at the way that they're built, they're not necessarily even the best current candidates for consciousness.
Among the AI systems that exist today. For example, many people point out that large language models don't seem like they're agents, or they don't seem embodied. They don't seem to interact continuously with an environment which you might think is important for consciousness. But many systems are plausibly much closer to doing that sort of thing, which is why we look at systems that are from robotics or that navigate virtual environments or that work in different ways than large language models do. And I will now turn things over to my colleague Patrick.
PATRICK
Thanks, Rob. Hi, everyone. So Rob's talked a bit about what we did in this project. I'm going to talk briefly about why we did things in the way that we did, and then I'm also going to comment very briefly on what's next for research in this area.
So next slide, please, Rob. Okay, so on the topic of why we did things the way we did, the first question I want to say something about is why did we make the assumption of computational functionalism? And this is an important question, because computational functionalism is quite a contentious hypothesis. So just to remind you about what this claim is, computational functionalism, as Rob said, is the claim that what matters for consciousness is computations. So according to this view, the reason why humans are conscious is that the right kinds of computations are going on in our brains. And what it would take for an AI system to be conscious is for it to perform the right kind of computations. Now, this is, as I just said, quite a controversial hypothesis. Many researchers in this area doubt it. For example, Anil Seth suggests that consciousness may require living cells, and there's many other researchers who take similar views to that position. So why do we make this assumption? Well, there's a few factors that play into this. One is that we'd think that computational functionalism is a plausible view. That doesn't mean to say that we're confident that it's true. I think there's probably a range of views about computational functionalism among the authors, but as a group, we're certainly not confident about this. It may even be too strong to say that we think it's more likely to be true than not true. But crucially, we think it's plausible enough that it's worth exploring its implications.
Another consideration is that many scientific theories of consciousness are expressed in computational terms. And also, if computational functionalism is true, then there's an important further question about which AI systems perform the kinds of computations which are associated with consciousness. So ultimately, the reason why we make this assumption is that we think it's a productive one in the sense that by working starting from this assumption, we can reduce our overall uncertainty about consciousness in AI to a significant degree.
So we're uncertain about whether computational functionalism is true. We're uncertain about which specific scientific theories of consciousness are true. We don't know which AI systems are conscious. But if we kind of explore this area of the space of possibilities, we think we can make progress, so we think we can reduce our overall uncertainty.
Next slide. Another question about why we did what we did is why did we focus on the internal processes that AI systems go through rather than on their behavior or on their capabilities? Why do we think that thinking about internal processes is a better way of determining whether they're likely to be conscious? Well, this is also an interesting question. And it's interesting because there's this kind of important challenge to the approach that we took, which is that the science of consciousness is relatively immature. There are still lots of competing theories in this science. And also, crucially, these theories are based mostly, although not entirely, on data from humans. So one might very reasonably have doubts about whether these descriptions of what's going on in humans when humans have conscious experiences can be extended to provide a guide to consciousness in non humans. And in particular, that kind of argument has been made in debates about non-human animal consciousness. Our colleague Jonathan Birch, for example, who's a leading researcher in that area, argues that in the case of non-human animals, when you're trying to work out which are conscious, it's more productive with science in its current state. To concentrate on examining the capabilities of non-human animals rather than on trying to work out whether the internal processes in their brains are similar or not to the internal processes in human brains.
But in fact, we think that different methods make sense in these two cases, that is, the case of non-human animals versus the case of artificial intelligence. So, next slide, please, Rob. And the key difference which makes it the case that different methods are appropriate in these two cases is that we know that on the whole, animal brains work in relatively similar ways to human brains. And we can know that because of our shared evolutionary heritage even prior to doing detailed animal neuroscience. On the other hand, there's a relatively wide space of possible designs for AI systems. There are lots of different possible architectures and methods that you can use in AI in principle. So this kind of space of possible internal processes in AI is much wider than the space of processes which we're likely to find in animal brains. Now, there are a couple of different consequences of this .1 is that in the case of AI, it's relatively informative to find similar internal processes going on in a system to the processes which seem to be associated with consciousness in humans. That's just because AIS could relatively easily be different from humans. In the case of animals, finding similar processes is not so informative. But then on the flip side, finding similar capabilities and similar behaviors to those exhibited by humans in AI is less informative than finding similar capabilities in the case of animals.
And that's because it seems that in AI it's possible for very different underlying processes to give rise to superficially similar behaviors and capabilities. So there's a specific version of this problem which clearly arises in AI that exists today, which is that AI systems are sometimes designed specifically to imitate human behavior. So we know that there are AI systems which have superficially, but also in a very compelling way, humanlike capabilities, human-like behaviors which also work in quite unhuman-like ways. For that reason, looking at capabilities looks like, on the whole, an unreliable method, unless it can be very substantially refined in the case of AI.
Next slide. Okay, now turning to the question of what's next for AI consciousness research. Well, I'm sure there are lots of exciting directions which could be pursued at this point. But one which I'm interested in, I think Rob's interested in as well, is understanding what's called valenced conscious experience. So of course, as we're all familiar with, some conscious experiences feel good, like feeling a cool breeze on a hot day. Another conscious experiences feel bad like feeling pain or fear. These are famous conscious experiences. What we say is that the ones that feel good have positive valence and the ones that feel bad have a negative valence. But it seems as though in principle, there can also be neutral conscious experiences which don't have a valence one way or the other. So the question is, could we find indicators for specifically valenced consciousness in AI? That seems to be a question which goes beyond the general one that we were asking in this report.
And we think that this question is important because it seems that pleasure and suffering have special moral significance. Next slide. Now, that brings us to the topic which is one of the major motives for our reports and which we're certainly going to be talking about further today, which is this point that a big part of the reason why thinking about AI consciousness matters. Why working out how to assess whether AI systems are likely to be conscious matters is because our society is soon going to have to decide whether to build systems which could plausibly be conscious. So there's a huge, very difficult philosophical question which is what moral principles should we use to make this momentous choice? I certainly don't know the answer to that question. It's possible we'll explore it a little bit in a few minutes. But we do think that there's a simple step which could be taken now, which is to continue the kinds of work that we've been doing to try to understand what might be good indicators of consciousness in AI. And then in particular, for groups engaged in building AI systems to recognize the possibility that they might build conscious systems, that they might do so even without trying to do just that. And for those groups to develop methods for assessing whether the systems that they're working on are likely to be conscious.
And we certainly strongly recommend that they proceed with great caution if they get a positive result when they're doing that kind of assessment. So thanks very much for listening, everyone. Thanks again to our collaborators and I'm looking forward to the discussion. Thanks.
JEFF
Great. Thank you so much, Patrick and Rob. Now, Grace, do you have any thoughts from your perspective to share?
GRACE
Yeah, I just wanted to talk a little bit about kind of the state of consciousness research in neuroscience to give context to the theories that are discussed in the report and kind of, I guess to situate ourselves maybe in the history of science here. So I should say I'm not directly a consciousness researcher myself, but I do study attention which interweaves with consciousness studies in various ways, but so I don't have a dog in the fight of these different theories, which is possibly a good position to be in to discuss the overall state of things.
But yeah, it is the case that the neuroscientific study of consciousness in terms of being a proper scientific field of research is pretty new. I mean, you could argue neuroscience itself is pretty new in the whole scheme of history of science, but certainly people taking the scientific study of consciousness seriously is definitely new. Because there was a joke that you had to be tenured to study consciousness. And so the idea that now there are actually full labs devoted to this and people are really trying to make it rigorous and thorough and you have these theories and everything that's definitely progress, but it points to the fact that this research is still in its infancy. And so the theories that are laid out here are the major theories that are discussed amongst these researchers. But from my perspective, I don't think there's a sense that these are anywhere close to the final drafts of these theories. And that's important for when you then step through the conclusions. And just the fact that Rob said the report doesn't choose a specific theory to go with because there isn't consensus.
There's these multiple theories that in many ways have conflicts with each other. And so it really is still a young area. And also the way that the scientific study is framed in order to be precise and rigorous. It's usually framed more as studying the neural correlates of consciousness. So not even trying to make a strong claim necessarily about causal connection, but just what do you observe the patterns of neural activity to be when a subject reports a conscious experience? And that's another detail. It's really a lot of times the neural correlates of conscious report. What are people saying that they experienced as their conscious experience? There is a detail of kind of these no report paradigms, but they still ultimately rely on the fact that at some point a human was able to say that they experienced something. And so those are also caveats that bound the scientific study of it and are necessary to make it a scientifically rigorous thing to study. But obviously that's going to from a philosophical perspective, that's going to have implications as well.
And yeah, the other thing that I kind of wanted to talk about in kind of situating this sort of thing so the scientists are going into the brain and it's messy and there's a lot of stuff going on. And the hope is to find the simplified principles that correlate with this conscious report and correlate with people saying something is conscious versus not. And so there the work is to take the big, messy, complex thing and try to come up with the simplest description of what you need in order to get conscious report. When you then actually kind of look at that in isolation, sometimes those indicators as the report kind of turns them into seem really simple.
And I think we have to keep in mind that these theories were not developed for the purposes of trying to see if an AI system is conscious. They're developed in the context of assuming that a human is conscious and looking at their neural activity or even a lot of at least the kind of background knowledge for these theories comes from non human animal work and so they're understanding where they're coming from in that sense. The fact that they're not designed to be directly translated into these sort of easy to identify computational principles that could be in artificial systems, I think is important. I think it's important for this work of trying to take a theory and assess an artificial system. But I also think that there's a lesson for the people, the neuroscientists who study consciousness in this as well. Because as this happens a lot, when you do mathematical modeling, you can be working with a topic area and kind of think you have a mental model of how it works.
And then when you actually go to write it down, you realize some aspects are lacking, maybe or the pieces don't fit together the way that you thought they did. And it's the turning of the kind of mental model and the pile of experimental data and the word models that people use to describe how they think something functions. When you actually have to turn that into an equation or code or actually try to build it, you can kind of see where you might be missing things.
And I think that this is a nice opportunity for the neuroscientists who study consciousness to see their theories in a new light when they're kind of put into these cold, stark indicators and really reflect on if that is summarizing everything that they think is important or that there are things about the brain that are kind of going unspoken that they think are actually really important as well or things about the abilities of humans or animals that are important as well. So, yeah, I think that's important to keep in mind that these theories were not designed to lead to a description that is used for AI. But it's still a very helpful exercise in my mind to go through this and see what they look like in the end, when they're kind of pared down to the simplest form that can be translated into an AI system.
So overall, I think the report has benefits to be able to take this neuroscience literature and bring it to an AI audience and put it in those terms, but also should have benefits for the neuroscience side itself in terms of thinking about how these theories really pan out, how they relate to each other, what they could be missing. If the scientists who created and worked on these theories, if they would agree with the conclusions of the report or even agree that an artificial system that had these properties was conscious. In the end, I think that that's an important thing for those scientists to reflect on. Great.
JEFF
Thank you so much, Grace. And finally, Yoshua, do you have thoughts to share?
YOSHUA
Sure. Maybe I'll start with talking about the computational, functionalism, computational basis of consciousness and subjective experience. We've been using the word consciousness, but really I think it's important to clarify that the word consciousness can be associated to all sorts of things and we're trying to focus on subjective experience, which is the part that may seem very mysterious to many people, including researchers. My personal view, so maybe not the unanimous view of the others, is that physics is computation and many physicists share that view. It's just a bunch of equations that could be implemented in any way you want.
At the end of the day, you get the same changes in the state of the world and your brain is physics too. Now, I think some of the questions about how this could be turned into computations in a computer may have nothing to do with something non material that could possibly be happening in biology. Maybe there is something about the physics like it requires quantum computations that maybe we don't know yet how to do. But actually, if we look at the progress of AI in the last few decades, we're moving forward quite rapidly towards very strong capabilities and we never seem to be requiring any kind of quantum computation in order to get that power, which of course doesn't guarantee that it's continued to be the case.
But that suggests that the level of abstraction that, say, neural networks used in machine learning have is already doing a good job for providing a lot of the explanations and neural correlates of our abilities. So another interesting question that has to do with AI research is why are we conscious in the first place? And the perspective that would come naturally to machine learning researchers or AI researchers is evolution notice with these forms of computations because that gives us an advantage either individually or collectively. There's a social aspect to consciousness as well. And so if there is an advantage, then it's something worth investigating from an AI perspective. It may be something that AI researchers want to put into their AI systems, which is a question I'll come back to that Patrick talked about: do we want to have conscious machines or not?
So one of the things my group has been working on is precisely this. So some of these theories of consciousness, in particular the global workspace theory and attention schema theory and others, really can be interpreted as providing an advantage in terms of our ability to learn and manipulate abstraction. So this is connected to the property of thoughts and attention, selecting very few bits of information that go through working memory at any moment.
And then we sequentially go through a very small number of bits in this way that help us take decisions and organize our understanding of the world at a very abstract level, that compresses information, perceptual information in a way that helps us better understand the world, take better decisions, better model it and so on. Now let me go back to this question of whether AI are conscious or will be in the future. So our report suggests that none of the current AI systems have enough of the characteristics that those theories suggest. First of all, the different theories we chose are not the end of it. So this is a continuously moving field. There, you know, are newspapers coming regularly suggesting other variants often related to existing theories. So we shouldn't take these as the end story of how consciousness works in the brain. And also what the report suggests is actually those properties, those in these theories or maybe other ones that could be related that may come up in the near future. Those attributes are not impossible to put in AI. So it's very plausible that in coming years we would be able to build machines that compute in ways that are at least suggestive of consciousness in the human sense.
And I think this raises a lot of important questions. My take on this is we should not build conscious machines until we know better. There are many reasons for this. In particular, whether we succeed in building machines that are actually conscious or not. If humans perceive those AI systems as conscious, that has a lot of implications that could be extremely destabilizing for society. We associate moral status to other conscious beings, and that's connected with a very strong social contract which works for humans. We have all kinds of characteristics. We have a finite lifetime. We are bounded sort of intellectual capabilities.
And these properties may not apply to AI that can reproduce. There's no limit in how you could copy an AI system over and over. So they might be like, immortal, they might be much smarter than us. All sorts of things that I think would make the current interpretation of consciousness at a moral and social view questionable. I think until we know better, we shouldn't do that. There's another, more pragmatic reason why I think we shouldn't build conscious machines. Because with consciousness also comes a notion of self and even self preservation objectives like agency. This was one of the theories that was described. And if we go on that route, this could be very dangerous from an AI safety point of view. In other words, we might be building machines that have their own goals that are not well aligned with human norms and values in ways that could be extremely dangerous for humanity.
And of course, this is a subject that's intensely debated in the last few months, which makes this report particularly interesting. But this is one point I want to emphasize. Let's see the last point I want to make that's not really in the report, but connected to what we talk about in the report. But comes some recent work coming out of my group just in the last few months suggesting so it's another theory of consciousness that is completely computational. It's related to several existing theories. But what's interesting about this one is that subjective experience with all the attributes that we associate with it, like ineffability and subjectivity and richness and fleetingness, these properties emerge of this model as side effects of the need to perform a particular kind of computation that is important from a learning perspective. But you could obtain potentially the same computations with a different implementation that wouldn't have these side effects. So evolution has sort of converged in this particular way of achieving particular useful computations that may give rise to our sensation of being conscious and having free will and all these things to which we attribute a little bit of sort of magical properties. And we should be careful about that instinct we have about our own perception of being conscious in light of those results from neuroscience and AI. I'll stop here.
GRACE
Great.
JEFF
Thank you so much, Yoshua and everybody, for those remarks. Very interesting. It gives us a lot to talk about. I want to flag both for the panelists and for everybody in the audience, that a very lively discussion is already happening in the Q/A tab. And so I encourage everybody to go check out the Q/A tab and read through some of the questions and comments that have already been entered and feel free to reply directly to each other and have those conversations. We can have one conversation here and another in the Q/A tab. I will jump right into questions that attempt to synthesize and present you with what people are asking about in the Q/A tab. I will not be able to get to everything, but please know, everybody, that we will send all the questions to the panelists after the talk, whether or not we can get to them during the presentation. So some of the questions are descriptive, others of the questions are moral or legal or political in nature. So I can start with a general descriptive question for you. As you noted in the initial presentation and some of your remarks, and as several people have noted in the comments, you focus on a particular perspective about consciousness, according to which consciousness is about computations.
And so you look at scientific theories of consciousness that identify different computations, and then you search for markers related to those theories, and then you look for those markers in particular kinds of AI systems. And as some people note, that does not exhaust the space of theories and perspectives about consciousness that are plausible and popular right now. So on a more permissive end of the spectrum, as one person notes, there are, for example, panpsychist theories of consciousness and other theories that are relatively undemanding that allow for the possibility that even quite simple systems could be conscious. Those theories might imply that lots of systems can be conscious, whether or not they have your markers. And then at the other more restrictive end of the spectrum, you have biological theories according to which, for various reasons, you really do need to be made out of, for example, carbon based cells and neurons in order to realize consciousness.
And according to those theories, a system can hit all of your markers but still be non conscious if that system is made out of silicon based transistors and chips. So I wonder, on a personal level, to the panelists, what kind of credence do you have in these more permissive or more restrictive theories? Do you find them plausible? Do you find them good candidates? And how would you incorporate them into your search for AI consciousness?
ROBERT
I can hop in first. Yeah. I think in people in the report, I'm probably on the higher end in my credence in computational functionalism of some kind, I'm maybe like 70%. But I am very compelled by arguments by people like Anil Seth and Peter Godfrey Smith, philosopher of biology, who has written extensively on consciousness. So I do sometimes wake up in the middle of the night wondering if computational functionalism is off on the wrong track.
And I'll also just take this opportunity to say, one thing we call for in the report is more detailed work investigating the assumption of computational functionalism. We think it's again, sufficiently implausible that it's really very important to explore its implications. But we could also get a lot more clarity on these issues if arguments for and against computational functionalism got hashed out in more detail. And I'll lastly just say I wanted to plug a really nice remark by Anil Seth that I think is really exactly the kind of response we wanted where he said, I disagree with some of the assumptions, and I'm guessing that's computational functionalism, but that's totally fine because I might well be wrong. So we're very excited to see people kind of exploring different parts of the space of possibilities that we could be facing with AI consciousness.
PATRICK
Great.
JEFF
Thank you, Rob. Yoshua?
YOSHUA
Yeah, so my credence on computational fictionalism is 99.99%, maybe because I'm a computer scientist, right? And the whole field of computer science is founded on the idea that computations can be done on any substrate. And there's not been any example of that that exists, as far as I know. It's not just AI. It dates back from Turing and the Turing machine around the Second World War. And it's also connected to, as I said in my little pitch, everything we know from physics. So I don't see how having carbon atoms prevents computations from explaining what's going on. It's just a different kind of computation. It may not be the computations going on at the level of these artificial neurons that you typically find in deep learning. That's very possible. But it's still computation. It's just now computation happening at the atomic level, but it's still computation. What's the level of abstraction that's needed to replicate human intelligence and consciousness? Well, nobody really knows, and that's open. But I think the question of whether you need something that's non computational, because for me, if it's not computational, it's not even physical, so it's not even materialistic. And I don't see how you could buy that unless you believe in some spiritual beings explaining our consciousness.
And I also have a comment on panpsychism. How could I say it feels like it's completely overgeneralizing. The things we know that are conscious are human beings. And because of many similarities, we have some maybe good reasons to think that other animals may be conscious. But everything we know about human consciousness has the kind of properties that we discuss in the report, for example, that are completely disjoint and not applicable to just arbitrary groups of atoms or even single electrons or whatever crazy things that you could come to with these theories. So I'm not saying these theories are false, but they seem so far removed from the biological reality, like what happens when a person is conscious or not conscious, that, for me, they don't rate very high as scientific theories that are supposed to explain what we know about consciousness. They may feel good again, because I think it may make us feel good with our intuitive religious understanding of the world, but in terms of matching what we observe, the correlates of consciousness seem pretty much bringing zero information.
JEFF
All right, great. Thank you. Grace or Patrick, did you have anything you wanted to.
PATRICK
I mean so when you asked about credences in computational functionalism, I guess the thought is that to the extent that we're doubtful about computational functionalism, maybe we're doubtful about the value of this project to kind of reveal whether AI. Systems are likely to be conscious or not and therefore, whether they're likely to have a certain special kind of moral significance or not. And for me, I think how my credences fit together is something like this: if there's such a thing as consciousness, if the concept of consciousness makes sense and is a useful one to apply beyond the human case, then I think most likely it's a computational phenomenon. I think I'd give more credence to the computational view in that situation than to non computational views, because I think the computational views have got more promise in explaining the properties of consciousness. But what keeps me awake at night to go back to what Rob said, is the possibility that the concept of consciousness is somehow confused or that.
YOSHUA
It.
PATRICK
Doesn'T make sense to apply it beyond the human case that it's unproductive for moral theorizing or conceptually confused in some way to ask the question whether AI systems are conscious or not.
YOSHUA
But, Patrick, I don't think these two views are incompatible. So I actually think that consciousness is computational in nature, and that is confusing and kind of not clear that it's meaningful to extend that concept very far from human beings, especially regarding the moral aspects of things. That's what I mean. And the social and moral aspects, I don't think these are incompatible points of views.
PATRICK
Yeah, I think our views are pretty similar.
JEFF
Grace, did you have anything you wanted to add?
GRACE
I mean, in terms of computational functionalism? My gut is that it's largely correct or certainly there will at least need to be a common set of computations and then maybe there also needs to be other stuff. But on the whole, I just feel like we're several paradigm shifts away from really understanding all of this. So it's hard for me to say anything with any confidence or vigor.
JEFF
Yeah, that seems like the answer about which we can be most confident. Okay, great. Thank you everybody. I can now ask a question on the moral, legal, political side, and again, several people have asked questions along those lines as well. So I think, as you yourself said, part of why so many people are so interested in this topic is because we do associate consciousness and then related capacities like sentience, the ability to consciously experience positively and negatively valenced states like pleasure and pain and happiness and suffering. Many people associate that reasonably, in my view, with a certain kind of intrinsic moral and legal and political significance. The idea being that if you have consciousness and or sentience, if there is something it is like to be you, and if it can feel good or bad to be you especially, then you matter for your own sake. And I should consider your interests and needs and vulnerabilities when making decisions that affect you and that might extend to a decision about whether to create you in the first place, as well as a decision about how to treat you if and when I do create you.
And so I would like to ask all of you if you care to respond. First of all, do you associate consciousness or sentience with that kind of intrinsic moral or legal or political significance? Do you think that when a being is conscious or sentient or sufficiently likely to be conscious or sentient by our lights, that we should extend them a certain kind of intrinsic value and consider their potential interest, needs, vulnerabilities when making decisions that affect them? And since we are making these decisions under uncertainty, I also wanted to ask a little bit about how you think about the risks associated with false positives and false negatives, with potential over or under attribution of consciousness and moral status. What are the risks associated with accidentally seeing an object as a subject and what are the risks associated with accidentally seeing a subject as an object? And how do you weigh those when deciding how to calibrate this kind of test and practice? So yeah, do you associate this with moral status and how do you deal with this under?
YOSHUA
Yoshua, I'm not a philosopher, so take that from the angle of a computer scientist. But my interpretation of this question is we are asking the wrong question. It's not whether we should attribute moral status to entities that have particular properties, like being conscious or something like that. Is that that's how we are? Humans are compelled to have empathy and compassion for other types of beings, in particular other human beings, because that's something that evolution put in us, because it helped us to help humanity to succeed and maybe become a dominant species. There are exceptions. You have sociopaths and so on. But for the most part, humans have those innate feelings. And by association, because our brain works by association, we often generalize that to other entities that look like us, mammals in particular, or we also have very strong empathy for babies of other species, right? My partner would not eat meat, but especially if it's coming from the baby of the species, it's not coming from a philosophical kind of argument. It's just an innate thing that we have. I can share that feeling, maybe not as strongly as she does, I think females in general.
So I think we're just asking the wrong question. And when we come to this for machines, I think it would be a huge mistake to build things that would play into our innate response mechanisms towards entities that look like us. So there was this black mirror episode where there are AI clones of a person in a virtual world that we feel for because they are so humanlike, even though it's just a simulation. Right? We can't prevent ourselves from attributing a moral status to those virtual agents because they look human. And I think that's the reality. What we do with that, I think, is then social norms to not break the way our society works with the introduction of entities that don't correspond to something we evolved for but is not going to be true anymore, with machines that could be potentially imitating us in many ways and maybe even have some of the attributes we put in the report. But is that really what society needs? I think that's a big question mark.
PATRICK
Thank you.
JEFF
Very interesting, Rob.
ROBERT
Yeah, so I think my views on this are similar to Yasha's in many ways on the question of what the grounds of moral status are, as philosophers would say. For my own part, I'm quite confident that if an entity is sentient, that is, if it has valenced conscious experiences like pleasure and pain, that that alone is sufficient for us to care about it and show concern for it. Which is why I would be excited for the project that Patrick mentioned. I'm less clear on how to think about entities that are merely conscious, that maybe only have neutral experiences. You could imagine a future large language model maybe that fits more of our indicators that only had experiences of understanding or maybe even some very abstract conscious experience that we can't even comprehend. I would obviously be extremely careful in how I treated that thing, but I'm a little bit less sure how to think about that case.
And then lastly, I just wanted to flag one very characteristic element of how we like to think about this in the report is about uncertainty and managing all of the different cases that might come up. And so I also wanted to flag consciousness itself could be too narrow of a thing to focus on, and we don't want to put all of our focus on that. There are, I think, compelling arguments that even if something is not conscious, if it has desires or goals that it wants to pursue, then that itself is something that should be respected. So I would also love to see equivalent or analogous reports on whether AI systems could have the kind of preferences or desires that might merit consideration.
YOSHUA
It's already the case. I mean, that lots of reinforcement learning agents have valence and have goals. It's like no rocket science here already exists.
ROBERT
So just very quickly on that, I'll just direct you to Patrick's work. Patrick is an expert on that sort of thing. And then just one very last point, which is just kind of reiterate what Yasha was saying. Yasha has recently been writing very eloquently and forcefully about risks from AI to humans in terms of their behavior being aligned with our interests and things like that. And yeah, adding consciousness or sentience into that mix is potentially extremely dangerous because it could morally constrain that project and also just lead us to act in certain ways that are dangerous to ourselves.
So there's a lot of interesting things to say about the relationship between risks from AI and risks to AI, let's say. And it's very good that people not conflate those two questions. One kind of convergent policy proposal for both of those is that we need to just be extremely careful, slow down, think very hard about what we're doing, have more transparency and reflection about what we're doing. I think that's something that's very important for both of those issues.
YOSHUA
If I may, I'd like to articulate in two sentences why the concern from a safety point of view that Robert just talked about? So if we build machines and we start seeing attributes of consciousness, and then we just complete the picture to give them essentially all of our attributes of consciousness. In other words, they have their own goals. In particular, they have a self preservation goal. And if those machines are smarter than us in sufficient ways to be dangerous to us, then we are in a very risky situation from the point of view of humanity losing control of its future, because there would be something like a new species of entities that may have goals that don't match, that may lead harm to humans. We don't want to do that, obviously.
GRACE
Great.
JEFF
Thank you, Grace.
GRACE
Yeah, I think there's a pragmatic answer that allows for the current high level of uncertainty, and that's if these systems seem conscious to us, then we need to follow that logic through, even if we don't know the truth of the matter. So, yeah, if it's a very human-like system, it's natural that people are going to feel that it's mean. I think you could have a system that is conscious and doesn't have some of the things that you were listing Yoshua, like the, you know, angle or anything like that. So I don't think necessarily if it's a conscious system, it has those things, or if it has those things conscious or anything like that, but certainly we would feel like it is. And the question is, what are the benefits or risks to society if you tell people they have to treat this thing like it's conscious or that they don't?
So if you have something that feels conscious, looks and behaves like a human, and we tell people you can do whatever you want to this thing, it has no moral status, is that going to lead to people? Some people make the argument that you can use that as kind of catharsis, where people could treat the non sentient robot terribly and then they won't do that to humans. Other people think you might start to devalue actual conscious life if you give people things that seem conscious and tell them that they can treat them poorly. So I think that's the pragmatic answer. Given the level of uncertainty. If there was a world where we could be certain that something is conscious, even if it doesn't feel like it is to us or looks like us in any way, I think then the next steps are more complicated because it doesn't just slot into okay, it has moral status the same as a human now. Because a lot of the things that we associate with something having moral status and how we treat the being with moral status, it's about being humane to them, it's about being treating them the way that humans would want to be treated.
And they might have completely different things that need to be done or not done to them to be considered moral to a completely different type of consciousness and intelligence potentially. So even if we can say with certainty that an artificial system is conscious, I don't know if we know very clearly what follows from that. Even if we agree that we're going to treat it as a moral agent, I don't know if we know clearly what follows from that.
JEFF
Yeah, great point. This is a lesson that we have learned often the hard way over the past several decades on the animal minds and the animal ethics side of things. And I think we need to relearn or remember those questions on the AI minds and AI ethics side of things. And I appreciate everybody for articulating that here, that there might be broad similarities between the minds of biological and non biological beings, but a lot of the details might be different. Even if there is some kind of valenced subjective experience, the actual interests and needs are going to be very different. The levels of intelligence and power are potentially going to be very different. It might disrupt. Expectations we have about what it needs to have a moral relationship or a legal or political relationship with someone. So it might be that in some broad, thin sense the concepts extend, but in any kind of more detailed or thicker sense, we have to rethink everything.
PATRICK
Right.
JEFF
Okay, we have about five minutes left. And again, tons of questions and comments. We will not be able to even scratch the surface. But I can ask one more detailed question about your discussion of global workspace theory that came up several times in the comments. Several people asked a question of the forum. You mentioned that large language models do not have the relevant kind of feedback loop at the kind of transistor level. But what about at other levels of explanation? What about, for example, the actual application of the models and how they draw from their own past responses when making predictions? Is there a kind of feedback loop happening there that might be relevant for global workspace theory? There were a few questions of that form, so it'd be great if somebody could address that.
ROBERT
Yeah, I'll say something very quick and then I'm going to pass the baton to Patrick just as a heads up. So a quick clarification. It's not actually about the transistor level, it's about the level of the virtual neurons. It's in that sense that it's feed forward. And then one thing that I haven't actually looked at those questions, but there was an interesting discussion on Twitter that happened where, yeah, you might think that the place you get the feedback loops is the fact that the model will output a word and then look back over the entire string and then output the next word. So you could argue that it's using the whole text output as a kind of global workspace. And if you're interested in the extended mind, you could maybe make an argument that that's a kind of global workspace. That said, I think that there's kind of challenges to that view, which I will punt to Patrick.
PATRICK
Yeah, it just seems to me that any system which interacts with an environment in the sense that its outputs influence the environment and therefore have a knock on effect on its subsequent inputs, is one in which there's a kind of recurrent causal loop connecting the system itself with its environment. And it seems as though if we allow that to be the kind of loop which is described in the global workspace theory, then we're just giving an uncharitable interpretation of the global workspace theory because that's not what they intend. Instead, the thought in the global workspace theory is that there's an internal recurrent loop within the system between the modules and the global workspace. But although Rob has suggested that I'm the best person to answer this question, really the most qualified person here to answer the question by far is Yoshua, because he understands both the global workspace theory and the AI systems much better than Rob or me.
YOSHUA
I agree with your answers. So I have a machine learning interpretation of the bottleneck in the global workspace theory and it allows for forcing particular kinds of dependencies and abstractions to emerge very sparse dependencies that involve like very few variables. It forces that to emerge because of the bottleneck at the internal level. If you were to consider the output words of a transformer as the bottleneck, it doesn't really work for a number of reasons because this is what it's outputting. It's like if you were too forced to say everything that comes into your working memory and also that it could be expressed as words, which is not completely obvious. So it's really a different schema, as you say. The fact that it's an internal bottleneck makes a whole difference and the actions that are taken are not just a copy of that bottleneck, but they might be what is appropriate in the context. You might be lying, for example, or you might realize that your thought has something incoherent and you might want to say something different. You wouldn't have that if you interact with Chat GBT, although people are actually trying to design things like this that are closer to an internal thinking train of thoughts with Transformers and with Chat Chippity in particular to try to emulate some of the properties of the workspace for helping to reason. So there is movement in that direction, but it's not really the same.
PATRICK
Okay, great. So I was just going to say one response which we sometimes get when we say a system like a transformer based large language model or something doesn't meet the criteria, people are often quick to respond by saying well, you could change the system in such and such a way and then maybe it would meet the criteria. And we don't disagree with that at all. We think that there are relatively clear steps which could be taken using existing AI techniques to build systems which would meet more of the indicator properties than the ones which exist at the moment.
YOSHUA
Yeah, I would add that there are other properties that are not really discussed in the global workspace theory which would be missing in my opinion, especially about subjective experience. So the global workspace theory doesn't explain subjective experience, at least not all the properties that are associated with it. I mentioned earlier things like ineffability like the fact that we are conscious of something richer than what we are able to express with words, at least in a limited number of words. And that's something you don't get with Transformers, especially if you put the bottleneck at the output. You might get something like this if you suddenly had a huge hidden layer in the middle somewhere that could play that role. That is possible.
ROBERT
Yeah.
YOSHUA
There are also other properties of conscious thought that are not expressed in Transformers as they are now. For example, attention in Transformers is what we call soft attention, actually something invented in my lab in 2014. And it's not at all like the kind of attention that makes a hard decision, usually somewhat stochastic, about what we're going to attend next, either in the perceptual or in something about our interpretation, our thoughts, our memories. And that is very different in nature from the kind of attention that are currently working well in AI, but doesn't mean that it won't be in future systems. Right, but just saying they're not present currently.
JEFF
Thank you for that exchange. We are a little bit over time now. So Grace, I'm going to give you the last word and then we can wrap up if you still have a comment.
GRACE
Yeah, I just wanted to make a quick point about this idea of there kind of being this external recurrence because you can resample your environment that you impacted. I think if you're looking at the architecture of the model, that is a pretty big difference from there being internal recurrence. But if you take the perspective of, like, a naive neuroscientist who was trying to understand this system and they only had access to the activity of the neurons over time, which is what happens a lot in neuroscience you might think that there is internal recurrence because there would be correlations between activity of neurons over time and that kind of thing, or at least in the information represented in the system over time. And so on some fuzzier more abstract level, maybe it does look like there's recurrence, but we actually know the architecture that generates it. And if you're subscribing to theories of consciousness where the architecture that generates it matters, then it's a different outcome.
JEFF
Okay, great. Thank you very much, Grace. Okay, this is the time now to thank everybody again for taking the time to join us and tell us about your report, answer some of our questions. Thanks also again to everybody in the audience for showing up. We had really amazing attendance and a fantastic conversation happening and apologies for not being able to get at all of the even general topics of the questions, to say nothing of the specific questions. But it really was a great conversation and we will share all the questions and comments and exchanges with the panelists following the talk again. Yes, thank you to everybody. This is obviously the beginning of a much longer conversation about various tests for conscious and sentient AI systems and what follows for their moral, legal, political significance.
So really looking forward to having those conversations. Grateful to everybody for participating in them. Just a note to everybody that you can find a link to the report in the chat. So please do check out the report. You can also find a link to the Mind Ethics and Policy program. You can sign up to our email list for future events. We will be having in early October a talk by Peter Godfrey Smith, a philosopher who is more skeptical about AI consciousness and will explain his skepticism to us. So do sign up for that email list if you want to keep having this conversation with us. And with that, I hope everybody has had a great start to your fall. Great start to your semester. I have to go teach class now, so I will sign off. But thank you again to everybody and have a great rest of the night. And thanks again to our co-sponsors as well. Bioethics and Mind, brain and consciousness. Have a good night, everybody.
ROBERT Thanks so much, Jeff. All right, I think I'm going to go. Yeah. Goodbye, everyone.
Intro
Recent 80k guest and philosopher specializing in AI consciousness Rob Long (@rgb) recently participated in a panel discussion on his paper "Consciousness in Artificial Intelligence: Insights from the Science of Consciousness" (pdf) with co-authors Patrick Butlin, Yoshua Bengio, and Grace Lindsay and moderator Jeff Sebo (@jeffsebo).
You can watch it on Youtube (below), watch/listen as a podcast on Spotify [1], or read the transcript[2] below.
Paper abstract
Youtube description
This event took place on Tuesday September 5, 2023 and was hosted by the NYU Mind, Ethics, and Policy Program.
About the event
This panel discussion featured four authors from the recently released and widely discussed AI consciousness report. This report argues for, and exemplifies, a rigorous and empirically grounded approach to AI consciousness: assessing existing AI systems in detail, in light of the best-supported neuroscientific theories of consciousness. The paper surveys several prominent scientific theories of consciousness, including recurrent processing theory, global workspace theory, higher-order theories, predictive processing, and attention schema theory. From these theories the authors derive "indicator properties" of consciousness, elucidated in computational terms that allow them to assess AI systems for these properties. They use these indicator properties to assess several recent AI systems, and discuss how future systems might implement them. In this event, the authors summarized the report, offered perspectives from philosophy, cognitive science, and computer science, and responded to questions and comments.
About the panelists
Patrick Butlin is a philosopher of mind and cognitive science and a Research Fellow at the Future of Humanity Institute at the University of Oxford. His current research is on consciousness, agency and other mental capacities and attributes in AI.
Robert Long is a Research Affiliate at the Center for AI Safety. He recently completed his PhD in philosophy at New York University, during which he also worked as a Research Fellow at the Future of Humanity Institute. He works on issues related to possible AI consciousness and sentience.
Yoshua Bengio is recognized worldwide as one of the leading experts in artificial intelligence, known for his conceptual and engineering breakthroughs in artificial neural networks and deep learning. He is a Full Professor in the Department of Computer Science and Operations Research at Université de Montréal and the Founder and Scientific Director of Mila – Quebec AI Institute, one of the world’s largest academic institutes in deep learning. He is also the Scientific Director of IVADO. His scientific contributions have earned him numerous awards. He is the 2018 laureate of the A.M. Turing Award, “the Nobel Prize of Computing,” alongside Geoffrey Hinton and Yann LeCun for their important contributions and advances in deep learning. In 2022, he was appointed Knight of the Legion of Honor by France and named co-laureate of Spain’s Princess of Asturias Award for technical and scientific research. Later that year, Professor Bengio became the most cited computer scientist in the world in terms of h-index. He is a Fellow of the Royal Society of London and of Canada, an Officer of the Order of Canada and a Canadian CIFAR AI Chair.
Grace Lindsay is currently an Assistant Professor of Psychology and Data Science at New York University. After a BS in neuroscience from the University of Pittsburgh and a year at the Bernstein Center for Computational Neuroscience in Freiburg, Germany, Grace got her PhD at the Center for Theoretical Neuroscience at Columbia University in the lab of Ken Miller. Following that, she was a Sainsbury Wellcome Centre/Gatsby Computational Neuroscience Unit Research Fellow at University College London.
About the host
The NYU Mind, Ethics, and Policy Program is dedicated to advancing understanding of the consciousness, sentience, sapience, and moral, legal, and political status of nonhumans, including animals and artificial intelligences. We believe that all beings with the capacity to suffer deserve to be treated with respect and compassion, and that our policies and practices should reflect this. Our goal is to promote research and scholarship that will help to shape a more just and humane world for all.
* * As it happens, this program description was co-authored by GPT-3, a large language model! Thank you to our co-sponsors for your generous support of this event:
Transcript
JEFF
Hi everybody. My name is Jeff Sebo. Welcome to this event. Thank you for being here during what I know is a very busy week and day for everybody. This is the first day of class at NYU, and the fact that we have an event today with this many people interested, I think is a testament to how interesting the topic and the report that will be the basis for our conversation are.
So this is an event hosted by the NYU Mind Ethics and Policy Program, and we are also grateful to our co-sponsors, the NYU Center for Bioethics and the NYU Center for Mind, Brain and Consciousness. Thank you for co-sponsoring this event. The NYU Mind Ethics and Policy Program examines the nature and intrinsic value of non-human minds, including biological and nonbiological minds, with special focus on invertebrates and artificial intelligence systems. We are interested in examining what kinds of beings can be conscious and sentient and sapient, and what kinds of moral and legal and political status they should have.
And so we were very interested when this team released this report about a week ago, a report on what leading scientific theories of consciousness have to say about possible AI consciousness. And so we thought it would be useful, especially since the report, as soon as it was released, was generating a lot of discussion. It received coverage in Nature and Science and a lot of conversation on the Internet. We thought it would be valuable to bring some of the authors of the report together. There were 19 authors from different fields and different institutes, and we thought we could bring four of them together, including the two lead authors and representatives of other fields as well, to talk about the report, summarize some of the main points, and then offer their own individual perspectives about some of the implications.
So I will now briefly introduce the speakers, and then we can hear some remarks from them, and then we can open up the discussion and hear questions and comments from you and have a conversation. So here are the speakers.
Patrick Butlin is a philosopher of mind and cognitive science and a Research Fellow at the Future of Humanity Institute at the University of Oxford. His current research is on consciousness, agency and other mental capacities and attributes in AI. Robert Long is a research affiliate at the center for AI Safety. He recently completed his PhD in philosophy at New York University, during which he also worked as a research fellow at the Future of Humanity Institute. He works on issues related to possible AI consciousness and sentience. Yoshua Bengio is recognized worldwide as one of the leading experts in artificial intelligence, known for his conceptual and engineering breakthroughs in artificial neural networks and deep learning. He is a full professor in the Department of Computer Science and Operations Research at the University de Montreal, and the founder and Scientific Director of Mila Quebec AI Institute one of the world's largest academic institutes in deep learning. He is also the winner of many awards and has many other accomplishments that you can easily find online. And finally, Grace Lindsay is currently an assistant professor of Psychology and Data Science at New York University. After a BS in neuroscience from the University of Pittsburgh and a year at the Bernstein Center for Computational Neuroscience in Freiburg, Germany, Grace got her PhD at the center for Theoretical Neuroscience at Columbia University in the lab of Ken Miller. Following that, she was a Sainsbury Wellcome Center Gatsby Computational Neuroscience Unit Research Fellow at University College London.
So thank you all so much for, first of all, writing this very interesting report, and second of all, being here to talk with us about it. I know a lot of people here are interested in hearing from you and discussing this with you. Just a note to everybody that the format and structure of this event will be that we will hear brief remarks from each of the four authors and speakers. Patrick and Rob will speak together about a summary of the report, followed by a philosophical perspective. And then we can hear cognitive science and computer science perspectives from Grayson and Yoshua, respectively. And all along the way, people in the audience can submit questions and comments, objections, whatever you like in the Q/A tab on zoom. And then when we reach the discussion portion of this event, I can read questions, comments, objections that have been entered and upvoted from the Q/A tab. So please go ahead and open that up and add your questions and comments whenever you like throughout the session. Okay, so with that in mind, without further ado, we can start the show. So, Patrick and Rob, whenever you are ready, feel free to share your screen and tell us about your report.
ROBERT
All right, so here we go, Jeff. And if you can, let me know that everything is in working order.
JEFF
Everything is in working order. Yeah, go for it.
ROBERT
Excellent. So let's begin.
PATRICK
Yeah.
ROBERT
First, I just want to say thank you so much to the audience for turning out. We're extremely excited to discuss these issues with you. A big part of why we wrote this report is to discuss these topics more widely and hear from different people. And thanks so much to you, Jeff, and to the NYU Mind Ethics and Policy Program for hosting this. So, my name is Robert Long, and along with Patrick Butlin, I'm one of the lead authors on this report. As Jeff mentioned, there are also 17 esteemed colleagues from various disciplines, and I'm going to briefly say what we did in the report and why we did it and why it matters. And, I mean, Patrick will be covering some of that.
So you've all probably read about the case of Blake Lemoyne, a Google engineer who was fired from Google after he became convinced that an AI system he was interacting with was conscious. You've probably seen the increased deployment of chatbots, including romantic chatbots, that interact with people in very human-like, compelling ways. And you may have seen a tweet from OpenAI's chief scientist Ilya Sutzkiver saying that it may be that today's large neural networks are slightly conscious. So AI consciousness is obviously a topic of increasing public concern and public interest, and it's also a perennially fascinating scientific question. Speaking for myself, as AI consciousness has become a bigger topic recently, I've often been frustrated by some of the characteristics that conversation about it has. I think when people talk about AI consciousness, there's often a lack of specific scientific evidence. There's very little detailed analysis of existing systems. The conversation is often kind of emotive and highly charged and often in dichotomies. Like how could AI systems possibly be conscious? We know that that's absurd. Or on the other hand, people being certain that they definitely are conscious. And what we wanted to provide in this report is kind of a better footing for these kinds of conversations. We want to bring a more nuanced and a more evidence based approach to the report. So in addition to the actual findings that we have, we really want to spur a conversation and spur further research along these lines.
So one aspect I'm going to talk about briefly is also conceptual clarity, something that philosophers love about what exactly we're talking about when we talk about AI consciousness. And before I do that again, just an advertisement for the paper. We put it up on Archive about two weeks ago, I think, and you can find it there.
And there is a list of this really wonderful team that helped us out. We have people from neuroscience, AI and philosophy. So there it is. So what are we talking about in this report? It's called consciousness and AI. Insights from the science of consciousness. It's very important both in this talk and we do this in the paper to pin down what we're talking about. So what I'll be talking about and what we discuss in the paper is what philosophers call phenomenal consciousness. And this means subjective experience. Thomas Nagel famously said that consciousness is what it is like to be an entity. So, for example, if you've tuned in, you're now currently listening to my voice, you're seeing my face and this wall on the screen, and there's a subjective aspect to your experience. There's something that it's like for you to be seeing a white wall or hearing my voice. And what we're asking about in this report is whether AI systems now or in the near future could have subjective experiences in this way, if they could be conscious in this sense.
And it's very important in conversations about this to be clear about what we're not talking about. We're not talking about whether AI systems are intelligent in a certain way. We're certainly not talking about AGI or artificial general intelligence. We're not talking about language, understanding or rationality. And also we're not talking about whether AI systems experience the world in exactly the way that humans do. So Thomas Nagel, when he introduced this phrase, famously asked what it's like to be a bat? It's very plausible that bats might be conscious, that they might have subjective experiences of the world. But if they are, the way that they experience the world is presumably very different than the way we experience the world. And that also comes along with the fact that the way they think about the world is presumably very different and they don't have the full suite of human cognitive capacities. So when we're talking about AI, we're wondering about consciousness in this sense and not necessarily any further assumptions about human like intelligence or experiences.
So how do we go about this in this report? Here's what I think are kind of the main foundations of what we do. First of all, we want the conversation to be empirical. So that means that we're looking at scientific theories of consciousness. So there's a broad field of consciousness science where scientists examine the human brain and animal brains and ask what sort of processes are associated with having conscious experiences. And this is somewhat distinct from philosophical questions about consciousness. We're going to look at these empirical theories and see what they can tell us about potential AI consciousness. Another key aspect is that we take this working assumption of this very broad thesis about the nature of consciousness, which we call computational functionalism. And very broadly, this is saying that what matters for consciousness is the computations that a system performs and not necessarily what it's made of. So this is compatible with consciousness coming from computations done by biological neurons or computations happening in silicon, transistors and chips. Patrick's going to talk a little bit more about exactly why we made this assumption and what it amounts to. And then we try to get precise. We try to derive indicator properties from the science of consciousness that we can use to examine different AI systems, and then we apply these to a broad class of AI systems from several different kinds of research programs, several different kinds of systems. And lastly, we're not aiming for absolute certainty. It's my opinion, and I think most of my co-authors, if you're asking for absolute certainty about AI consciousness, you are going to be disappointed because there still is a lot we don't know about consciousness. That's in spite of the fact that we do know some things, which is what we try to show in this report.
So it's a long report. So I would like to just talk about one specific example of this methodology and how it works in the paper. So one of the most prominent scientific theories of consciousness is called the global Workspace theory. And at a broad level, the global workspace theory says that consciousness is about global broadcast throughout a system. So it has this picture of the mind where there are these specialized independent modules that are responsible for different kinds of information processing. So for example, as you can see in that diagram, one such module might be Perceptual processing. So Vision is responsible for visual information processing. Or maybe there's a module dedicated to making decisions and choosing actions. The global workspace theory says that there's also a global workspace that will select the output from one of these usually independent modules and then broadcast that to all of the modules. This is a way of coordinating all of the processing that they are each doing individually. So consciousness is related to global broadcasts of information throughout a system, and a system having a certain architectural or information processing property that's related to the computational functionalist assumption that we have. And importantly, as we'll see, there needs to be a certain kind of feedback loop between modules and a global workspace. The global workspace taking input from the modules and then also broadcasting it back to them.
So from that we get some indicator properties to say what kind of features does an AI system need to have if it's going to satisfy, at least according to global workspace theory, the indicators that might be associated with consciousness. So one of the top questions you might have is, well, what do we say about large language models? So briefly, large language models, you're probably all familiar with them, these are some of the most prominent systems today. The GPT systems are an example. These are often built with what's called the transformer architecture. And now is certainly not a time for a full explanation of what that amounts to. But one key aspect for our analysis is that the transformer architecture is feed forward and that is to say when the system is taking in an input, say, a certain amount of text and it's going to produce an output, which is the text that it will continue to produce. For example, when you're chatting with it as it's processing that information, the information just goes from the input layer to the output layer kind of continuously. And when we look at this architecture and ask whether the way that information is flowing through, it kind of fits with the picture from global Workspace theory. Our provisional analysis in the paper is that as far as we can tell, there's nothing like a global workspace that is both receiving and broadcasting information to and from anything like individual modules. To put it another way, there don't seem to be the right kind of feedback loops in the transformer architecture for it to be satisfying these particular indicators of consciousness. So our answer in the paper is probably not according to global workspace theory, they do not satisfy these indicators.
So that's the kind of thing we do in the paper. Take a scientific theory, ask more precisely what kind of architectural and informational processing features it says are associated with consciousness. Look at how an AI system works and see how much it matches that. As I said, we do this with a lot of theories. We have, like, five very prominent scientific theories of consciousness that we look at. One thing that's worth remarking on is, why do we look at five different theories? Why not just, for example, go with global work theory? And that's kind of related to our points about nuance and uncertainty. We do think that the science of consciousness is making progress on discovering the processes associated with consciousness, but we think it's premature to commit to one particular theory. There's nothing like a consensus in consciousness science about which one of these stories is the right one for consciousness. So we kind of want to cast a broad net and say, here are a variety of things that might be associated with consciousness according to different theories, and let's look at all of them.
Jeff, I saw that my Internet said it was unstable. Did I skip at all? Are we good?
JEFF
Briefly. But I heard everything you were saying.
PATRICK
Cool.
ROBERT
And yeah, as mentioned, one thing that we think it's important to do is look beyond just large language models. They have understandably received a lot of the attention. They're very compelling. Their capabilities are extremely impressive and most saliently. They sometimes say that they're conscious, so it's understandable that they're one of the main systems that people wonder about. But at least speaking for myself, I think when you look at the way that they're built, they're not necessarily even the best current candidates for consciousness.
Among the AI systems that exist today. For example, many people point out that large language models don't seem like they're agents, or they don't seem embodied. They don't seem to interact continuously with an environment which you might think is important for consciousness. But many systems are plausibly much closer to doing that sort of thing, which is why we look at systems that are from robotics or that navigate virtual environments or that work in different ways than large language models do. And I will now turn things over to my colleague Patrick.
PATRICK
Thanks, Rob. Hi, everyone. So Rob's talked a bit about what we did in this project. I'm going to talk briefly about why we did things in the way that we did, and then I'm also going to comment very briefly on what's next for research in this area.
So next slide, please, Rob. Okay, so on the topic of why we did things the way we did, the first question I want to say something about is why did we make the assumption of computational functionalism? And this is an important question, because computational functionalism is quite a contentious hypothesis. So just to remind you about what this claim is, computational functionalism, as Rob said, is the claim that what matters for consciousness is computations. So according to this view, the reason why humans are conscious is that the right kinds of computations are going on in our brains. And what it would take for an AI system to be conscious is for it to perform the right kind of computations. Now, this is, as I just said, quite a controversial hypothesis. Many researchers in this area doubt it. For example, Anil Seth suggests that consciousness may require living cells, and there's many other researchers who take similar views to that position. So why do we make this assumption? Well, there's a few factors that play into this. One is that we'd think that computational functionalism is a plausible view. That doesn't mean to say that we're confident that it's true. I think there's probably a range of views about computational functionalism among the authors, but as a group, we're certainly not confident about this. It may even be too strong to say that we think it's more likely to be true than not true. But crucially, we think it's plausible enough that it's worth exploring its implications.
Another consideration is that many scientific theories of consciousness are expressed in computational terms. And also, if computational functionalism is true, then there's an important further question about which AI systems perform the kinds of computations which are associated with consciousness. So ultimately, the reason why we make this assumption is that we think it's a productive one in the sense that by working starting from this assumption, we can reduce our overall uncertainty about consciousness in AI to a significant degree.
So we're uncertain about whether computational functionalism is true. We're uncertain about which specific scientific theories of consciousness are true. We don't know which AI systems are conscious. But if we kind of explore this area of the space of possibilities, we think we can make progress, so we think we can reduce our overall uncertainty.
Next slide. Another question about why we did what we did is why did we focus on the internal processes that AI systems go through rather than on their behavior or on their capabilities? Why do we think that thinking about internal processes is a better way of determining whether they're likely to be conscious? Well, this is also an interesting question. And it's interesting because there's this kind of important challenge to the approach that we took, which is that the science of consciousness is relatively immature. There are still lots of competing theories in this science. And also, crucially, these theories are based mostly, although not entirely, on data from humans. So one might very reasonably have doubts about whether these descriptions of what's going on in humans when humans have conscious experiences can be extended to provide a guide to consciousness in non humans. And in particular, that kind of argument has been made in debates about non-human animal consciousness. Our colleague Jonathan Birch, for example, who's a leading researcher in that area, argues that in the case of non-human animals, when you're trying to work out which are conscious, it's more productive with science in its current state. To concentrate on examining the capabilities of non-human animals rather than on trying to work out whether the internal processes in their brains are similar or not to the internal processes in human brains.
But in fact, we think that different methods make sense in these two cases, that is, the case of non-human animals versus the case of artificial intelligence. So, next slide, please, Rob. And the key difference which makes it the case that different methods are appropriate in these two cases is that we know that on the whole, animal brains work in relatively similar ways to human brains. And we can know that because of our shared evolutionary heritage even prior to doing detailed animal neuroscience. On the other hand, there's a relatively wide space of possible designs for AI systems. There are lots of different possible architectures and methods that you can use in AI in principle. So this kind of space of possible internal processes in AI is much wider than the space of processes which we're likely to find in animal brains. Now, there are a couple of different consequences of this .1 is that in the case of AI, it's relatively informative to find similar internal processes going on in a system to the processes which seem to be associated with consciousness in humans. That's just because AIS could relatively easily be different from humans. In the case of animals, finding similar processes is not so informative. But then on the flip side, finding similar capabilities and similar behaviors to those exhibited by humans in AI is less informative than finding similar capabilities in the case of animals.
And that's because it seems that in AI it's possible for very different underlying processes to give rise to superficially similar behaviors and capabilities. So there's a specific version of this problem which clearly arises in AI that exists today, which is that AI systems are sometimes designed specifically to imitate human behavior. So we know that there are AI systems which have superficially, but also in a very compelling way, humanlike capabilities, human-like behaviors which also work in quite unhuman-like ways. For that reason, looking at capabilities looks like, on the whole, an unreliable method, unless it can be very substantially refined in the case of AI.
Next slide. Okay, now turning to the question of what's next for AI consciousness research. Well, I'm sure there are lots of exciting directions which could be pursued at this point. But one which I'm interested in, I think Rob's interested in as well, is understanding what's called valenced conscious experience. So of course, as we're all familiar with, some conscious experiences feel good, like feeling a cool breeze on a hot day. Another conscious experiences feel bad like feeling pain or fear. These are famous conscious experiences. What we say is that the ones that feel good have positive valence and the ones that feel bad have a negative valence. But it seems as though in principle, there can also be neutral conscious experiences which don't have a valence one way or the other. So the question is, could we find indicators for specifically valenced consciousness in AI? That seems to be a question which goes beyond the general one that we were asking in this report.
And we think that this question is important because it seems that pleasure and suffering have special moral significance. Next slide. Now, that brings us to the topic which is one of the major motives for our reports and which we're certainly going to be talking about further today, which is this point that a big part of the reason why thinking about AI consciousness matters. Why working out how to assess whether AI systems are likely to be conscious matters is because our society is soon going to have to decide whether to build systems which could plausibly be conscious. So there's a huge, very difficult philosophical question which is what moral principles should we use to make this momentous choice? I certainly don't know the answer to that question. It's possible we'll explore it a little bit in a few minutes. But we do think that there's a simple step which could be taken now, which is to continue the kinds of work that we've been doing to try to understand what might be good indicators of consciousness in AI. And then in particular, for groups engaged in building AI systems to recognize the possibility that they might build conscious systems, that they might do so even without trying to do just that. And for those groups to develop methods for assessing whether the systems that they're working on are likely to be conscious.
And we certainly strongly recommend that they proceed with great caution if they get a positive result when they're doing that kind of assessment. So thanks very much for listening, everyone. Thanks again to our collaborators and I'm looking forward to the discussion. Thanks.
JEFF
Great. Thank you so much, Patrick and Rob. Now, Grace, do you have any thoughts from your perspective to share?
GRACE
Yeah, I just wanted to talk a little bit about kind of the state of consciousness research in neuroscience to give context to the theories that are discussed in the report and kind of, I guess to situate ourselves maybe in the history of science here. So I should say I'm not directly a consciousness researcher myself, but I do study attention which interweaves with consciousness studies in various ways, but so I don't have a dog in the fight of these different theories, which is possibly a good position to be in to discuss the overall state of things.
But yeah, it is the case that the neuroscientific study of consciousness in terms of being a proper scientific field of research is pretty new. I mean, you could argue neuroscience itself is pretty new in the whole scheme of history of science, but certainly people taking the scientific study of consciousness seriously is definitely new. Because there was a joke that you had to be tenured to study consciousness. And so the idea that now there are actually full labs devoted to this and people are really trying to make it rigorous and thorough and you have these theories and everything that's definitely progress, but it points to the fact that this research is still in its infancy. And so the theories that are laid out here are the major theories that are discussed amongst these researchers. But from my perspective, I don't think there's a sense that these are anywhere close to the final drafts of these theories. And that's important for when you then step through the conclusions. And just the fact that Rob said the report doesn't choose a specific theory to go with because there isn't consensus.
There's these multiple theories that in many ways have conflicts with each other. And so it really is still a young area. And also the way that the scientific study is framed in order to be precise and rigorous. It's usually framed more as studying the neural correlates of consciousness. So not even trying to make a strong claim necessarily about causal connection, but just what do you observe the patterns of neural activity to be when a subject reports a conscious experience? And that's another detail. It's really a lot of times the neural correlates of conscious report. What are people saying that they experienced as their conscious experience? There is a detail of kind of these no report paradigms, but they still ultimately rely on the fact that at some point a human was able to say that they experienced something. And so those are also caveats that bound the scientific study of it and are necessary to make it a scientifically rigorous thing to study. But obviously that's going to from a philosophical perspective, that's going to have implications as well.
And yeah, the other thing that I kind of wanted to talk about in kind of situating this sort of thing so the scientists are going into the brain and it's messy and there's a lot of stuff going on. And the hope is to find the simplified principles that correlate with this conscious report and correlate with people saying something is conscious versus not. And so there the work is to take the big, messy, complex thing and try to come up with the simplest description of what you need in order to get conscious report. When you then actually kind of look at that in isolation, sometimes those indicators as the report kind of turns them into seem really simple.
And I think we have to keep in mind that these theories were not developed for the purposes of trying to see if an AI system is conscious. They're developed in the context of assuming that a human is conscious and looking at their neural activity or even a lot of at least the kind of background knowledge for these theories comes from non human animal work and so they're understanding where they're coming from in that sense. The fact that they're not designed to be directly translated into these sort of easy to identify computational principles that could be in artificial systems, I think is important. I think it's important for this work of trying to take a theory and assess an artificial system. But I also think that there's a lesson for the people, the neuroscientists who study consciousness in this as well. Because as this happens a lot, when you do mathematical modeling, you can be working with a topic area and kind of think you have a mental model of how it works.
And then when you actually go to write it down, you realize some aspects are lacking, maybe or the pieces don't fit together the way that you thought they did. And it's the turning of the kind of mental model and the pile of experimental data and the word models that people use to describe how they think something functions. When you actually have to turn that into an equation or code or actually try to build it, you can kind of see where you might be missing things.
And I think that this is a nice opportunity for the neuroscientists who study consciousness to see their theories in a new light when they're kind of put into these cold, stark indicators and really reflect on if that is summarizing everything that they think is important or that there are things about the brain that are kind of going unspoken that they think are actually really important as well or things about the abilities of humans or animals that are important as well. So, yeah, I think that's important to keep in mind that these theories were not designed to lead to a description that is used for AI. But it's still a very helpful exercise in my mind to go through this and see what they look like in the end, when they're kind of pared down to the simplest form that can be translated into an AI system.
So overall, I think the report has benefits to be able to take this neuroscience literature and bring it to an AI audience and put it in those terms, but also should have benefits for the neuroscience side itself in terms of thinking about how these theories really pan out, how they relate to each other, what they could be missing. If the scientists who created and worked on these theories, if they would agree with the conclusions of the report or even agree that an artificial system that had these properties was conscious. In the end, I think that that's an important thing for those scientists to reflect on. Great.
JEFF
Thank you so much, Grace. And finally, Yoshua, do you have thoughts to share?
YOSHUA
Sure. Maybe I'll start with talking about the computational, functionalism, computational basis of consciousness and subjective experience. We've been using the word consciousness, but really I think it's important to clarify that the word consciousness can be associated to all sorts of things and we're trying to focus on subjective experience, which is the part that may seem very mysterious to many people, including researchers. My personal view, so maybe not the unanimous view of the others, is that physics is computation and many physicists share that view. It's just a bunch of equations that could be implemented in any way you want.
At the end of the day, you get the same changes in the state of the world and your brain is physics too. Now, I think some of the questions about how this could be turned into computations in a computer may have nothing to do with something non material that could possibly be happening in biology. Maybe there is something about the physics like it requires quantum computations that maybe we don't know yet how to do. But actually, if we look at the progress of AI in the last few decades, we're moving forward quite rapidly towards very strong capabilities and we never seem to be requiring any kind of quantum computation in order to get that power, which of course doesn't guarantee that it's continued to be the case.
But that suggests that the level of abstraction that, say, neural networks used in machine learning have is already doing a good job for providing a lot of the explanations and neural correlates of our abilities. So another interesting question that has to do with AI research is why are we conscious in the first place? And the perspective that would come naturally to machine learning researchers or AI researchers is evolution notice with these forms of computations because that gives us an advantage either individually or collectively. There's a social aspect to consciousness as well. And so if there is an advantage, then it's something worth investigating from an AI perspective. It may be something that AI researchers want to put into their AI systems, which is a question I'll come back to that Patrick talked about: do we want to have conscious machines or not?
So one of the things my group has been working on is precisely this. So some of these theories of consciousness, in particular the global workspace theory and attention schema theory and others, really can be interpreted as providing an advantage in terms of our ability to learn and manipulate abstraction. So this is connected to the property of thoughts and attention, selecting very few bits of information that go through working memory at any moment.
And then we sequentially go through a very small number of bits in this way that help us take decisions and organize our understanding of the world at a very abstract level, that compresses information, perceptual information in a way that helps us better understand the world, take better decisions, better model it and so on. Now let me go back to this question of whether AI are conscious or will be in the future. So our report suggests that none of the current AI systems have enough of the characteristics that those theories suggest. First of all, the different theories we chose are not the end of it. So this is a continuously moving field. There, you know, are newspapers coming regularly suggesting other variants often related to existing theories. So we shouldn't take these as the end story of how consciousness works in the brain. And also what the report suggests is actually those properties, those in these theories or maybe other ones that could be related that may come up in the near future. Those attributes are not impossible to put in AI. So it's very plausible that in coming years we would be able to build machines that compute in ways that are at least suggestive of consciousness in the human sense.
And I think this raises a lot of important questions. My take on this is we should not build conscious machines until we know better. There are many reasons for this. In particular, whether we succeed in building machines that are actually conscious or not. If humans perceive those AI systems as conscious, that has a lot of implications that could be extremely destabilizing for society. We associate moral status to other conscious beings, and that's connected with a very strong social contract which works for humans. We have all kinds of characteristics. We have a finite lifetime. We are bounded sort of intellectual capabilities.
And these properties may not apply to AI that can reproduce. There's no limit in how you could copy an AI system over and over. So they might be like, immortal, they might be much smarter than us. All sorts of things that I think would make the current interpretation of consciousness at a moral and social view questionable. I think until we know better, we shouldn't do that. There's another, more pragmatic reason why I think we shouldn't build conscious machines. Because with consciousness also comes a notion of self and even self preservation objectives like agency. This was one of the theories that was described. And if we go on that route, this could be very dangerous from an AI safety point of view. In other words, we might be building machines that have their own goals that are not well aligned with human norms and values in ways that could be extremely dangerous for humanity.
And of course, this is a subject that's intensely debated in the last few months, which makes this report particularly interesting. But this is one point I want to emphasize. Let's see the last point I want to make that's not really in the report, but connected to what we talk about in the report. But comes some recent work coming out of my group just in the last few months suggesting so it's another theory of consciousness that is completely computational. It's related to several existing theories. But what's interesting about this one is that subjective experience with all the attributes that we associate with it, like ineffability and subjectivity and richness and fleetingness, these properties emerge of this model as side effects of the need to perform a particular kind of computation that is important from a learning perspective. But you could obtain potentially the same computations with a different implementation that wouldn't have these side effects. So evolution has sort of converged in this particular way of achieving particular useful computations that may give rise to our sensation of being conscious and having free will and all these things to which we attribute a little bit of sort of magical properties. And we should be careful about that instinct we have about our own perception of being conscious in light of those results from neuroscience and AI. I'll stop here.
GRACE
Great.
JEFF
Thank you so much, Yoshua and everybody, for those remarks. Very interesting. It gives us a lot to talk about. I want to flag both for the panelists and for everybody in the audience, that a very lively discussion is already happening in the Q/A tab. And so I encourage everybody to go check out the Q/A tab and read through some of the questions and comments that have already been entered and feel free to reply directly to each other and have those conversations. We can have one conversation here and another in the Q/A tab. I will jump right into questions that attempt to synthesize and present you with what people are asking about in the Q/A tab. I will not be able to get to everything, but please know, everybody, that we will send all the questions to the panelists after the talk, whether or not we can get to them during the presentation. So some of the questions are descriptive, others of the questions are moral or legal or political in nature. So I can start with a general descriptive question for you. As you noted in the initial presentation and some of your remarks, and as several people have noted in the comments, you focus on a particular perspective about consciousness, according to which consciousness is about computations.
And so you look at scientific theories of consciousness that identify different computations, and then you search for markers related to those theories, and then you look for those markers in particular kinds of AI systems. And as some people note, that does not exhaust the space of theories and perspectives about consciousness that are plausible and popular right now. So on a more permissive end of the spectrum, as one person notes, there are, for example, panpsychist theories of consciousness and other theories that are relatively undemanding that allow for the possibility that even quite simple systems could be conscious. Those theories might imply that lots of systems can be conscious, whether or not they have your markers. And then at the other more restrictive end of the spectrum, you have biological theories according to which, for various reasons, you really do need to be made out of, for example, carbon based cells and neurons in order to realize consciousness.
And according to those theories, a system can hit all of your markers but still be non conscious if that system is made out of silicon based transistors and chips. So I wonder, on a personal level, to the panelists, what kind of credence do you have in these more permissive or more restrictive theories? Do you find them plausible? Do you find them good candidates? And how would you incorporate them into your search for AI consciousness?
ROBERT
I can hop in first. Yeah. I think in people in the report, I'm probably on the higher end in my credence in computational functionalism of some kind, I'm maybe like 70%. But I am very compelled by arguments by people like Anil Seth and Peter Godfrey Smith, philosopher of biology, who has written extensively on consciousness. So I do sometimes wake up in the middle of the night wondering if computational functionalism is off on the wrong track.
And I'll also just take this opportunity to say, one thing we call for in the report is more detailed work investigating the assumption of computational functionalism. We think it's again, sufficiently implausible that it's really very important to explore its implications. But we could also get a lot more clarity on these issues if arguments for and against computational functionalism got hashed out in more detail. And I'll lastly just say I wanted to plug a really nice remark by Anil Seth that I think is really exactly the kind of response we wanted where he said, I disagree with some of the assumptions, and I'm guessing that's computational functionalism, but that's totally fine because I might well be wrong. So we're very excited to see people kind of exploring different parts of the space of possibilities that we could be facing with AI consciousness.
PATRICK
Great.
JEFF
Thank you, Rob. Yoshua?
YOSHUA
Yeah, so my credence on computational fictionalism is 99.99%, maybe because I'm a computer scientist, right? And the whole field of computer science is founded on the idea that computations can be done on any substrate. And there's not been any example of that that exists, as far as I know. It's not just AI. It dates back from Turing and the Turing machine around the Second World War. And it's also connected to, as I said in my little pitch, everything we know from physics. So I don't see how having carbon atoms prevents computations from explaining what's going on. It's just a different kind of computation. It may not be the computations going on at the level of these artificial neurons that you typically find in deep learning. That's very possible. But it's still computation. It's just now computation happening at the atomic level, but it's still computation. What's the level of abstraction that's needed to replicate human intelligence and consciousness? Well, nobody really knows, and that's open. But I think the question of whether you need something that's non computational, because for me, if it's not computational, it's not even physical, so it's not even materialistic. And I don't see how you could buy that unless you believe in some spiritual beings explaining our consciousness.
And I also have a comment on panpsychism. How could I say it feels like it's completely overgeneralizing. The things we know that are conscious are human beings. And because of many similarities, we have some maybe good reasons to think that other animals may be conscious. But everything we know about human consciousness has the kind of properties that we discuss in the report, for example, that are completely disjoint and not applicable to just arbitrary groups of atoms or even single electrons or whatever crazy things that you could come to with these theories. So I'm not saying these theories are false, but they seem so far removed from the biological reality, like what happens when a person is conscious or not conscious, that, for me, they don't rate very high as scientific theories that are supposed to explain what we know about consciousness. They may feel good again, because I think it may make us feel good with our intuitive religious understanding of the world, but in terms of matching what we observe, the correlates of consciousness seem pretty much bringing zero information.
JEFF
All right, great. Thank you. Grace or Patrick, did you have anything you wanted to.
PATRICK
I mean so when you asked about credences in computational functionalism, I guess the thought is that to the extent that we're doubtful about computational functionalism, maybe we're doubtful about the value of this project to kind of reveal whether AI. Systems are likely to be conscious or not and therefore, whether they're likely to have a certain special kind of moral significance or not. And for me, I think how my credences fit together is something like this: if there's such a thing as consciousness, if the concept of consciousness makes sense and is a useful one to apply beyond the human case, then I think most likely it's a computational phenomenon. I think I'd give more credence to the computational view in that situation than to non computational views, because I think the computational views have got more promise in explaining the properties of consciousness. But what keeps me awake at night to go back to what Rob said, is the possibility that the concept of consciousness is somehow confused or that.
YOSHUA
It.
PATRICK
Doesn'T make sense to apply it beyond the human case that it's unproductive for moral theorizing or conceptually confused in some way to ask the question whether AI systems are conscious or not.
YOSHUA
But, Patrick, I don't think these two views are incompatible. So I actually think that consciousness is computational in nature, and that is confusing and kind of not clear that it's meaningful to extend that concept very far from human beings, especially regarding the moral aspects of things. That's what I mean. And the social and moral aspects, I don't think these are incompatible points of views.
PATRICK
Yeah, I think our views are pretty similar.
JEFF
Grace, did you have anything you wanted to add?
GRACE
I mean, in terms of computational functionalism? My gut is that it's largely correct or certainly there will at least need to be a common set of computations and then maybe there also needs to be other stuff. But on the whole, I just feel like we're several paradigm shifts away from really understanding all of this. So it's hard for me to say anything with any confidence or vigor.
JEFF
Yeah, that seems like the answer about which we can be most confident. Okay, great. Thank you everybody. I can now ask a question on the moral, legal, political side, and again, several people have asked questions along those lines as well. So I think, as you yourself said, part of why so many people are so interested in this topic is because we do associate consciousness and then related capacities like sentience, the ability to consciously experience positively and negatively valenced states like pleasure and pain and happiness and suffering. Many people associate that reasonably, in my view, with a certain kind of intrinsic moral and legal and political significance. The idea being that if you have consciousness and or sentience, if there is something it is like to be you, and if it can feel good or bad to be you especially, then you matter for your own sake. And I should consider your interests and needs and vulnerabilities when making decisions that affect you and that might extend to a decision about whether to create you in the first place, as well as a decision about how to treat you if and when I do create you.
And so I would like to ask all of you if you care to respond. First of all, do you associate consciousness or sentience with that kind of intrinsic moral or legal or political significance? Do you think that when a being is conscious or sentient or sufficiently likely to be conscious or sentient by our lights, that we should extend them a certain kind of intrinsic value and consider their potential interest, needs, vulnerabilities when making decisions that affect them? And since we are making these decisions under uncertainty, I also wanted to ask a little bit about how you think about the risks associated with false positives and false negatives, with potential over or under attribution of consciousness and moral status. What are the risks associated with accidentally seeing an object as a subject and what are the risks associated with accidentally seeing a subject as an object? And how do you weigh those when deciding how to calibrate this kind of test and practice? So yeah, do you associate this with moral status and how do you deal with this under?
YOSHUA
Yoshua, I'm not a philosopher, so take that from the angle of a computer scientist. But my interpretation of this question is we are asking the wrong question. It's not whether we should attribute moral status to entities that have particular properties, like being conscious or something like that. Is that that's how we are? Humans are compelled to have empathy and compassion for other types of beings, in particular other human beings, because that's something that evolution put in us, because it helped us to help humanity to succeed and maybe become a dominant species. There are exceptions. You have sociopaths and so on. But for the most part, humans have those innate feelings. And by association, because our brain works by association, we often generalize that to other entities that look like us, mammals in particular, or we also have very strong empathy for babies of other species, right? My partner would not eat meat, but especially if it's coming from the baby of the species, it's not coming from a philosophical kind of argument. It's just an innate thing that we have. I can share that feeling, maybe not as strongly as she does, I think females in general.
So I think we're just asking the wrong question. And when we come to this for machines, I think it would be a huge mistake to build things that would play into our innate response mechanisms towards entities that look like us. So there was this black mirror episode where there are AI clones of a person in a virtual world that we feel for because they are so humanlike, even though it's just a simulation. Right? We can't prevent ourselves from attributing a moral status to those virtual agents because they look human. And I think that's the reality. What we do with that, I think, is then social norms to not break the way our society works with the introduction of entities that don't correspond to something we evolved for but is not going to be true anymore, with machines that could be potentially imitating us in many ways and maybe even have some of the attributes we put in the report. But is that really what society needs? I think that's a big question mark.
PATRICK
Thank you.
JEFF
Very interesting, Rob.
ROBERT
Yeah, so I think my views on this are similar to Yasha's in many ways on the question of what the grounds of moral status are, as philosophers would say. For my own part, I'm quite confident that if an entity is sentient, that is, if it has valenced conscious experiences like pleasure and pain, that that alone is sufficient for us to care about it and show concern for it. Which is why I would be excited for the project that Patrick mentioned. I'm less clear on how to think about entities that are merely conscious, that maybe only have neutral experiences. You could imagine a future large language model maybe that fits more of our indicators that only had experiences of understanding or maybe even some very abstract conscious experience that we can't even comprehend. I would obviously be extremely careful in how I treated that thing, but I'm a little bit less sure how to think about that case.
And then lastly, I just wanted to flag one very characteristic element of how we like to think about this in the report is about uncertainty and managing all of the different cases that might come up. And so I also wanted to flag consciousness itself could be too narrow of a thing to focus on, and we don't want to put all of our focus on that. There are, I think, compelling arguments that even if something is not conscious, if it has desires or goals that it wants to pursue, then that itself is something that should be respected. So I would also love to see equivalent or analogous reports on whether AI systems could have the kind of preferences or desires that might merit consideration.
YOSHUA
It's already the case. I mean, that lots of reinforcement learning agents have valence and have goals. It's like no rocket science here already exists.
ROBERT
So just very quickly on that, I'll just direct you to Patrick's work. Patrick is an expert on that sort of thing. And then just one very last point, which is just kind of reiterate what Yasha was saying. Yasha has recently been writing very eloquently and forcefully about risks from AI to humans in terms of their behavior being aligned with our interests and things like that. And yeah, adding consciousness or sentience into that mix is potentially extremely dangerous because it could morally constrain that project and also just lead us to act in certain ways that are dangerous to ourselves.
So there's a lot of interesting things to say about the relationship between risks from AI and risks to AI, let's say. And it's very good that people not conflate those two questions. One kind of convergent policy proposal for both of those is that we need to just be extremely careful, slow down, think very hard about what we're doing, have more transparency and reflection about what we're doing. I think that's something that's very important for both of those issues.
YOSHUA
If I may, I'd like to articulate in two sentences why the concern from a safety point of view that Robert just talked about? So if we build machines and we start seeing attributes of consciousness, and then we just complete the picture to give them essentially all of our attributes of consciousness. In other words, they have their own goals. In particular, they have a self preservation goal. And if those machines are smarter than us in sufficient ways to be dangerous to us, then we are in a very risky situation from the point of view of humanity losing control of its future, because there would be something like a new species of entities that may have goals that don't match, that may lead harm to humans. We don't want to do that, obviously.
GRACE
Great.
JEFF
Thank you, Grace.
GRACE
Yeah, I think there's a pragmatic answer that allows for the current high level of uncertainty, and that's if these systems seem conscious to us, then we need to follow that logic through, even if we don't know the truth of the matter. So, yeah, if it's a very human-like system, it's natural that people are going to feel that it's mean. I think you could have a system that is conscious and doesn't have some of the things that you were listing Yoshua, like the, you know, angle or anything like that. So I don't think necessarily if it's a conscious system, it has those things, or if it has those things conscious or anything like that, but certainly we would feel like it is. And the question is, what are the benefits or risks to society if you tell people they have to treat this thing like it's conscious or that they don't?
So if you have something that feels conscious, looks and behaves like a human, and we tell people you can do whatever you want to this thing, it has no moral status, is that going to lead to people? Some people make the argument that you can use that as kind of catharsis, where people could treat the non sentient robot terribly and then they won't do that to humans. Other people think you might start to devalue actual conscious life if you give people things that seem conscious and tell them that they can treat them poorly. So I think that's the pragmatic answer. Given the level of uncertainty. If there was a world where we could be certain that something is conscious, even if it doesn't feel like it is to us or looks like us in any way, I think then the next steps are more complicated because it doesn't just slot into okay, it has moral status the same as a human now. Because a lot of the things that we associate with something having moral status and how we treat the being with moral status, it's about being humane to them, it's about being treating them the way that humans would want to be treated.
And they might have completely different things that need to be done or not done to them to be considered moral to a completely different type of consciousness and intelligence potentially. So even if we can say with certainty that an artificial system is conscious, I don't know if we know very clearly what follows from that. Even if we agree that we're going to treat it as a moral agent, I don't know if we know clearly what follows from that.
JEFF
Yeah, great point. This is a lesson that we have learned often the hard way over the past several decades on the animal minds and the animal ethics side of things. And I think we need to relearn or remember those questions on the AI minds and AI ethics side of things. And I appreciate everybody for articulating that here, that there might be broad similarities between the minds of biological and non biological beings, but a lot of the details might be different. Even if there is some kind of valenced subjective experience, the actual interests and needs are going to be very different. The levels of intelligence and power are potentially going to be very different. It might disrupt. Expectations we have about what it needs to have a moral relationship or a legal or political relationship with someone. So it might be that in some broad, thin sense the concepts extend, but in any kind of more detailed or thicker sense, we have to rethink everything.
PATRICK
Right.
JEFF
Okay, we have about five minutes left. And again, tons of questions and comments. We will not be able to even scratch the surface. But I can ask one more detailed question about your discussion of global workspace theory that came up several times in the comments. Several people asked a question of the forum. You mentioned that large language models do not have the relevant kind of feedback loop at the kind of transistor level. But what about at other levels of explanation? What about, for example, the actual application of the models and how they draw from their own past responses when making predictions? Is there a kind of feedback loop happening there that might be relevant for global workspace theory? There were a few questions of that form, so it'd be great if somebody could address that.
ROBERT
Yeah, I'll say something very quick and then I'm going to pass the baton to Patrick just as a heads up. So a quick clarification. It's not actually about the transistor level, it's about the level of the virtual neurons. It's in that sense that it's feed forward. And then one thing that I haven't actually looked at those questions, but there was an interesting discussion on Twitter that happened where, yeah, you might think that the place you get the feedback loops is the fact that the model will output a word and then look back over the entire string and then output the next word. So you could argue that it's using the whole text output as a kind of global workspace. And if you're interested in the extended mind, you could maybe make an argument that that's a kind of global workspace. That said, I think that there's kind of challenges to that view, which I will punt to Patrick.
PATRICK
Yeah, it just seems to me that any system which interacts with an environment in the sense that its outputs influence the environment and therefore have a knock on effect on its subsequent inputs, is one in which there's a kind of recurrent causal loop connecting the system itself with its environment. And it seems as though if we allow that to be the kind of loop which is described in the global workspace theory, then we're just giving an uncharitable interpretation of the global workspace theory because that's not what they intend. Instead, the thought in the global workspace theory is that there's an internal recurrent loop within the system between the modules and the global workspace. But although Rob has suggested that I'm the best person to answer this question, really the most qualified person here to answer the question by far is Yoshua, because he understands both the global workspace theory and the AI systems much better than Rob or me.
YOSHUA
I agree with your answers. So I have a machine learning interpretation of the bottleneck in the global workspace theory and it allows for forcing particular kinds of dependencies and abstractions to emerge very sparse dependencies that involve like very few variables. It forces that to emerge because of the bottleneck at the internal level. If you were to consider the output words of a transformer as the bottleneck, it doesn't really work for a number of reasons because this is what it's outputting. It's like if you were too forced to say everything that comes into your working memory and also that it could be expressed as words, which is not completely obvious. So it's really a different schema, as you say. The fact that it's an internal bottleneck makes a whole difference and the actions that are taken are not just a copy of that bottleneck, but they might be what is appropriate in the context. You might be lying, for example, or you might realize that your thought has something incoherent and you might want to say something different. You wouldn't have that if you interact with Chat GBT, although people are actually trying to design things like this that are closer to an internal thinking train of thoughts with Transformers and with Chat Chippity in particular to try to emulate some of the properties of the workspace for helping to reason. So there is movement in that direction, but it's not really the same.
PATRICK
Okay, great. So I was just going to say one response which we sometimes get when we say a system like a transformer based large language model or something doesn't meet the criteria, people are often quick to respond by saying well, you could change the system in such and such a way and then maybe it would meet the criteria. And we don't disagree with that at all. We think that there are relatively clear steps which could be taken using existing AI techniques to build systems which would meet more of the indicator properties than the ones which exist at the moment.
YOSHUA
Yeah, I would add that there are other properties that are not really discussed in the global workspace theory which would be missing in my opinion, especially about subjective experience. So the global workspace theory doesn't explain subjective experience, at least not all the properties that are associated with it. I mentioned earlier things like ineffability like the fact that we are conscious of something richer than what we are able to express with words, at least in a limited number of words. And that's something you don't get with Transformers, especially if you put the bottleneck at the output. You might get something like this if you suddenly had a huge hidden layer in the middle somewhere that could play that role. That is possible.
ROBERT
Yeah.
YOSHUA
There are also other properties of conscious thought that are not expressed in Transformers as they are now. For example, attention in Transformers is what we call soft attention, actually something invented in my lab in 2014. And it's not at all like the kind of attention that makes a hard decision, usually somewhat stochastic, about what we're going to attend next, either in the perceptual or in something about our interpretation, our thoughts, our memories. And that is very different in nature from the kind of attention that are currently working well in AI, but doesn't mean that it won't be in future systems. Right, but just saying they're not present currently.
JEFF
Thank you for that exchange. We are a little bit over time now. So Grace, I'm going to give you the last word and then we can wrap up if you still have a comment.
GRACE
Yeah, I just wanted to make a quick point about this idea of there kind of being this external recurrence because you can resample your environment that you impacted. I think if you're looking at the architecture of the model, that is a pretty big difference from there being internal recurrence. But if you take the perspective of, like, a naive neuroscientist who was trying to understand this system and they only had access to the activity of the neurons over time, which is what happens a lot in neuroscience you might think that there is internal recurrence because there would be correlations between activity of neurons over time and that kind of thing, or at least in the information represented in the system over time. And so on some fuzzier more abstract level, maybe it does look like there's recurrence, but we actually know the architecture that generates it. And if you're subscribing to theories of consciousness where the architecture that generates it matters, then it's a different outcome.
JEFF
Okay, great. Thank you very much, Grace. Okay, this is the time now to thank everybody again for taking the time to join us and tell us about your report, answer some of our questions. Thanks also again to everybody in the audience for showing up. We had really amazing attendance and a fantastic conversation happening and apologies for not being able to get at all of the even general topics of the questions, to say nothing of the specific questions. But it really was a great conversation and we will share all the questions and comments and exchanges with the panelists following the talk again. Yes, thank you to everybody. This is obviously the beginning of a much longer conversation about various tests for conscious and sentient AI systems and what follows for their moral, legal, political significance.
So really looking forward to having those conversations. Grateful to everybody for participating in them. Just a note to everybody that you can find a link to the report in the chat. So please do check out the report. You can also find a link to the Mind Ethics and Policy program. You can sign up to our email list for future events. We will be having in early October a talk by Peter Godfrey Smith, a philosopher who is more skeptical about AI consciousness and will explain his skepticism to us. So do sign up for that email list if you want to keep having this conversation with us. And with that, I hope everybody has had a great start to your fall. Great start to your semester. I have to go teach class now, so I will sign off. But thank you again to everybody and have a great rest of the night. And thanks again to our co-sponsors as well. Bioethics and Mind, brain and consciousness. Have a good night, everybody.
ROBERT
Thanks so much, Jeff. All right, I think I'm going to go. Yeah. Goodbye, everyone.
GRACE
Bye, Rob. It's just us. Bye.
And either now or soon, other podcasting platforms (need to figure out the Spotify video podcast system).
Made with assembly.ai + a few manual tweaks and paragraph breaks added by me. Surely imperfect. Feel free to comment/message with mistakes.