First there was the world of inanimate matter, the First Arena. Life arose from that, the Second Arena. Only 100s of thousands of years ago humans evolved from higher primates and the Third Arena, human culture, appeared. We are now on the threshold of the Fourth Arena. How do we characterize it?
Note: These thoughts are off the top of my head [thinking-out-loud]. It was all I could do to get them out. That’s enough for now. Refinement will have to wait.
Let us start with these deep learning models trained on large bodies of data. ChatGPT has such a model at its functional core. They have been termed Foundation Models because they “can be adapted to a wide range of downstream tasks.” I have come to think of each such models as repositories of digital wilderness.
What do we do with the wilderness? We explore it, map it, in time settle it and develop it. We cultivate and domesticate it. AI safety researchers call that alignment. The millions of people who have been using ChatGPT are part of that process. We may not think of ourselves in that way, but that, in part, is how OpenAI thinks of us. Even as we pursue our own ends while interacting with ChatGPT, OpenAI is collecting those sessions and will be using them to fine-tune the system, to align it.
Yes, it would be nice to have a system “pre-aligned” before it is released to end users. But I don’t think that’s how things are going to work out. The process by which these engines are trained on large amounts of data is powerful, but it is also messy. While some alignment can be achieved by a small in-house team tweaking the system, ultimately it will have to interact with a much larger body of users. Why not think of alignment as a process concomitant with use?
What are the business implications of a technology where the end users will inevitably playing an important role in fine-tuning and evolving the technology they are using?
Robots and the physical world
One problem that has come up is that of embodiment. These Foundation mMdels, these tracts of digital wilderness, have no access to the real world. Language models are trained on texts, visual models are trained on images. Neither have the capacity to interact with the world.
In an essay he wrote shortly after he took a position as VP of AI for Halodi Robotics, Eric Jang wrote:
Reality has a surprising amount of detail, and I believe that embodied humanoids can be used to index that all that untapped detail into data. Just as web crawlers index the world of bits, humanoid robots will index the world of atoms. If embodiment does end up being a bottleneck for Foundation Models to realize their potential, then humanoid robot companies will stand to win everything.
Those robots will thus be creating more digital wilderness. But also helping to develop it, to align it.
Symbolic Systems and alignment
One issue that has come up is that of the role of the Old School technologies of symbolic systems. Are they obsolete or, on the contrary, will the remain important? While there is wide-spread sentiment that they are obsolete, and that the technology can achieve its fullest flowering simply by scaling up machine learning, I disagree, nor am I alone.
The question is how we are going to integrate symbolic technology with machine learning. One problem is that, traditionally, such systems have been developed “by hand.” The code must be crafted by people who are experts in the application domain. This is costly and time-consuming.
Off-hand I don’t what can be done. Yes, work is being done with hybrid systems where symbolic technology is grafted on to and underlying base of machine learning. I would like to see symbolic capabilities emerge from the underlying artificial neural net technology – perhaps analogous to the way in which language develops in humans, something I’ve discussed in my recent working paper, Relational Nets Over Attractors, A Primer: Part 1, Design for a Mind, Version 2.
I note, though, that I regard the development of symbolic technology as a stream in the overall process of aligning software grounded in vast plots of digital wilderness. It will thus be a gradual process distributed widely thoughout the community of users.
AI as platform, Adept
Two years ago venture capitalist Mark Andreesen talked of AI as a platform, not a feature. He said:
I think that the deeper answer is that there’s an underlying question that I think is an even bigger question about AI that reflects directly on this, which is: Is AI a feature or an architecture? Is AI a feature, we see this with pitches we get now. We get the pitch and it’s like here are the five things my product does, right, points one two three four five and the, oh yeah, number six is AI, right? It’s always number six because it’s the bullet that was added after they created the rest of the deck. Everything is gonna’ kind of have AI sprinkled on it. That’s possible.
We are more believers in a scenario where AI is a platform, an architecture. In the same sense that the mainframe was an architecture or the minicomputer is an architecture, the PC, the internet, the cloud has an architecture. We think AI is the next one of those. And if that’s the case, when there’s an architecture shift in our business, everything above the architecture gets rebuilt from scratch. Because the fundamental assumptions about what you’re building change. You’re no longer building a website or you’re no longer building a mobile app, you’re no longer building any of those things. You’re building an AI engine that is, in the ideal case, just giving you're the answer to whatever the question is. And if that’s the case then basically all applications will change. Along with that all infrastructure will change. Basically the entire industry will turn over again the same way it did with the internet, and the same way it did with mobile and cloud and so if that’s the case then it’s going to be an absolutely explosive....
In practice, we’re building a general system that helps people get things done in front of their computer: a universal collaborator for every knowledge worker. Think of it as an overlay within your computer that works hand-in-hand with you, using the same tools that you do. We all have parts of our job that energize us more than others – with Adept, you’ll be able to focus on the work you most enjoy and ask our model to take on other tasks. For example, you could ask our model to “generate our monthly compliance report” or “draw stairs between these two points in this blueprint” – all using existing software like Airtable, Photoshop, an ATS, Tableau, Twilio to get the job done together. We expect the collaborator to be a good student and highly coachable, becoming more helpful and aligned with every human interaction.
This product vision excites us not only because of how immediately useful it could be to everyone who works in front of a computer, but because we believe this is actually the most practical and safest path to general intelligence. Unlike giant models that generate language or make decisions on their own, ours are much narrower in scope–we’re an interface to existing software tools, making it easier to mitigate issues with bias. And critical to our company is how our product can be a vehicle to learn people’s preferences and integrate human feedback every step of the way.
Judging from what they’ve written, they’re not quite where Andreesen’s conception is, but they’re moving in that direction. All you have to do is put AI at the heart of the application, rather than treating it as an add-on, and you’re there.
Robot Toys, an exercise for the reader
Do some research on the state of robotic toys and companions. Start with, say, the 1990s Tamagotchi, which wasn’t a robot at all, but a little gadget one had to care for. Do a web search on “robot toys.” Think about the Tamagotchi, think about children and dolls/action figures, and think about those robot toys. Take what you find and insert it hear. Perhaps ChatGPT can help you.
Robots and Humans, living and working together
In 1995 Neil Stephenson, published The Diamond Age: Or, A Young Lady's Illustrated Primer. It centers on a young girl, Nell, who is given a precious book, A Young Lady's Illustrated Primer, which she has with her as she grows up. It serves her as both a tutor and a companion.
That’s where we’re headed. But I have no idea when we’ll get there, a century, two, three? Who knows.
At a very young age each child will be given such a book, or perhaps a robot – why not both? – which will stay with them for the rest of their life, functioning variously as a companion, tutor, and workmate, their personal robot companion (PRC).
At the moment time of their third birthday. By this time the child has plenty of experience getting around physically and is getting better with speaking. They will have seen other kids with their PRCs and no doubt have interacted with them. They’ll know that, when the time comes, they’ll be getting one too.
The robot will have to be of an appropriate size, a book too for that matter. I will have to be replaced at the appropriate age. That will require a ritual, as did the original gifting of the PRC, and/or book.
We will evolve toward a society where robots, AIs, and people will be constantly interacting with one another. There where will communities of mixed groups, others of only one kind of being. I have no idea what that will be like. But it does sound like the Fourth Arena will be deeply entrenched in that world.
Beyond AGI, super-intelligence, and the Singularity
What about AGI (artificial general intelligence)? What about it? Originally it was simply AI, and it was right about the corner. But, alas, it really wasn’t. “AGI” was coined in the first decade of the millennium to revivify those old hopes.
While I followed work in AI back in the day, I did it out of a sense of professional obligation. I didn’t think it was going anywhere. The very different AI that we’ve got now IS going somewhere. It’s time to ditch those old dreams in favor, both of current reality and what we can build in the near-term future, and of new dreams.
I feel much the same about the idea of super-intelligence. Oh, I know how the word was used. I could follow the conversations. But I don’t think there is anything there, no substantial conceptual development.
I feel the same about the idea of a Technological Singularity. AGI recursively rewriting its own code until FOOM! super-intelligence? Nah. Not going to happen. Someone termed it The Rapture for Nerds. That sounds about right.
No, something else is afoot. We’re living on the cusp of the Fourth Arena. What could be grander?
First there was the world of inanimate matter, the First Arena. Life arose from that, the Second Arena. Only 100s of thousands of years ago humans evolved from higher primates and the Third Arena, human culture, appeared. We are now on the threshold of the Fourth Arena. How do we characterize it?
Note: These thoughts are off the top of my head [thinking-out-loud]. It was all I could do to get them out. That’s enough for now. Refinement will have to wait.
* * * * *
OpenAI released GPT-3 in the summer of 2020. It was clear to me that, yes, it has that potential. I registered my response, initially in a comment over at Marginal Revolution, and then on New Savanna, First thoughts on the implications of GPT-3 [here be dragons, we're swimming and flying with them]. Having explored ChatGPT for the past two weeks, that potential emerges before me with even greater clarity. Of course, we could blow it, nothing is guaranteed. But still...
One must wonder, dream, and hope.
Foundation Models as digital wilderness
Let us start with these deep learning models trained on large bodies of data. ChatGPT has such a model at its functional core. They have been termed Foundation Models because they “can be adapted to a wide range of downstream tasks.” I have come to think of each such models as repositories of digital wilderness.
What do we do with the wilderness? We explore it, map it, in time settle it and develop it. We cultivate and domesticate it. AI safety researchers call that alignment. The millions of people who have been using ChatGPT are part of that process. We may not think of ourselves in that way, but that, in part, is how OpenAI thinks of us. Even as we pursue our own ends while interacting with ChatGPT, OpenAI is collecting those sessions and will be using them to fine-tune the system, to align it.
Yes, it would be nice to have a system “pre-aligned” before it is released to end users. But I don’t think that’s how things are going to work out. The process by which these engines are trained on large amounts of data is powerful, but it is also messy. While some alignment can be achieved by a small in-house team tweaking the system, ultimately it will have to interact with a much larger body of users. Why not think of alignment as a process concomitant with use?
What are the business implications of a technology where the end users will inevitably playing an important role in fine-tuning and evolving the technology they are using?
Robots and the physical world
One problem that has come up is that of embodiment. These Foundation mMdels, these tracts of digital wilderness, have no access to the real world. Language models are trained on texts, visual models are trained on images. Neither have the capacity to interact with the world.
In an essay he wrote shortly after he took a position as VP of AI for Halodi Robotics, Eric Jang wrote:
Those robots will thus be creating more digital wilderness. But also helping to develop it, to align it.
Symbolic Systems and alignment
One issue that has come up is that of the role of the Old School technologies of symbolic systems. Are they obsolete or, on the contrary, will the remain important? While there is wide-spread sentiment that they are obsolete, and that the technology can achieve its fullest flowering simply by scaling up machine learning, I disagree, nor am I alone.
The question is how we are going to integrate symbolic technology with machine learning. One problem is that, traditionally, such systems have been developed “by hand.” The code must be crafted by people who are experts in the application domain. This is costly and time-consuming.
Off-hand I don’t what can be done. Yes, work is being done with hybrid systems where symbolic technology is grafted on to and underlying base of machine learning. I would like to see symbolic capabilities emerge from the underlying artificial neural net technology – perhaps analogous to the way in which language develops in humans, something I’ve discussed in my recent working paper, Relational Nets Over Attractors, A Primer: Part 1, Design for a Mind, Version 2.
I note, though, that I regard the development of symbolic technology as a stream in the overall process of aligning software grounded in vast plots of digital wilderness. It will thus be a gradual process distributed widely thoughout the community of users.
AI as platform, Adept
Two years ago venture capitalist Mark Andreesen talked of AI as a platform, not a feature. He said:
I agree, and believe that Adept: Useful General Intelligence, is moving in that direction. From their blog:
Judging from what they’ve written, they’re not quite where Andreesen’s conception is, but they’re moving in that direction. All you have to do is put AI at the heart of the application, rather than treating it as an add-on, and you’re there.
Robot Toys, an exercise for the reader
Do some research on the state of robotic toys and companions. Start with, say, the 1990s Tamagotchi, which wasn’t a robot at all, but a little gadget one had to care for. Do a web search on “robot toys.” Think about the Tamagotchi, think about children and dolls/action figures, and think about those robot toys. Take what you find and insert it hear. Perhaps ChatGPT can help you.
Robots and Humans, living and working together
In 1995 Neil Stephenson, published The Diamond Age: Or, A Young Lady's Illustrated Primer. It centers on a young girl, Nell, who is given a precious book, A Young Lady's Illustrated Primer, which she has with her as she grows up. It serves her as both a tutor and a companion.
That’s where we’re headed. But I have no idea when we’ll get there, a century, two, three? Who knows.
At a very young age each child will be given such a book, or perhaps a robot – why not both? – which will stay with them for the rest of their life, functioning variously as a companion, tutor, and workmate, their personal robot companion (PRC).
At the moment time of their third birthday. By this time the child has plenty of experience getting around physically and is getting better with speaking. They will have seen other kids with their PRCs and no doubt have interacted with them. They’ll know that, when the time comes, they’ll be getting one too.
The robot will have to be of an appropriate size, a book too for that matter. I will have to be replaced at the appropriate age. That will require a ritual, as did the original gifting of the PRC, and/or book.
We will evolve toward a society where robots, AIs, and people will be constantly interacting with one another. There where will communities of mixed groups, others of only one kind of being. I have no idea what that will be like. But it does sound like the Fourth Arena will be deeply entrenched in that world.
Beyond AGI, super-intelligence, and the Singularity
What about AGI (artificial general intelligence)? What about it? Originally it was simply AI, and it was right about the corner. But, alas, it really wasn’t. “AGI” was coined in the first decade of the millennium to revivify those old hopes.
While I followed work in AI back in the day, I did it out of a sense of professional obligation. I didn’t think it was going anywhere. The very different AI that we’ve got now IS going somewhere. It’s time to ditch those old dreams in favor, both of current reality and what we can build in the near-term future, and of new dreams.
I feel much the same about the idea of super-intelligence. Oh, I know how the word was used. I could follow the conversations. But I don’t think there is anything there, no substantial conceptual development.
I feel the same about the idea of a Technological Singularity. AGI recursively rewriting its own code until FOOM! super-intelligence? Nah. Not going to happen. Someone termed it The Rapture for Nerds. That sounds about right.
No, something else is afoot. We’re living on the cusp of the Fourth Arena. What could be grander?