When AI can mimic human word strings by “meaningless computation”, what does that say about human word strings?

The best generative language models like ChatGPT-4 still astonish us with both what they can and can’t do. And that's for one ostensibly simple reason: an AI still doesn’t “understand” what it’s doing. It can’t even “know” what we want it from it; only descend to whatever reward function it is given. That infamously introduces weird failure modes galore, many of which will likely be with us for a while (along with other issues like endless generation of fake news and propaganda). But this essay isn’t about AI helping, replacing or destroying us. It’s not about what incalculable admixture of fantastic and horrible awaits us in the coming AI wonderland I keep calling the Algorithmocene.

This post is instead about how generative AI creates an urgent need for all of us to stop for something almost no one has time for in our breathless, brave new AI world: a moment of metaphysical philosophy. Specifically, why now is a really good time to question how we think we know anything. What does it mean when we humans claim something is "true" or "real"? I would argue that an AI-powered post-truth world makes a fresh examination from first principles more urgent than ever. 

So here are my pedestrian's two cents.

Let’s start with the basics. Everyone is quick to point out that generative language platforms are just word prediction machines based on statistical models. They binge on ginormous training datasets and learn which words we humans tend to put together in which combinations in which contexts. That LLMs now do this so well contains an important lesson about us humans. Our word strings may be infinitely variable, but they also have a very high degree of predictability. That's a key point to be picked up later.

For now (and isolated - ahem - concerns about AI sentience notwithstanding), let’s simply accept that all computers today mindlessly compute without “understanding” what they’re doing. But given how closely AI can now mimic human language, the obvious questions we should be asking ourselves are: How are our human words principally different? What do we humans mean when we claim to “understand” something? How do we know what is “truth”?

After all, we humans are more than capable of creating our own absurd chains of reasoning untethered to sound conclusions. Anyone who has ever taught math has seen struggling students “mindlessly” compute without understanding what they’re doing. When later explaining their work, they will often point to certain symbol combinations seen earlier and insist: “well, if these symbols were used here, why can’t they be used there?” And who among us has not despaired when listening to others explain their opposing views on a hot-button political topic. We want to throw up our hands and say “They don’t even understand what they’re saying! They’re just mindlessly mixing word strings picked up somewhere!” Finally, part of living a human life is to do and say all sorts of things we later scratch our heads over and wonder “what was I thinking!?”

So while we’re on the basics, let’s remember the most basic fact of life in the universe. We humans, like all living organisms, are essentially statistical prediction machines. Sometimes called the free energy principle by neuroscientist Karl Friston and followers, this basic thermodynamic insight traces its pedigree back to Erwin Schrödinger’s classic 1944 book What is Life? and even earlier to Ludwig Boltzmann. What characterizes life is not matter or energy, which are both abundant and expendable, but persistent ordered structures, which are precious and hard to maintain.

Statistical modeling is the only way living things can maintain their low entropy state against the never ending onslaught of statistical chaos. More broadly, any persistent ordered structure in the universe, whether a random rock in a stream or a random rocker on stage, can be understood as a statistical model of the disorder it constantly bathes in. Rocks in streams, for example, maintain their order by the sheer strength of the chemical bonds in their crystal structure. But they also interact with part of the stream flowing across their surface (forming what’s known in statistics as a “Markov blanket”, though in this case it would be a rather wet blanket). The mechanical stresses on the rock and the weak chemical interactions with different molecules in the stream are passive statistical models of the surrounding water’s pressure, temperature, pH, salinity and so on.

Living things on the other hand have to do more than just passively model. We have to proactively work to expel the entropy around and within us. That’s because everything life does — breathing, eating, thinking, sleeping and all that living stuff — creates new entropy waste that has to be exported to the rest of the universe. (Some propose that a strict maximum of entropy is exported, but that’s a stronger, nontrivial claim.) We do this via complex, orchestrated molecular machinations (aka metabolism) specific to each organism, environment and life history. These in aggregate modify both our internal state and outside environment in such a way that the entropy demons constantly trying to possess us are cast out into the world.

To focus here just on the generic concept more than any specifics or end result, we can think of an organism as being what engineers in control theory call a model predictive controller, or MPC. An MPC is just what it sounds like: a suite of statistical models that predict both an organism’s environment and its internal state, along with a suite of controlling actions to pro-actively maintain the organism’s internal order.

Again, everything an organism does that uses energy—which is everything all the time — boils down to the workings of an MPC. Anything falling outside of this framework starts to accumulate entropy and moves the organism from a state of living towards a state of dying. That’s just the basic math of life. All of which naturally invites, what else, a computer metaphor. We can even think of evolution as a selection for MPC “software” which runs on various “hardware” components in the environment, like water, minerals, sugars, nucleic acids and so on.

Be that as it may, at some point in our history, human MPCs evolved a unique feature. This feature is prompting me at this moment to move my fingers across this keyboard and you the reader to stare into a screen. That in turn links our mental universes across space and time in a dance of word strings created with recursive grammar — word strings that can be hierarchically linked to each other. The ability to link words, a bit like linking those plastic monkeys in a Barrel of Monkeys, is incredibly special. As far as we know, only the human brain makes words that are “sticky” enough to do this. In principle, it gives our universe of word strings the power of universal computation aka Turing Machines. To the extent that this feature presumably appears no where else in nature, one might even say we humans are Earth’s “first AI”.

As a brief aside, we most likely had no choice but to evolve complex language. Since humans left the forests for the open Savannah, we’ve always been uniquely unsuited for surviving alone in the wild. Slow and weak, without fangs, fur, claws or hooves, we only had our smarts and each other. Our only path forward was outsourcing our survival to everyone and everything we could find at hand. That requires a lot of purposeful communication and computation. So for those early pioneers of the genus homo, it was presumably either a) lucky set of mutations that enabled language to double as universal computation or b) extinction.

But however our language came about, the word strings we generate in speech and writing are still just part of our MPC machinery. Sure, human models have much more context and connectivity than current AI models. Our models are not only based on previously stored word strings from other humans. They’re vastly more complicated and influenced by statistical input data mostly invisible to us: accumulated life experience, surrounding culture, what we ate for breakfast, blood pressure, the weather — the list is endless. All the same: the thoughts we have, the words strings we stream, are 100% the output of our statistical prediction models. Just like an AI.

But if the word strings both we and AI spit out are just whatever our statistical models serve up, do they actually mean anything? For that matter, what is even the meaning of our supposed “meanings” if everything is just a string of, if you will, plastic monkeys? In the case of an AI we have no problem saying their machinations don’t inherently “mean” anything. Indeed, it’s the position taken for granted at the beginning of this essay. We’re generally quite fine with saying an AI merely plays a sophisticated “language game” of moving monkeys around.

To point out that we humans do principally the same thing may be an uncomfortable thought at first, but it is certainly not original to this essay. The 20th century philosopher Ludwig Wittgenstein didn’t know about Markov blankets or AI, but his famous description of the human “language game” makes essentially the same point. The overlooked fact about human language, he noted, is this: We never really understand what words “mean” as much as we become skilled at using them. Or rather, what we call “meaning” is only our skills becoming so second-nature that we don’t even notice them. This should be obvious to anyone who has ever opened up a dictionary. Each word is “defined” in terms of other words, which is of course, hopelessly circular. Or rather, it would be hopeless if we didn’t already have the anchor of prior experience in simply using words. Wittgenstein’s insight seems painfully obvious in hindsight, but strangely enough seems to have escaped Western philosophy for over two thousand years (notwithstanding Shakespeare’s Juliet asking “What’s in a name?”).

And in fairness to all the past great thinkers who apparently missed this point, when taken to heart it does cut hard against the grain of deep-seated intuition. That’s certainly not at all what it feels like when we’re thinking, speaking or writing. The meanings behind word strings generated with the richness of complex language, especially if it’s on a topic we care about, seem to glow with their own incandescence. When we tell our loved ones how we feel, argue a political issue or describe a beautiful painting, we’re sure that we’re doing more than playing syntactical monkey games. This nigh self-evident view sits, for example, in the center of John Searle's famous "Chinese Room" - one the most widely cited arguments in the philosophy of mind. In a nutshell, the thought experiment imagines a hapless translator of Chinese who doesn't understand a word of Chinese, but can follow the detailed instructions (i.e. syntax) of a translation book. That such a translator is in principle possible is claimed to show syntax is not semantics.

Many others, notably the likes of Daniel Dennett and Douglas Hofstadter, raise extensive objections to the Chinese Room argument. Here I'll just point out that, in the context of our MPCs, the distinction between syntax and semantics is something of an illusion. The illusion works so well because, for us MPC-beings who type into keyboards and stare at screens, the real syntax and semantics sit much, much deeper than the patterns of pixels in front of us. That is, the vast majority of our human “language game” is scattered invisibly throughout our bodies. It’s in our metabolism and nonstop entropy exporting. The syntax we see is just the tip of an iceberg that has a causal ancestry dating back to our birth. And that’s of course exactly where semantics, our sense of “meaning” lives. When viewed this way in their entirety, syntax and semantics for us MPC-humans are something like two sides of the same coin (apologies for once again mixing metaphors, but that’s what you get with complex language)

As another aside, this gives a rather natural way to view the difference between humans and current AI. An AI lacks “understanding” not because it is merely playing a language game (and certainly not because it runs on a silicon substrate), but because it doesn’t engage in model predictive controlling. Rather, its statistical models are more like the passive rock in the stream example earlier. It mirrors its environment without actively engaging it like a living MPC. The language game of a human is different. The same water flowing over a human body has to be pro-actively engaged in some way that costs energy and produces an entropy waste problem. Dealing with all of that is what makes water actually “mean” something to us that it doesn’t mean to an AI.

So finally, what does all this have to do with metaphysics; the meaning of truth, reality and all that? Everything, I would say.

In comparing ourselves to a generative AI that convincingly mimics our word strings, we have to face something important about ourselves. Our own sense of meaning is so visceral, hidden and integral to our being that it’s easy to forget they’re all just human meanings. When we throw around words like “truth” and “reality”, the MPC machinations at the molecular level that go into their syntax and semantics are almost entirely invisible. They simply feel so second-nature and unproblematic that we’re sure their meaning must transcend us. As the brilliantly affable Neil deGrasse Tyson likes to say “The great thing about science is that it’s true whether you believe it or not.” And another brilliantly affable fellow, Albert Einstein, was famously sure that the Moon is there even when nobody looks. (1)

Why are objections to Wittgenstein's straightforward point about human language often so visceral? My guess is that, as noted earlier, human word strings are both infinitely variable and yet highly predictable. That puts them in the sweet spot for the brain’s dopamine reward system. In particular, the dopamine reward often associated with grand world-splainin’ words like “truth” and “reality” lends them more or less magic properties in our minds. We instinctively believe we need truth and reality but they don’t need us. Or so we all believed until Bell's Inequality and a slew of famous EPR experiments made many of us pause with a big “… uh… hang on a minute…”

But we didn’t need quantum theory to teach us the naïveté of our instincts. Before Wittgenstein or any modern physics, the deep conceptual conundrums at the heart of any recursive language were already apparent. Many famous mind-bending examples from Russel’s barber paradox all the way to Gödel’s Theorem teach us this. Frankly, I think they beat us over the head with a frying pan. But we humans are nothing if not hard headed, and so we go on believing.

Just to be clear, the frying-pan-resistant-belief I'm panning here is NOT: “we humans in the 21st century see truth and reality as they really are” (that would be ridiculously naive, not to mention awkwardly convenient). Rather, the belief that the best discerning human minds tend to stubbornly cling to is basically the motto of the X-Files: "It doesn't matter if we limited humans never see ultimate truth: The Truth is Out There!"

So powerful are our instincts in this regard that we’d rather tie ourselves into cognitive pretzels with metaphysical fantasy stories like “natural kinds”, “transcendental realism” and “Platonic Realms” rather than give them up.

By now I hope the hopeless incoherence of all these fantasies feels fairly obvious. In particular, they’re incoherent on their own terms. Once you admit that what you mean by “truth” and “reality” is a construction of the mind's MPC modeling efforts, then you also admit your stories about truth and reality are mind constructions. And any meta-story you want to tell about your stories about them is a mind construction. It is then empty, meaningless and useless to try to talk of any of this existing outside the mind when every single word you use, like “truth”, “reality”, “exist”, “outside” and “mind” are themselves all mind constructions. We can’t jump outside the narratives of our human MPC any more than we can jump over our own shadows. The very notion is as gibberish as the square root of Tuesday.

Or put another way. We humans can’t look outside the domain of the human language game. But to the extent we pretend to look outside, then we’d have to admit that everything we do, think and say is literally as meaningless as a generative AI spewing out its word salad of 0s and 1s.

Of course that means everything I’ve written here is also just a story of my MPC. So to briefly recap my story for clarity: its narrative arc started with a particular scientific model of reality: the view that everything obeys the second law of thermodynamics. From there it followed that our thoughts and beliefs are at the end of a long chain of statistical prediction modeling that’s necessary in the moment to export entropy. The story then pondered “what does it all mean?” only to conclude that whatever we say it means, it’s our meaning. Looking outside to Platonic woo-woo land for transcendent meaning is just drinking more Kool-Aid flavored dopamine. That this entire story is also just a creation of my own statistical modeling is not a contradiction; it’s exactly the point.

As a final thought, and leaning into a future essay, it should be emphasized that none of this is meant as a post-modernist hell of “reality is all relative” or “any truth is as good as another”. No, not even close. In fact I will argue in said later essay for just the opposite. In the meantime, I don't think a rational person's cage should really be rattled by the view here. Dropping our metaphysical fantasies is certainly not a reason to throw up our hands and panic. Life doesn’t fall apart without a magic metaphysical “reality” anymore than it falls apart without sky gods or the illusion of free will. Biologist Richard Dawkins once famously said atheists “go one god further” in rejecting the magical thinking of religion. If the same atheists have the courage of their convictions, I believe they can also go one epistemological god further and reject the magical thinking of transcendent “reality”. Within the wonderful and beautiful scientific worldview that our human brains are capable of laying out, we can be sure of this: evolution made sure our scientific stories are generally stable and useful. And it gave us the tools to constantly improve on them. 

And if there is something human civilization urgently needs to improve on, it’s the stories we tell. Dropping empty metaphysical fantasies about truth and reality is, I think, at least one small step along that way.

Footnotes:

(1) This example of Einstein objecting to quantum mechanics is, for me at least, a reminder of how our subconscious skill at the language game can fool us. And it's somewhat ironic that he of all people fell for this trap. After all, his relativity theory is based precisely on a "Wittgenstein-like" approach to the language of physics. The Einstein that created relativity understood that words like space, time, mass and energy had no magic, transcendent meaning. They are meaningful only in their operational definitions, i.e. the way they are used. 

(Note: this essay is a revised version of one of my postings of my Algorithmocene page on Medium)