I’ve updated quite hard against computational functionalism (CF) recently (as an explanation for phenomenal consciousness), from ~80% to ~30%. Of course it’s more complicated than that, since there are different ways to interpret CF and having credences on theories of consciousness can be hella slippery.

So far in this sequence, I’ve scrutinised a couple of concrete claims that computational functionalists might make, which I called theoretical and practical CF. In this post, I want to address CF more generally.

Like most rationalists I know, I used to basically assume some kind of CF when thinking about phenomenal consciousness. I found a lot of the arguments against functionalism, like Searle’s Chinese room, unconvincing. They just further entrenched my functionalismness. But as I came across and tried to explain away more and more criticisms of CF, I started to wonder, why did I start believing in it in the first place? So I decided to trace back my intellectual steps by properly scrutinising the arguments for CF.

In this post, I’ll first give a summary of the problems I have with CF, then summarise the main arguments for CF and why I find them sus, and finally briefly discuss what my view means for AI consciousness and mind uploading.

My assumptions

• I assume realism about phenomenal consciousness: Given some physical process, there is an objective fact of the matter whether or not that process is having a phenomenal experience, and what that phenomenal experience is. I am in camp #2 of Rafael’s two camps.
• I assume a materialist position: that there exists a correct theory of phenomenal consciousness that specifies a map between the third-person properties of a physical system and whether or not it has phenomenal consciousness (and if so, the nature of that phenomenal experience).
• I assume that phenomenal consciousness is a sub-component of the mind.

Defining computational functionalism

Here are some definitions of CF I found:

• Computational functionalism: the mind is the software of the brain. (Piccinini 2010)
• [Putnam] proposes that mental activity implements a probabilistic automaton and that particular mental states are machine states of the automaton’s central processor. (SEP)
• Computational functionalism is the view that mental states and events – pains, beliefs, desires, thoughts and so forth – are computational states of the brain, and so are defined in terms of “computational parameters plus relations to biologically characterized inputs and outputs”  (Shagir 2005)

Here’s my version:

Computational functionalism: the activity of the mind is the execution of a program.

I’m most interested in CF as an explanation for phenomenal consciousness. Insofar as phenomenal consciousness is a real thing, and that phenomenal consciousness can be considered an element of the mind,^[1] CF then says about phenomenal consciousness:

Computational functionalism (applied to phenomenal consciousness): phenomenal consciousness is the execution of a program.

What I want from a theory of phenomenal consciousness is for it to tell me what third-person properties to look for in a system to decide if, and if so what, phenomenal consciousness is present.

Computational functionalism (as a classifier for phenomenal consciousness): The right program running on some system is necessary and sufficient for the presence of phenomenal consciousness in that system.^[2] If phenomenal consciousness is present, all aspects of the corresponding experience is specified by the program.

The second sentence might be contentious, but this follows from the last definition. If this sentence isn’t true, then you can’t say that conscious experience is that program, because the experience has properties that the program does not. If the program does not fully specify the experience, then the best we can say is that the program is but one component of the experience, a weaker statement.

When I use the phrase computational functionalism (or CF) below, I’m referring to the “classifier for phenomenal consciousness” version I’ve defined above.

Arguments against computational functionalism so far

Previously in the sequence, I defined and argued against two things computational functionalists tend to say:

1. Theoretical CF: A simulation of a human brain on a computer, with physics perfectly simulated down to the atomic level, would cause the same conscious experience as that brain.

2. Practical CF: A simulation of a human brain on a classical computer, capturing the dynamics of the brain on some coarse-grained level of abstraction, that can run on a computer small and light enough to fit on the surface of Earth, with the simulation running at the same speed as base reality^[3], would cause the same conscious experience as that brain.

I argued against practical CF here and theoretical CF here. These two claims are two potential ways to cash out the CF classifier I defined above. Practical CF says that a particular conscious experience in a human brain is identical to the execution of a program that is simple enough to run on a classical computer on Earth, which requires it to be implemented on a level of abstraction of the brain higher than biophysics. Theoretical CF says the program that creates the experience in the brain is (at least a submodule of) the “program” that governs all physical degrees of freedom in the brain.^[4]

These two claims both have the strong subclaim that the simulation must have the same conscious experience as the brain they are simulating. We could weaken the claims to instead say “the simulation would have a similar conscious experience”, or even just “it would have a conscious experience at all”. These weaker claims are much less sensitive to my arguments against theoretical & practical CF.

But as I said above, if the conscious experience is different, that tells me that the experience cannot be fully specified by the program being run, and therefore the experience cannot be fully explained by that program. If we loosen the requirement of the same experience happening, this signals that the experience is also sensitive to other details like hardware, which constitutes a weaker statement than my face-value reading of the CF classifier.

I hold that a theory of phenomenal consciousness should probably have some grounding in observations of the brain, since that’s the one datapoint we have. So if we look at the brain, does it look like something implementing a program? In the practical CF post, I argue that the answer is no, by calling into question the presence of a “software level of abstraction” (cf. Marr’s levels) below behavior and above biophysics.

In the theoretical CF post, I give a more abstract argument against the CF classifier. I argue that computation is fuzzy, it’s a property of our map of a system rather than the territory. In contrast, given my realist assumptions above, phenomenal consciousness is not a fuzzy property of a map, it is the territory. So consciousness cannot be computation.

When I first realized these problems, it updated me only a little bit away from CF. I still said to myself “well, all this consciousness stuff is contentious and confusing. There are these arguments against CF I find convincing, but there are also good arguments in favor of it, so I don’t know whether to stop believing in it or not.” But then I actually scrutinized the arguments in favor CF, and realized I don’t find them very convincing. Below I’ll give a review of the main ones, and my problems with them.

Arguments in favor of computational functionalism

Please shout if you think I’ve missed an important one!

We can reproduce human capabilities on computers, why not consciousness?

AI is achieving more and more things that we used to think were exclusively human. First they came for chess. Then they came for Go, visual recognition, art, natural language. At each step the naysayers dragged their heels, saying “But it’ll never be able to do thing $x$ that humans can do!”.

Are the non-functionalists just making the same mistake by claiming that consciousness will never be achieved on computers?

First of all, I don’t think computational functionalism is strictly required for consciousness on computers.^[5] But sure, computational functionalism feels closely related to the claim that AI will become conscious, so let’s address it here.

This argument assumes that phenomenal consciousness is in the same reference class as the set of previously-uniquely-human capabilities that AI has achieved. But is it? Is phenomenal consciousness a cognitive capability?

Hang on a minute. Saying that phenomenal consciousness is a cognitive capability sounds a bit… functionalist. Yes, if consciousness is a nothing but particular function that the brain does, and AI is developing the ability to reproduce more and more functions of the human brain, then it seems reasonable to expect the consciousness function to eventually arrive in AI.

But if you don’t accept that phenomenal consciousness is a function, then it’s not in the same reference class as natural language etc. Then the emergence of those capabilities in AI does not tell us about the emergence of consciousness.

So this argument is circular. Consciousness is a function, AI can do human functions, so consciousness will appear in AI. Take away the functionalist assumption, and the argument breaks down.

The computational lens helps explain the mind

Functionalism, and computational functionalism in particular, was also motivated by the rapid progress of computer science and early forms of AI in the second half of the 20th Century (Boden 2008; Dupuy 2009). These gains helped embed the metaphor of the brain as a computer, normalise the language of information processing as describing what brains do, and arguably galvanise the discipline of cognitive science. But metaphors are in the end just metaphors (Cobb 2020). The fact that mental processes can sometimes be usefully described in terms of computation is not a sufficient basis to conclude that the brain actually computes, or that consciousness is a form of computation.

(Seth 2024)

Human brains are the main (only?) datapoint from which we can induct a theory of phenomenal consciousness. So when asking what properties are required for phenomenal consciousness, we can investigate what properties of the human brain are necessary for the creation of a mind.

The human brain seems to give rise to the human mind at least a bit like how a computer gives rise to the execution of programs. Modelling the brain as a computer has proven to have a lot of explanatory power: via many algorithmic models including models of visual processing, memory, attention, decision-making, perception & learning, and motor control.

These algorithms are useful maps of the brain and mind. But is computation also the territory? Is the mind a program? Such a program would need to exist as a high-level abstraction of the brain that is causally closed and fully encodes the mind.

In my previous post assessing practical CF, I explored whether or not such an abstraction exists. I concluded that it probably doesn’t. There is probably no software/hardware separation in the brain. Such a separation is not genetically fit since it is energetically expensive and there is no need for brains to download new programs in the way computers do. There is some empirical evidence consistent with this: The mind and neural spiking is sensitive to many biophysical details like neurotransmitter trajectories and mitochondria.

The computational lens is powerful for modelling the brain. But if you look close enough, it breaks down. Computation is a map of the mind, but it probably isn’t the territory.

Chalmer’s fading qualia

David Chalmers argued for substrate independence with his fading qualia thought experiment. Imagine you woke up in the middle of the night to find out that Chalmers had kidnapped you and tied you to a hospital bed in his spooky lab at NYU. “I’m going to do a little experiment on you, but don’t worry it’s not very invasive. I’m just going to remove a single neuron from your brain, and replace it with one of these silicon chips my grad student invented.”

He explains that the chip performs the exact input/output behavior as the real neuron he’s going to remove, right down to the electrical signals and neurotransmitters. “Your experience won’t change” he claims, “your brain will still function just as it used to, so your mind will still all be there”. Before you get the chance to protest, his grad student puts you under general anesthetic and performs the procedure.

When you wake up David asks “Are you still conscious?” and you say “yes”. “Ok, do another one,” he says to his grad student, and you’re under again. The grad student continues to replace neurons with silicon chips one by one, checking each time if you are still conscious. Since each time it’s just a measly neuron that was removed, your answer never changes.

After one hundred billion operations, every neuron has been replaced with a chip. “Are you still conscious?” you answer “Yes”, because of course you do, your brain is functioning exactly like it did before. “Aha! I have proved substrate independence once and for all!” Chalmers exclaims, “Your mind is running on different hardware, yet your conscious experience has remained unchanged.”

Surely there can’t be a single neuron replacement that turns you into a philosophical zombie? That would mean your consciousness was reliant on that single neuron, which seems implausible.

The other option is that your consciousness gradually fades over the course of the operations. But surely you would notice that your experience was gradually fading and report it? To not notice the fading would be a catastrophic failure of introspection.

There are a couple of rebuttals to this I want to focus on.

But non-functionalists don’t trust cyborgs?

Schwitzgebel points out that Chalmers has an “audience problem”. Those he is trying to convince of functionalism are those who do not yet believe in functionalism. These non-functionalists are skeptical that the final product of the experiment, the person made of silicon chips, is conscious. So despite the fully silicon person reporting consciousness, the non-functionalist does not believe them^[6], since behaving like you are conscious is not conclusive evidence that you’re conscious.

The non-functionalist audience is also not obliged to trust the introspective reports at intermediate stages. A person with 50% neurons and 50% chips will report unchanged consciousness, but for the same reason as for the final state, the non-functionalist need not believe that report. Therefore, for a non-functionalist, it’s perfectly possible that the patient could continue to report normal consciousness while in fact their consciousness is fading.

Are neuron-replacing chips physically possible?

In how much detail would Chalmer’s silicon chips have to simulate the in/out behavior of the neurons? If the neuron doctrine was true, the chips could simply have protruding wires that give and receive electrical impulses. Then, it could have a tiny computer on board that has learned the correct in/out mapping.

But as I brought up in a previous post, neurons do not only communicate via electrical signals.^[7] The precise trajectories of neurotransmitters might also be important. When neurotransmitters arrive, where on the surface they penetrate, and how deep they get all influence the pattern of firing of the receiving neuron and what neurotransmitters it sends on to other cells.

The silicon chip would need detectors for each species of neurotransmitter on every point of its surface. It must use that data to simulate the neuron’s processing of the neurotransmitters. To simulate many precise trajectories within the cell could be very expensive. Could any device ever run such simulations quickly enough (so as to keep up with the pace of the biological neurons) on a chip small enough (so as to fit in amongst the biological neurons)?

It must also synthesize new neurotransmitters to send out to other neurons. To create the new neurotransmitters, the chip needs to have a supply of chemicals to build new neurotransmitters with. As to not run out, the chip will have to re-use the chemicals from incoming neurotransmitters.

And hey, since a large component of our expensive simulation is going to be tracking the transformation of old neurotransmitters to new neurotransmitters, we can dispose of that simulation since we’re actually just running those reactions in reality. Wait a minute, is this still a simulation of a neuron? Because it’s starting to just feel like a neuron.

Following this to its logical conclusion: when it comes down to actually designing these chips, a designer may end up discovering that the only way to reproduce all of the relevant in/out behavior of a neuron, is just to build a neuron!^[8]

Putnam’s octopus

So I’m not convinced by any of the arguments so far. This makes me start to wonder, where did CF come from in the first place? What were the ideas that first motivated it? CF was first defined and argued for by Hilary Putnam in the 60s, who justified it with the following argument.

An octopus mind is implemented in a radically different way than a human mind. Octopuses have a decentralized nervous system with most neurons located in their tentacles, a donut-shaped brain, and a vertical lobe instead of hippocampi (hippocampuses?) or neocortexes (neocortexi?).

But octopuses can have experiences like ours. Namely: octopuses seem to feel pain. They demonstrate aversive responses to harmful stimuli and they learn to avoid situations where they have been hurt. So we have the pain experience being created by two very different physical implementations. So pain is substrate-independent! Therefore multiple realizability, therefore CF.

I think this only works when we interpret multiple realizability at a suitably coarse-grained level of description of mental states (Bechtel & Mundale 2022). You can certainly argue that Octopus pain and human pain are of the same “type” (they play a similar function or have similar effects on the animal’s behavior). But since we’re interested in the phenomenal texture of that experience, we’re left with the question: how can we assume that octopus pain and human pain have the same quality?

If you want to use octopuses to argue that phenomenal consciousness is a program, the best you can do is a circular argument. How might we argue that human pain and octopus pain are the same experience? They seem to be playing the same function - both experiences are driving the animal to avoid harmful stimuli. Oh, so octopus pain and human pain are the same because they play the same function, in other words, because functionalism?

This concludes the tour of (what I interpret to be) the main arguments for CF.

What does a non-functionalist world look like?

What this means for AI consciousness

I still think conscious AI is possible even if CF is wrong.

If CF is true then AI might be conscious, since the AI could be running the same algorithms that make the human mind conscious. But does negating CF make AI consciousness impossible? To claim this without further argument is a denying the antecedent fallacy.^[9]

To say that CF is false is to say that consciousness isn’t totally explained by computation. But it’s another thing to say that the computational lens doesn’t tell you anything about how likely it is to be conscious. To claim that consciousness cannot emerge from silicon, one cannot just deny functionalism but they must also explain why biology has the secret sauce while chips do not.^[10]

If computation isn’t the source of consciousness, it could still be correlated with whatever that true source is. Cao 2022 argues that since function constrains implementation, then function tells us at least some things about other properties of the system (like the physical makeup):

From an everyday perspective, it may seem obvious that function constrains material make-up. A bicycle chain cannot be made of just anything—certain physical properties are required in order for it to perform as required within the functional organisation of the bicycle. Swiss cheese would make for a terrible bicycle chain, let alone a mind.

(Cao 2022)

For example, consciousness isn’t a function under this view, it probably still plays a function in biology.^[11] If that function is useful for future AI, then we can predict that consciousness will eventually appear in AI systems, since whatever property creates consciousness will be engineered into AI to improve its capabilities.

What this means for mind uploading

There is probably no simple abstraction of brain states that captures the necessary dynamics that encode consciousness. Scanning one’s brain and finding the “program of your mind” might be impractical because your mind, memories, personality etc are deeply entangled into the biophysical details of your brain. Geoffrey Hinton calls this mortal computation, a kind of information processing that involves an inseparable entanglement between software and hardware. If the hardware dies, the software dies with it.

Perhaps we will still be able to run coarse-grained simulations of our brains that capture various traits of ourselves, but if CF is wrong, those simulations will not be conscious. This makes me worried about a future where the lightcone is tiled with what we think to be conscious simulations, when in fact they are zombies with no moral value.

Conclusion

Computational functionalism has some problems: the most pressing one (in my book) is the problem of algorithmic arbitrariness. But are there strong arguments in favour of CF to counterbalance these problems? In this post, I went through the main arguments and have argued that they are not strong. So the overall case for CF is not strong either.

I hope you enjoyed this sequence. I’m going to continue scratching my head about computational functionalism, as I think it’s an important question and it’s tractable to update on its validity. If I find more crucial considerations, I might add more posts to this sequence.

Thanks for reading!

1. ^{^}

    I suspect this jump might be a source of confusion and disagreement in this debate. “The mind” could be read in a number of ways. It seems clear that many aspects of the mind can be explained by computation (e.g. its functional properties). In this article I’m only interested in the phenomenal consciousness element of the mind.

2. ^{^}

    CF could accommodate for “degrees of consciousness” rather than a binary on/off conception of consciousness, by saying that the degree of consciousness is defined by the program being run. Some programs are not conscious at all, some are slightly conscious, and some are very conscious.

3. ^{^}

    1 second of simulated time is computed at least every second in base reality.

4. ^{^}

    down to some very small length scale.

5. ^{^}

   See the "What this means for AI consciousness" section.

6. ^{^}

    Do non-fuctionalists say that we can’t trust introspective reports at all? Not necessarily. A non-functionalst would believe the introspective reports of other fully biological humans, because they are biological humans themselves and they are extrapolating the existence of their own consciousness to the other person. We’re not obliged to believe all introspective reports. A non-functionalist could poo-poo the report of the 50/50 human for the same reason that they poo-poo the reports of LaMBDA: reports are not enough to guarantee consciousness.

7. ^{^}

    Content warning: contentious claims from neuroscience. Feel free to skip or not update much, I won't be offended.

8. ^{^}

    Perhaps it doesn’t have to be exactly a neuron, there may still be some substrate flexibility - e.g., we have the freedom to rearrange certain internal chemical processes without changing the function. But in this case we have less substrate flexibility than computational functionalists usually assume, the replacement chip still looks very different to a typical digital computer chip.

9. ^{^}

    Denying the antecedent: A implies B, and not A, so not B. In our case: CF implies conscious AI, so not CF implies not conscious AI.

10. ^{^}

     The AI consciousness disbeliever must state a “crucial thesis” that posits a link between biology and consciousness tight enough to exclude the possibility of consciousness on silicon chips, and argue for that thesis.

11. ^{^}

     For example, modelling the environment in an energy efficient way.