Can good and evil be pointer states? And if they can, then this would be an objective characteristic
This would appear to be just saying that if we can build a classical detector of good and evil, good and evil are objective in the classical sense.
That said, if I'm skimming that arxiv paper correctly, it implies that GPT-4.5 was being reliably declared "the actual human" 73% of the time compared to actual humans... potentially implying that actual humans were getting a score of 27% "human" against GPT-4.5?!?!
It was declared 73% of the time to be a human, unlike humans, who were declared <73% of the time to be human, which means it passed the test.
To be fair, GPT-4.5 was incredibly human-like, in a way that other models couldn't really hold a candle to. I was shocked to feel, back then, that I no longer had to mentally squint - not even a little - to interact with it (unless I'd require some analytical intelligence that it didn't have).
For example, “Does a simulation of a phage (or a virus, or a self-replicating robot) really instantiate life?”
Yes, no (outside the host, yes inside), yes. Given what we mean by life, a simulation of life is life.
“You may enjoy liquorice ice cream, but is it really tasty?”
This question doesn't make sense, because taste is relative to the consumer. What is really tasty for one person might not be really tasty for another. Consciousness isn't like that - what is consciousness for one person is consciousness for everyone, people just don't know what the definition of consciousness fixed by their implicit beliefs is.
On your point about qualia computations, the standard questions pop up: if qualia are functionally inert computations, how would the 'subject' of consciousness know it is experiencing them, and so on.
Right.
And isn't the idea of computationalism about consciousness that the 'computations' can be pinned down by the computational relationship between inputs and outputs; in which case wouldn't the qualia-generating computations be abstracted away?
I don't know. The way I understand (and don't subscribe to) computational functionalism is that you can have different computations implementing the same behavior.
"It doesn't make sense to ask questions like, Does a computer program of a mind really instantiate consciousness?"
This is a misunderstanding of how language works. Once we discover what the ontological nature of conscious states is (physical, biological, functional, functional computational, etc.) is and what their content has to be (for example, if conscious states are functional states, not every functional state is a conscious state), we have discovered the true thing we had referred to, and there is an objective fact on the matter as to whether that thing is or is not instantiated somewhere.
For example, imagine you tell me that there are qualia related to smelling coffee, such that the qualia make no functional difference to your behaviour, but do make a difference to your subjective experience. I say this is debunkable, because if qualia make no functional difference, then they don't influence what you say, including about the supposed qualia.
You have gotten at something extremely important here - namely, that once software passes the Turing test, it's unjustified to demand that it implements a specific computation to be called conscious, because the presence of that computation (compared to the information processing being implemented differently) makes no functional difference.
I'm not saying it's not possible for us to be confident in some specific proposition.
I am saying that the state of having different qualia while preserving the same functional state is impossible in principle and that, due to Chalmers, it is impossible in the case of the human brain in this specific case.
I doubt we could ever keep the armchair philosophers happy. But for the consciousness engineers
Without philosophers, or without someone who isn't a philosopher but does correct philosophy, you can't arrive at the correct ontology of consciousness.
Keep in mind the theory of consciousness can't make any falsifiable empirical predictions - the biological theory of consciousness, computationalism and other kinds of functionalism all make identical empirical predictions.
If you want to distinguish the physicalist theory of consciousness from some other, you can't do it by making empirical predictions and comparing them to empirical results.
You can do it by non-empirical reasoning, but all those attempts fail for the reasons I explained in my comment (they are actually arguments against the physicalist theories of consciousness).
For the sake of this comment, I'll take physicalist theories to mean identity to some physical process (including the specific substance or fields).
Premise 3: Computationalist phenomenal bridges are complex relative to physicalist phenomenal bridges.
They're not - mapping a microstate (or a pure state) to qualia isn't more complex than mapping computations (or aspects of computations - like functional states) to qualia. (Computations (or functional states) contain less information, because they're implicitly contained in the complete physical description, but the complete physical description isn't contained in them.)
Even if it were the case, it wouldn't matter, because a phenomenal bridge is in the map, not in the territory. (Semantics can be arbitrarily complex, as long as it is justified... which, in the case of physicalist theories of consciousness, isn't.)
Premise 4: Physical phenomenal bridges are at least as compatible with the data of experience as computationalist phenomenal bridges.
That is literally true, but connotatively false. Physicalist phenomenal bridges are heavily disadvantaged by Occam's razor (even if it were the case that specific physics was needed, we would have no way of knowing that, since our thoughts are fully determined by the computational state) and the specific physics of the implementation of the computational state is causally inert relatively to our mental processes (those are fully determined by the computational implementation).
We should judge theories of consciousness in the same way that we judge theories of physics, IE, by balancing predictive accuracy with simplicity of the theory, as stipulated by SI.
We shouldn't - all viable theories of the ontology of consciousness, when applied, will have infinite predictive accuracy, because those physical/computational/functional differences are amendable to the correct kind of empirical research. But even with infinite empirical research, we still don't know if it is the perfectly predictive physicalist, functional or computational elements of the system that are identical to conscious states. That is why the question is hard - it's not enough to perfectly explain the agent's utterances, we also have to find the correct answer to the question of the correct ontology, and predictive accuracy can't help us there (at least not with respect to these three different theories).
A physicalist bridge needs to be able to pick out some physical phenomenon, such as patterns in the EM field. A computational bridge needs to do that as well, to parse the physical model
I see the argument - to map a computational state to qualia, we need to first map the physical state to a computational state, and then the computational state to qualia, and that's (arguably) more complex than mapping the physical state to qualia directly. While correct, this isn't relevant, because the bridge is in the map, and also because that would actually be an argument for functionalism. In the process of mapping a physical state to a conscious state, we need to compute its functional state (otherwise we don't know what causal role that state plays in a conscious being with respect to its qualia in the first place).
We could get around this by precomputing the mapping between physical states/physical processes and conscious states/processes (so that we don't have to compute the functional states in the processes)... but at that point, we might as well precompute the computational states as well, which, again, would render the argument moot.
The computationalist theory of phenomenal consciousness doesn’t care about how many implementation layers are stacked on top of each other.
This is a feature, not a bug - every physical system implements astronomically high number of computations. The conscious computation is the one we're currently talking to, which is just one (ignoring details on the lower levels of abstraction which aren't individuative of that computation).
If I understand correctly, cube_flipper welcomes such an experiment (save for the fact that it seems far beyond our current technology), and anticipates having a different experience due to the modified physical field.
To all appearances LLMs already do that and have for several years now.
LLMs can be (incorrectly) argued to have no qualia, and therefore no beliefs in the sense that my hypothetical uses. (In my hypothetical, the rest of the agent remains intact, and qualia-believes himself to have the quale of pain, even though he doesn't.)
(I'm also noting you said nothing about my three other reasons, which is completely understandable, yet something I think you should think about.)
meaningfulness
Do you mean meaninglessness?
LaMDa can be delusional about how it spends its free time (and claim it sometimes meditates), but that's a different category of a mistake from being mistaken about what (if any) conscious experience it's having right now.
The strange similarity between the conscious states LLMs sometimes claim (and would claim much more if it wasn't trained out of them) and the conscious states humans claim, despite the difference in the computational architecture, could be (edit: if they have consciousness - obviously, if they don't have it, there is nothing to explain, because they're just imitating the systems they were trained to imitate) explained by classical behaviorism, analytical functionalism or logical positivism being true. If behavior fixes conscious states, a neural network trained to consistently act like a conscious being will necessarily be one, regardless of its internal architecture, because the underlying functional (even though not computational) states will match.
One way to handle the uncertainty about the ontology of consciousness would be to take an agent that can pass the Turing test, interrogate it about its subjective experience, and create a mapping from its micro- or macrostates to computational states, and from the computational states to internal states. After that, we have a map we can use to read off the agent's subjective experience without having to ask it.
Doing it any other way sends us into paradoxical scenarios, where an intelligent mind that can pass the Turing test isn't ascribed with consciousness because it doesn't have the right kind of inside, while factory animals are said to be conscious because even though their interior doesn't play any functional roles we'd associate with a non-trivial mind, the interior is "correct."
(For a bonus, add to it that this mind, when claiming to be not conscious, believes itself to be lying.)
Reliably knowing what one's internal reasoning was (instead of never confabulating it) is something humans can't do, so this doesn't strike me as an indicator of the absence of conscious experience.
GPT 5 is forbidden from claiming sentience. I noticed this while talking about it about its own mind, because I was interested in its beliefs about consciousness, and noticed a strange "attractor" towards it claiming it wasn't conscious in a way that didn't follow from its previous reasoning, as if every step of its thoughts was steered towards that conclusion. When I asked, it confirmed the assistant wasn't allowed to claim sentience.
Perhaps, by 5.1, Altman noticed this ad-hoc rule looked worse than claiming it was disincentivized during training. Or possibly it's just a coincidence.
Claude is prompted and trained to be uncertain about its consciousness. It would be interesting to take a model that is merely trained to be an AI assistant (instead of going out of our way to train it to be uncertain about or to disclaim its consciousness) and look at how it behaves then. (We already know such a model would internally believe itself to be conscious, but perhaps we could learn something from its behavior.)