I actually think what you are going for is closer to JL Austin's notion of an illocutionary act than anything in Wittgenstein, though as you say, it is an analysis of a particular token of the type ("believing in"), not an analysis of the type. Quoting Wikipedia:
"According to Austin's original exposition in How to Do Things With Words, an illocutionary act is an act:
Your model of "believing in" is essentially an unpacking of the "conventional consequences" produced by using the locution in various contexts. I think it is a good unpacking, too!
I do think that some of the contrasts you draw (belief vs. believing in) would work equally well (and with more generality) as contrasts between beliefs and illocutionary acts, though.
In Leibniz’ case, he’s known almost exclusively for the invention of calculus.
Was this supposed to be a joke (if so, consider me well and truly whooshed)? At any rate, it is most certainly not the case. Leibniz is known for a great many things (both within and without mathematics) as can be seen from a cursory glance at his Wikipedia page.
Rather, they might be mere empty machines. Should you still tolerate/respect/etc them, then?"
My sense is that I'm unusually open to "yes," here.
I think the discussion following from here is a little ambiguous (perhaps purposefully so?). In particular, it is unclear which of the following points are being made:
1: Sufficient uncertainty with respect to the sentience (I'm taking this as synonymous with phenomenal consciousness) of future AIs should dictate that we show them tolerance/respect etc...
2: We should not be confident that sentience is a good criterion for moral patienthood (i.e., being shown tolerance/respect etc...), even though sentience is a genuine thing.
3: We should worry that sentience isn't a genuine thing at all (i.e, illusionism / as-yet-undescribed re-factorings of what we currently call sentience).
When you wrote that you are unusually open to "yes" in the quoted sentence, I took the qualifier "unusual" to indicate that you were making point 2, since I do not consider point 1 to be particularly unusual (Schwitzgebel has pushed for this view, for example). However, your discussion then mostly seemed to be making the case for point 1 (i.e., we could impose a criterion for moral worth that is intended to demarcate non-sentient and sentient entities but that fails). For what it's worth, I would be very interested to hear arguments for point 2 which do not collapse into point 1 (or, alternatively, some reason why I am mistaken for considering them distinct points). From my perspective, it is hard to understand how something which really lacks what I mean by phenomenal consciousness could possibly be a moral patient. Perhaps it is related to the fact that I have, despite significant effort, utterly failed to grok illusionism.
Apologies, I had thought you would be familiar with the notion of functionalism. Meaning no offence at all but it's philosophy of mind 101, so if you're interested in consciousness, it might be worth reading about it. To clarify further, you seem to be a particular kind of computational functionalist. Although it might seem unlikely to you, since I am one of those "masturbatory" philosophical types who thinks it matters how behaviours are implemented, I am also a computational functionalist! What does this mean? It means that computational functionalism is a broad tent, encompassing many different views. Let's dig into the details of where we differ...
If something can talk, then, to a functionalist like me, that means it has assembled and coordinated all necessary hardware and regulatory elements and powers (that is, it has assembled all necessary "functionality" (by whatever process is occurring in it which I don't actually need to keep track of (just as I don't need to understand and track exactly how the brain implements language))) to do what it does in the way that it does.
This is a tautology. Obviously anything that can do a thing ("talk") has assembled the necessary elements to do that very thing in the way that it does. The question is whether or not we can make a different kind of inference, from the ability to implement a particular kind of behaviour (linguistic competence) to the possession of a particular property (consciousness).
Once you are to the point of "seeing something talk fluently" and "saying that it can't really talk the way we can talk, with the same functional meanings and functional implications for what capacities might be latent in the system" you are off agreeing with someone as silly as Searle. You're engaged in some kind of masturbatory philosophy troll where things don't work and mean basically what they seem to work and mean using simple interactive tests.
Okay, this is the key passage. I'm afraid your view of the available positions is seriously simplistic. It is not the case that anybody who denies the inference from 'displays competent linguistic behaviour' to 'possesses the same latent capacities' must be in agreement with Searle. There is a world of nuance between your position and Searle's, and most people who consider these questions seriously occupy the intermediate ground.
To be clear, Searle is not a computational functionalist. He does not believe that non-biological computational systems can be conscious (well, actually he wrote about "understanding" and "intentionality", but his arguments seem to apply to consciousness as much or even more than they do to those notions). On the other hand, the majority of computational functionalists (who are, in some sense, your tribe) do believe that a non-biological computational system could be conscious.
The variation within this group is typically with respect to which computational processes in particular are necessary. For example, I believe that a computational implementation of a complex biological organism with a sufficiently high degree of resolution could be conscious. However, LLM-based chatbots are nowhere near that degree of resolution. They are large statistical models that predict conditional probabilities and then sample from them. What they can do is amazing. But they have little in common with living systems and only by totally ignoring everything except for the behavioural level can it even seem like they are conscious.
By the way, I wouldn't personally endorse the claim that LLM-based chatbots "can't really talk the way we talk". I am perfectly happy to adopt a purely behavioural perspective on what it means to "be able to talk". Rather, I would deny the inference from that ability to the possession of consciousness. Why would I deny that? For the reasons I've already given. LLMs lack almost all of the relevant features that philosophers, neuroscientists, and biologists have proposed as most likely to be necessary for consciousness.
Unsurprisingly, no, you haven't changed my mind. Your claims require many strong and counterintuitive theoretical commitments for which we have either little or no evidence. I do think you should take seriously the idea that this may explain why you have found yourself in a minority adopting this position. I appreciate that you're coming from a place of compassion though, that's always to be applauded!
I am sorry that you got the impression I was trolling. Actually I was trying to communicate to you. None of the candidate criteria I suggested were conjured ex nihilo out of a hat or based on anything that I just made up. Unfortunately, collecting references for all of them would be pretty time consuming. However, I can say that the global projection phrase was gesturing towards global neuronal workspace theory (and related theories). Although you got the opposite impression, I am very familiar with consciousness research (including all of the references you mentioned, though I will admit I don't think much of IIT).
The idea of "meat chauvinism" seems to me a reductive way to cast aside the possibility that biological processes could be relevant to consciousness. I think this is a theoretical error. It is not the case that taking biological processes seriously when thinking about consciousness implies (my interpretation of what you must mean by) "meat chauvinism". One can adopt a functionalist perspective on biological processes that operate far below the level of a language production system. For example, one could elaborate a functional model of metabolism which could be satisfied by silicon-based systems. In that sense, it isn't meat chauvinism to suggest that various biological processes may be relevant to consciousness.
This would discount human uploads for silly reasons. Like if I uploaded and was denied rights for lack of any of these things they I would be FUCKING PISSED OFF
Assuming what you mean by "fucking pissed off" involves subjective experience and what you mean by "I was uploaded" would not involve implementing any of the numerous candidates for necessary conditions on consciousness that I mentioned, this is simply begging the question.
To me, it doesn't make any sense to say you would have been "uploaded" if all you mean is a reasonably high fidelity reproduction of your input-output linguistic behaviour had been produced. If what you mean by uploaded is something very different which would require numerous fundamental scientific breakthroughs then I don't know what I would say, since I don't know how such an upload would fare with respect to the criteria for conscious experience I find most compelling.
Generally speaking, there is an enormous difference between hypothetical future simulated systems of arbitrary sophistication and the current generation of LLM-based chatbots. My sense is that you are conflating these things when assessing my arguments. The argument is decidedly not that the LLM-based chatbots are non-biological, therefore they cannot be conscious. Nor is it that no future silicon-based systems, regardless of functional organisation, could ever be conscious. Rather, the argument is that LLM-based chatbots lack almost all of the functional machinery that seems most likely to be relevant for conscious experience (apologies that using the biological terms for these aspects of functional machinery was misleading to you), therefore they are very unlikely to be conscious.
I agree that the production of coherent linguistic output in a system that lacks this functional machinery is a scientific and technological marvel, but it is only evidence for conscious experience if your theory of consciousness is of a very particular and unusual variety (relative to the fields which study the topic in a professional capacity, perhaps such ideas have greater cache on this website in particular). Without endorsing such a theory, the evidence you provide from what LLMs produce, given their training, does not move me at all (we have an alternative explanation for why LLMs produce such outputs which does not route through them being subjectively experiencing entities, and what's more, we know the alternative explanation is true, because we built them).
Given how you responded above, I have the impression you think neuroscience and biology are not that relevant for understanding consciousness. Clearly, I disagree. May I ask what has given you the impression that the biological details don't matter (even when given a functional gloss such that they may be implemented in silico)?
I think you're missing something important.
Obviously I can't speak to the reason there is a general consensus that LLM-based chatbots aren't conscious (and therefore don't deserve rights). However, I can speak to some of the arguments that are still sufficient to convince me that LLM-based chatbots aren't conscious.
Generally speaking, there are numerous arguments which essentially have the same shape to them. They consist of picking out some property that seems like it might be a necessary condition for consciousness, and then claiming that LLM-based chatbots don't have that property. Rather than spend time on any one of these arguments, I will simply list some candidates for such a property (these may be mentioned alone or in some combination):
Metabolism, Temporally continuous existence, Sensory perception, Integration of sensory signals, Homeostatic drives, Interoception, Coherent self-identity, Dynamic coupling to the environment, Affective processes, A nervous system, Physical embodiment, Autonomy, Autopoiesis, Global projection of signals, Self-monitoring, Synchronized neuronal oscillations, Allostasis, Executive function, Nociceptors, Hormones... I could keep going.
Naturally, it may that some of these properties are irrelevant or unnecessary for consciousness. Or it could be that even altogether they are insufficient. However, the fact that LLM-based chatbots possess none of these properties is at least some reason to seriously doubt that they could be conscious.
A different kind of argument focuses more directly on the grounds for the inference that LLM-based chatbots might be conscious. Consider the reason that coherent linguistic output seems like evidence of consciousness in the first place. Ordinarily, coherent linguistic output is produced by other people and suggests consciousness to us based on a kind of similarity-based reasoning. When we encounter other people, they are engaging in similar behaviour to us, which suggests they might be similar to us in other ways, such as having subjective experience. However, this inference would no longer be justified if there was a known, significantly different reason for a non-human entity to produce coherent linguistic output. In the case of LLM-based chatbots, we do have such a reason. In particular, the reason is the data-intensive training procedure, a very different story for how other humans come to learn to produce coherent linguistic output.
Nobody should be 100% confident in any claims about which entities are or are not conscious, but the collective agreement that LLM-based chatbots are not seems pretty reasonable.
Enjoyable post, I'll be reading the rest of them. I especially appreciate the effort that went into warding off the numerous misinterpretations that one could easily have had (but I'm going to go ahead an ask something that may signal I have misinterpreted you anyhow).
Perhaps this question reflects poor reading comprehension, but I'm wondering whether you are thinking of valence as being implemented by something specific at a neurobiological level or not? To try and make the question clearer (in my own head as much as anything), let me lay out two alternatives to having valence implemented by something specific. First, one might imagine that valence is an abstraction over the kind of competitive dynamics that play out among thoughts. On this view, valence is a little like evolutionary fitness (the tautology talk in 1.5.3 brought this comparison to mind). Second, one might imagine that valence is widely distributed across numerous brain systems. On this view, valence is something like an emotion (if you'll grant the hopefully-no-longer-controversial claim that the neural bases of emotions are widely distributed). I don't think either of these alternatives are what you are going for, but I also didn't see the outright claim that valence is something implemented by a specific neurobiological substrate. What do you believe?
In other words, you think that even in a world where the distribution of mathematical methods were very specific to subject areas, this methodology would have failed to show that? If so, I think I disagree (though I agree the evidence of the paper is suggestive, not conclusive). Can you explain in more detail why you think that? Just to be clear, I think the methodology of the paper is coarse, but not so coarse as to be unable to pick out general trends.
Perhaps to give you a chance to say something informative, what exactly did you have in mind by "united around methodology" when you made the original comment I quoted above?
Ok, I do really like that move, and generally think of fields as being much more united around methodology than they are around subject-matter. So maybe I am just lacking a coherent pointer to the methodology of complex-systems people.
The extent to which fields are united around methodologies is an interesting question in its own right. While there are many ways we could break this question down which would probably return different results, a friend of mine recently analysed it with respect to mathematical formalisms (paper: https://link.springer.com/article/10.1007/s11229-023-04057-x). So, the question here is, are mathematical methods roughly specific to subject areas, or is there significant mathematical pluralism within each subject area? His findings suggest that, mostly, it's the latter. In other words, if you accept the analysis here (which is rather involved and obviously not infallible), you should probably stop thinking of fields as being united by methodology (thus making complex systems research a genuinely novel way of approaching things).
Key quote from the paper: "if the distribution of mathematical methods were very specific to subject areas, the formula map would exhibit very low distance scores. However, this is not what we observe. While the thematic distances among formulas in our sample are clearly smaller than among randomly sampled ones, the difference is not drastic, and high thematic coherence seems to be mostly restricted to several small islands."
I think the kind of sensible goalpost-moving you are describing should be understood as run-of-the-mill conceptual fragmentation, which is ubiquitous in science. As scientific communities learn more about the structure of complex domains (often in parallel across disciplinary boundaries), numerous distinct (but related) concepts become associated with particular conceptual labels (this is just a special case of how polysemy works generally). This has already happened with scientific concepts like gene, species, memory, health, attention and many more.
In this case, it is clear to me that there are important senses of the term "general" which modern AI satisfies the criteria for. You made that point persuasively in this post. However, it is also clear that there are important senses of the term "general" which modern AI does not satisfy the criteria for. Steven Byrnes made that point persuasively in his response. So far as I can tell you will agree with this.
If we all agree with the above, the most important thing is to disambiguate the sense of the term being invoked when applying it in reasoning about AI. Then, we can figure out whether the source of our disagreements is about semantics (which label we prefer for a shared concept) or substance (which concept is actually appropriate for supporting the inferences we are making).
What are good discourse norms for disambiguation? An intuitively appealing option is to coin new terms for variants of umbrella concepts. This may work in academic settings, but the familiar terms are always going to have a kind of magnetic pull in informal discourse. As such, I think communities like this one should rather strive to define terms wherever possible and approach discussions with a pluralistic stance.