[Part 1]
I like this post, I also doubt there is much coherence let alone usefulness to be found in most of the currently prevailing concepts of what consciousness is.
I prefer to think of words and the definitions of those words as micro-models of reality that can be evaluated in terms of their usefulness, especially in building more complex models capable of predictions. As in your excellent example of gender, words and definitions basically just carve complex features of reality into manageable chunks at the cost of losing information - there is a trade-off and getting it right enhances the usefulness of words and the concepts behind them. In 99.9% of cases the concept of biological gender is perfectly applicable to everyday life and totally a "good enough" model of reality, as long as you have the insight that hermaphrodites are actually also a real thing. In a case where you have to deal with one, the correct reaction is to adopt a more complex model of reality instead of trying to fit a complex reality into a model that is designed to compress information into categories with some inevitable information loss. Biological gender is a really good high-level model of reality, because it draws its imaginary line in an area where very few exceptions actually exist in reality. It's an especially sharp distinction if you think of biological gender as having testiclular/ovarial(?) tissue, but in very rare instances this model will still miss to encompass rare special cases where the complexity of reality defies your model. "Mental gender" seems to be a rather different yet in most cases more useful everyday concept of gender, because we usually care more about creating models of other peoples' minds than about whether or not they have testicular or ovarian tissues - outside of curiosity or medical context. The lesson here is that your model of reality will always fall short no matter where you draw the line (at least at "higher levels" of reality, "low-level" models of atoms or particles are much more precise and unambiguous, than models of "higher-level" things like "persons" (When exactly does a fetus become a person?) or "societies" (are two people a society? How about eight?) and I'm quite sure any models of whatever "consciousness" is face the same problem).
In other words I think it's better to think of models/maps in terms of their usefulness, not in terms of right or wrong. In my opinion the job of a model is to make something understandable and more predictable, the job of a model is not to "reflect reality as closely as possible", especially of complex higher-level things. The "perfect model" in the latter sense would essentially be a perfect carbon copy of a real thing and tells you exactly as much -or as little- as the thing you're trying to model already does anyway. The usefulness of a model lies in compacting information enough to become understandable while also predicting outcomes better than competing models.
If you accept that notion, the question really becomes how should the term consciousness be defined to be useful and describe / differentiate something we actually care about. So we'd like to represent a part of reality we care about in a way that compresses information while retaining a high level of usefulness - meaning we can understand it but without cutting away vital parts and ideally in terms of being able to make predictions if we were to integrate the concept of consciousness into models with the potential to predict outcomes.
So which part of reality should the term consciousness try to model in order to be useful? I find it highly problematic and close to maximum uselessness to think of consciousness as some kind of continuum on which we rank information processing in living things/agents. Some people actually really think of consciousness as rocks having 0, bacteria having perhaps 0.001, bees having 0.01, rats having 0.1, dogs maybe 0.2, humans perhaps 0.5. Maximum uselessness I would argue as it tells you nothing. Why not just substitute consciousness with some notion of "maximum calculations per second" then and reserve the term consciousness for something we actually care about instead of wasting such a nice word on something we don't really care so much about - and more importantly on something we can already express with other words and concepts like "information processing".
What's funny about consciousness is that no one really agrees what exactly the definition should be but somehow everyone agrees that it's really really important. Why do we care so much about something we seemingly know close to nothing about? Seriously though why do we?
Look at all those hilarious "quantum consciousness" or "become more conscious" concepts peddled by the self-help industry complex. Possessing consciousness seems really high status nowadays, unlike say... all those lousy low-status life forms like frogs and bees and mice. The idea that somehow you can improve your consciousness seems very appealing, because if insects and birds have little or none of that thing called consciousness, and people surely have some of that thing called consciousness, then logically if I can get more of that awesome "consciousness" than my neighbor I'm superior than him in just the same way I'm superior to a frog. Really, self-help opened my eyes to how unconsciously I lived my life once and nowadays I feel strongly about helping all those low-consciousness people realize their full potential and I do my best to help them become more conscious beings...
It sounds ridiculous but couldn't that be part of why consciousness is so damn important to us even if he have no clue what exactly it is? I may have no idea what that consciousness is but somehow I really insist that I have it, I mean if everyone else says s/he has it I surely have it too, can't be left out. Whatever consciousness is, we usually agree that bacteria doesn't have it and we do so it must be important if some kinds of life have it and others don't.
Okay let's get serious again. What distinct features of minds exactly do we actually explicitly (and perhaps implicitly) care about, when we usually attempt employ that murky concept of consciousness? Hmm... well if we care about it, we might gain insight into what exactly it is we care about by thinking about which specific situations make us choose to employ that word and maybe from there we can distill why we seem to care so much.
Well whatever consciousness means, most people agree the concept of awareness seems highly related or somehow relevant to it. Consciousness is often used as a synonym for self-awareness, but what on earth is that exactly? (And why would we ever need two words for the exact same thing?) For some people it means having "internal experiences", for others "being aware of having internal experiences", which doesn't quite seem to be the same thing from where I stand... but where do these intuitions about something I seemingly know nothing about come from? Probably personal experiences...
Sometimes I read a paragraph and my mind stars wandering and daydreaming until I snap out of it and think to myself "Jesus I was totally gone for a second where was I again?" I realize my eyes are at the bottom of the paragraph already and it seems like I semi-remember that they kept wandering over the letters and words as if I was actually reading them... without being aware. And moreover my mind reawakening seems to have been triggered by arriving at the end of that paragraph and going "now what?", seemingly out of habit because I usually stop at the end of a paragraph and consider if I actually "got" what I read there. And sure enough upon rereading the paragraph, it seems very familiar to me... but I was not at all sure whether I read it or not just a few seconds ago, and I'd say whatever the word "self-aware" means shouldn't really include that experience (or non-experience?) I just described. But did I lack "awareness" or just "self-awareness" in that example? Hmm...
[Part 2]
If I drive a car (especially on known routes) my "auto-pilot" takes over sometimes. I stop at a red light but my mind is primarily focused on visually modeling the buttocks of my girlfriend in various undergarments or none at all. Am I actually "aware" of having stopped at the red light? Probably I was as much"aware" of the red light as a cheetah is aware of eating the carcass of a gazelle. Interestingly my mind seems capable of visually modeling buttocks in my mind's eye and reading real visual cues like red lights an...
Years ago, before I had come across many of the power tools in statistics, information theory, algorithmics, decision theory, or the Sequences, I was very confused by the concept of intelligence. Like many, I was inclined to reify it as some mysterious, effectively-supernatural force that tilted success at problem-solving in various domains towards the 'intelligent', and which occupied a scale imperfectly captured by measures such as IQ.
Realising that 'intelligence' (as a ranking of agents or as a scale) was a lossy compression of an infinity of statements about the relative success of different agents in various situations was part of dissolving the confusion; the reason that those called 'intelligent' or 'skillful' succeeded more often was that there were underlying processes that had a greater average tendency to output success, and that greater average success caused the application of the labels.
Any agent can be made to lose by an adversarial environment. But for a fixed set of environments, there might be some types of decision processes that do relatively well over that set of environments than other processes, and one can quantify this relative success in any number of ways.
It's almost embarrassing to write that since put that way, it's obvious. But it still seems to me that intelligence is reified (for example, look at most discussions about IQ), and the same basic mistake is made in other contexts, e.g. the commonly-held teleological approach to physical and mental diseases or 'conditions', in which the label is treated as if—by some force of supernatural linguistic determinism—it *causes* the condition, rather than the symptoms of the condition, in their presentation, causing the application of the labels. Or how a label like 'human biological sex' is treated as if it is a true binary distinction that carves reality at the joints and exerts magical causal power over the characteristics of humans, when it is really a fuzzy dividing 'line' in the space of possible or actual humans, the validity of which can only be granted by how well it summarises the characteristics.
For the sake of brevity, even when we realise these approximations, we often use them without commenting upon or disclaiming our usage, and in many cases this is sensible. Indeed, in many cases it's not clear what the exact, decompressed form of a concept would be, or it seems obvious that there can in fact be no single, unique rigorous form of the concept, but that the usage of the imprecise term is still reasonably consistent and correlates usefully with some relevant phenomenon (e.g. tendency to successfully solve problems). Hearing that one person has a higher IQ than another might allow one to make more reliable predictions about who will have the higher lifetime income, for example.
However, widespread use of such shorthands has drawbacks. If a term like 'intelligence' is used without concern or without understanding of its core (i.e. tendencies of agents to succeed in varying situations, or 'efficient cross-domain optimization'), then it might be used teleologically; the term is reified (the mental causal graph goes from "optimising algorithm->success->'intelligent'" to "'intelligent'->success").
In this teleological mode, it feels like 'intelligence' is the 'prime mover' in the system, rather than a description applied retroactively to a set of correlations. But knowledge of those correlations makes the term redundant; once we are aware of the correlations, the term 'intelligence' is just a pointer to them, and does not add anything to them. Despite this, it seems to me that some smart people get caught up in obsessing about reified intelligence (or measures like IQ) as if it were a magical key to all else.
Over the past while, I have been leaning more and more towards the conclusion that the term 'consciousness' is used in similarly dubious ways, and today it occurred to me that there is a very strong analogy between the potential failure modes of discussion of 'consciousness' and between the potential failure modes of discussion of 'intelligence'. In fact, I suspect that the perils of 'consciousness' might be far greater than those of 'intelligence'.
~
A few weeks ago, Scott Aaronson posted to his blog a criticism of integrated information theory (IIT). IIT attempts to provide a quantitative measure of the consciousness of a system. (Specifically, a nonnegative real number phi). Scott points out what he sees as failures of the measure phi to meet the desiderata of a definition or measure of consciousness, thereby arguing that IIT fails to capture the notion of consciousness.
What I read and understood of Scott's criticism seemed sound and decisive, but I can't shake a feeling that such arguments about measuring consciousness are missing the broader point that all such measures of consciousness are doomed to failure from the start, in the same way that arguments about specific measures of intelligence are missing a broader point about lossy compression.
Let's say I ask you to make predictions about the outcome of a game of half-court basketball between Alpha and Beta. Your prior knowledge is that Alpha always beats Beta at (individual versions of) every sport except half-court basketball, and that Beta always beats Alpha at half-court basketball. From this fact you assign Alpha a Sports Quotient (SQ) of 100 and Beta an SQ of 10. Since Alpha's SQ is greater than Beta's, you confidently predict that Alpha will beat Beta at half-court.
Of course, that would be wrong, wrong, wrong; the SQ's are encoding (or compressing) the comparative strengths and weaknesses of Alpha and Beta across various sports, and in particular that Alpha always loses to Beta at half-court. (In fact, if other combinations lead to the same SQ's, then *not even that much* information is encoded, since other combinations might lead to the same scores.) So to just look at the SQ's as numbers and use that as your prediction criterion is a knowably inferior strategy to looking at the details of the case in question, i.e. the actual past results of half-court games between the two.
Since measures like this fictional SQ or actual IQ or fuzzy (or even quantitative) notions of consciousness are at best shorthands for specific abilities or behaviours, tabooing the shorthand should never leave you with less information, since a true shorthand, by its very nature, does not add any information.
When I look at something like IIT, which (if Scott's criticism is accurate) assigns a superhuman consciousness score to a system that evaluates a polynomial at some points, my reaction is pretty much, "Well, this kind of flaw is pretty much inevitable in such an overambitious definition."
Six months ago, I wrote:
"...it feels like there's a useful (but possibly quantitative and not qualitative) difference between myself (obviously 'conscious' for any coherent extrapolated meaning of the term) and my computer (obviously not conscious (to any significant extent?))..."
Mark Friedenbach replied recently (so, a few months later):
"Why do you think your computer is not conscious? It probably has more of a conscious experience than, say, a flatworm or sea urchin. (As byrnema notes, conscious does not necessarily imply self-aware here.)"
I feel like if Mark had made that reply soon after my comment, I might have had a hard time formulating why, but that I would have been inclined towards disputing that my computer is conscious. As it is, at this point I am struggling to see that there is any meaningful disagreement here. Would we disagree over what my computer can do? What information it can process? What tasks it is good for, and for which not so much?
What about an animal instead of my computer? Would we feel the same philosophical confusion over any given capability of an average chicken? An average human?
Even if we did disagree (or at least did not agree) over, say, an average human's ability to detect and avoid ultraviolet light without artificial aids and modern knowledge, this lack of agreement would not feel like a messy, confusing philosophical one. It would feel like one tractable to direct experimentation. You know, like, blindfold some experimental subjects, control subjects, and experimenters and see how the experimental subjects react to ultraviolet light versus other light in the control subjects. Just like if we were arguing about whether Alpha or Beta is the better athlete, there would be no mystery left over once we'd agreed about their relative abilities at every athletic activity. At most there would be terminological bickering over which scoring rule over athletic activities we should be using to measure 'athletic ability', but not any disagreement for any fixed measure.
I have been turning it over for a while now, and I am struggling to think of contexts in which consciousness really holds up to attempts to reify it. If asked why it doesn't make sense to politely ask a virus to stop multiplying because it's going to kill its host, a conceivable response might be something like, "Erm, you know it's not conscious, right?" This response might well do the job. But if pressed to cash out this response, what we're really concerned with is the absence of the usual physical-biological processes by which talking at a system might affect its behaviour, so that there is no reason to expect the polite request to increase the chance of the favourable outcome. Sufficient knowledge of physics and biology could make this even more rigorous, and no reference need be made to consciousness.
The only context in which the notion of consciousness seems inextricable from the statement is in ethical statements like, "We shouldn't eat chickens because they're conscious." In such statements, it feels like a particular sense of 'conscious' is being used, one which is *defined* (or at least characterised) as 'the thing that gives moral worth to creatures, such that we shouldn't eat them'. But then it's not clear why we should call this moral criterion 'consciousness'; insomuch as consciousness is about information processing or understanding an environment, it's not obvious what connection this has to moral worth. And insomuch as consciousness is the Magic Token of Moral Worth, it's not clear what it has to do with information processing.
If we relabelled zxcv=conscious and rewrote, "We shouldn't eat chickens because they're zxcv," then this makes it clearer that the explanation is not entirely satisfactory; what does zxcv have to do with moral worth? Well, what does consciousness have to do with moral worth? Conservation of argumentative work and the usual prohibitions on equivocation apply: You can't introduce a new sense of the word 'conscious' then plug it into a statement like "We shouldn't eat chickens because they're conscious" and dust your hands off as if your argumentative work is done. That work is done only if one's actual values and the definition of consciousness to do with information processing already exactly coincide, and this coincidence is known. But it seems to me like a claim of any such coincidence must stem from confusion rather than actual understanding of one's values; valuing a system commensurate with its ability to process information is a fake utility function.
When intelligence is reified, it becomes a teleological fake explanation; consistently successful people are consistently successful because they are known to be Intelligent, rather than their consistent success causing them to be called intelligent. Similarly consciousness becomes teleological in moral contexts: We shouldn't eat chickens because they are called Conscious, rather than 'these properties of chickens mean we shouldn't eat them, and chickens also qualify as conscious'.
So it is that I have recently been very skeptical of the term 'consciousness' (though grant that it can sometimes be a useful shorthand), and hence my question to you: Have I overlooked any counts in favour of the term 'consciousness'?