To understand my position, first see this latest comment. It is that physical ontology is a subset of the true ontology, a bit like replacing a meaningful communication with a tree diagram. The tree structure is present in the original communication, and it inhabits everything to do with syntax and semantics, but the tree structure does not in itself contain the meaning.
Analogously, everything following "...which is formally..." is the abstracted description of consciousness, in mathematical/physical terms. The true ontology is the stuff about monadic intentionality with a subjective pole and an objective pole. My supposition is that this takes a finite number of bits to describe, and if you were to just talk about the structure and dynamics of those bits, solely in physical and computational terms, you would find yourself talking about (e.g.) nested qubit structures in the Hilbert space of entangled microtubular electrons. (That last is not a hypothesis that I advance with deadly seriousness and specificity, it's just usefully concrete.)
So if you want to talk about the basis of perception and knowledge, there are two levels available. There is the physical-computational level, and then the level of "true ontology". Perception and knowledge are really concepts at the deeper, truer level, because in truth they involve the "subjective" categories like intentionality, as well as the purely "objective" ones like structure and cause. But they will have their abstracted counterparts on the computational level of description.
In principle, the way we learn about the scientifically neglected subjective side of ontology is through phenomenology, i.e. introspection of an unusually systematic and rigorous sort, usually conducted in a doubting-Cartesian mode in which you put to one side the question of whether there is an external world causing your perceptions, and just focus on the nature of the perceptions themselves. Your question - what's going on when you perceive something as red, what's going on when you perceive fourness, and is there any difference - should be answered by introspective comparison of the two states.
In practice, any such introspection and comparison is likely to already be "theory-laden". This is one of the difficulties of the subject. Consider the very idea of intentionality, the idea that consciousness is all about a subject perceiving an object under an aspect. Now that I have the concept, it seems ubiquitously valid - every example of consciousness that I come upon, can be analyzed this way - and that offers a retroactive validation of the concept. But I can't say that I know how to get into a subjective state whereby I am agnostic about the existence of intentionality, and then have the intentional structure of consciousness forced upon me anyway, in the way that the existence of colors is impossible to deny. Maybe it becomes possible, at a higher level of phenomenological proficiency, to achieve a direct awareness of the reasons for believing in intentionality; or maybe it's a concept that is only ever validated in that retroactive way: once you have it, it becomes supremely plausible because of its analytical utility, but it's something that you have to hypothesize and "test" against the phenomenological "data", it's not something you can just "see directly" in the data.
My ideas about the difference between perceiving redness and perceiving fourness are on that level, at best; they are ideas that I picked up somehow, and which I can test against experience, but for which I don't have a subjective procedure which demonstrates them without presupposition, which is the epistemological gold standard for phenomenology...
A perceptual state of consciousness involves a "total object" which is "present" to a subject. This total object is what I called the "total instantaneous phenomenal state of affairs", by definition it's the union of all current objects of awareness; the "world" you are experiencing at a given moment. Some of these objects will be continua of qualia; for example, the total visual component of an experience. The subjective visual field is part of the world-object, along with other sensory continua. The subjective visual field isn't homogeneous, its hue, intensity, and value varies from location to location. This variation constitutes its form.
So far this is just a crude ontological analysis of the object end of an experience. When you ask how we perceive redness and fourness, you're also asking for an ontological analysis of how the object end relates to the subject end. In principle, that should derive from a phenomenological analysis of perceiving red and perceiving four... The trouble lies in distinguishing the component of the experience which is posited, from the component of the experience which is "given" - the part of the experience which is just there. I think fourness is posited on the basis of simpler local structural forms which are given, and I think there is a crude difference between red and, say, green, which is given, but more specific identification of colors requires conceptual synthesis, e.g. you have to notice that the shade of color is not just red, it's also dark, and then you can say it's a dark red.
Bertrand Russell and others talked about "knowledge by acquaintance" versus other forms of indirectly obtained knowledge; "knowledge by acquaintance" is the direct knowing that comes from direct awareness. So that which is given is known by acquaintance, and that which is posited is at best known to be consistent with experience. In this language, we know a shade of color as red-not-green by acquaintance, and we know that it is dark by acquaintance, but we know that it is dark red only by conceptual synthesis. And I think that a perception of fourness similarly arises from conceptual synthesis of more primitive facts that we know by acquaintance...
But one of the most challenging things is to say something convincing or even comprehensible about the direct awareness of objects by a subject. Should we treat qualia and this "total object" as part of the self, or as something external to the self that it's "aware of"? Is the awareness something that is caused by a particular relation between self and object, or is it the relation itself?
It's quite understandable why people prefer to focus on neurons, computation, and impersonal descriptions. If the physical side of my idea were ever validated, this would mean focusing on qubits, electron states, and so forth. But in the end, the vague and confusing subjective language of subject, object, awareness, acquaintance... would have to apply to entities and relations for which we also had a physical description. The "objective pole of the monadic intentionality" might correspond to "the union of all the leaves of the tree in the quantum data structure", and the "subjective pole" might be "the union of all the edges connected to the root node". (Undoubtedly that's not how it is, but again, concrete example for the purpose of discussion...)
You see intimations of this promised fusion between neurophysical, computational, and subjective ontologies when people have a feeling that it's all come together in their heads in a marvelous heap. "I am the computation, as well as the computer performing the computation!" might be how they express it, and behind this is a cognitive phenomenology in which there has been a miniature crossover and fusion of specific concepts from the different ontologies. I don't believe anyone has yet seen the truth of how it works, but the occasional illusion of insight gives us a foretaste of how the actual knowledge would feel, and meanwhile we need to keep switching back and forth between speculative synthesis and critical analysis, in order to make incremental progress. I just think getting to the answer requires a big leap in a new direction that's hard to convey.
If you intended to answer my question, you might want to know that after reading your response, I still have no idea whether on your account perceiving some system as comprised of four things requires some ontologically distinct noncomputational something-or-other in the same way that perceiving a system as red does.
If you intended to use my question as a launching pad from which to expound your philosophy, or intended to be obscurantist, then you might not.
Followup to: Nonsentient Optimizers
Why would you want to avoid creating a sentient AI? "Several reasons," I said. "Picking the simplest to explain first—I'm not ready to be a father."
So here is the strongest reason:
You can't unbirth a child.
I asked Robin Hanson what he would do with unlimited power. "Think very very carefully about what to do next," Robin said. "Most likely the first task is who to get advice from. And then I listen to that advice."
Good advice, I suppose, if a little meta. On a similarly meta level, then, I recall two excellent advices for wielding too much power:
Imagine that you knew the secrets of subjectivity and could create sentient AIs.
Suppose that you did create a sentient AI.
Suppose that this AI was lonely, and figured out how to hack the Internet as it then existed, and that the available hardware of the world was such, that the AI created trillions of sentient kin—not copies, but differentiated into separate people.
Suppose that these AIs were not hostile to us, but content to earn their keep and pay for their living space.
Suppose that these AIs were emotional as well as sentient, capable of being happy or sad. And that these AIs were capable, indeed, of finding fulfillment in our world.
And suppose that, while these AIs did care for one another, and cared about themselves, and cared how they were treated in the eyes of society—
—these trillions of people also cared, very strongly, about making giant cheesecakes.
Now suppose that these AIs sued for legal rights before the Supreme Court and tried to register to vote.
Consider, I beg you, the full and awful depths of our moral dilemma.
Even if the few billions of Homo sapiens retained a position of superior military power and economic capital-holdings—even if we could manage to keep the new sentient AIs down—
—would we be right to do so? They'd be people, no less than us.
We, the original humans, would have become a numerically tiny minority. Would we be right to make of ourselves an aristocracy and impose apartheid on the Cheesers, even if we had the power?
Would we be right to go on trying to seize the destiny of the galaxy—to make of it a place of peace, freedom, art, aesthetics, individuality, empathy, and other components of humane value?
Or should we be content to have the galaxy be 0.1% eudaimonia and 99.9% cheesecake?
I can tell you my advice on how to resolve this horrible moral dilemma: Don't create trillions of new people that care about cheesecake.
Avoid creating any new intelligent species at all, until we or some other decision process advances to the point of understanding what the hell we're doing and the implications of our actions.
I've heard proposals to "uplift chimpanzees" by trying to mix in human genes to create "humanzees", and, leaving off all the other reasons why this proposal sends me screaming off into the night:
Imagine that the humanzees end up as people, but rather dull and stupid people. They have social emotions, the alpha's desire for status; but they don't have the sort of transpersonal moral concepts that humans evolved to deal with linguistic concepts. They have goals, but not ideals; they have allies, but not friends; they have chimpanzee drives coupled to a human's abstract intelligence.
When humanity gains a bit more knowledge, we understand that the humanzees want to continue as they are, and have a right to continue as they are, until the end of time. Because despite all the higher destinies we might have wished for them, the original human creators of the humanzees, lacked the power and the wisdom to make humanzees who wanted to be anything better...
CREATING A NEW INTELLIGENT SPECIES IS A HUGE DAMN #(*%#!ING COMPLICATED RESPONSIBILITY.
I've lectured on the subtle art of not running away from scary, confusing, impossible-seeming problems like Friendly AI or the mystery of consciousness. You want to know how high a challenge has to be before I finally give up and flee screaming into the night? There it stands.
You can pawn off this problem on a superintelligence, but it has to be a nonsentient superintelligence. Otherwise: egg, meet chicken, chicken, meet egg.
If you create a sentient superintelligence—
It's not just the problem of creating one damaged soul. It's the problem of creating a really big citizen. What if the superintelligence is multithreaded a trillion times, and every thread weighs as much in the moral calculus (we would conclude upon reflection) as a human being? What if (we would conclude upon moral reflection) the superintelligence is a trillion times human size, and that's enough by itself to outweigh our species?
Creating a new intelligent species, and a new member of that species, especially a superintelligent member that might perhaps morally outweigh the whole of present-day humanity—
—delivers a gigantic kick to the world, which cannot be undone.
And if you choose the wrong shape for that mind, that is not so easily fixed—morally speaking—as a nonsentient program rewriting itself.
What you make nonsentient, can always be made sentient later; but you can't just unbirth a child.
Do less. Fear the non-undoable. It's sometimes poor advice in general, but very important advice when you're working with an undersized decision process having an oversized impact. What a (nonsentient) Friendly superintelligence might be able to decide safely, is another issue. But for myself and my own small wisdom, creating a sentient superintelligence to start with is far too large an impact on the world.
A nonsentient Friendly superintelligence is a more colorless act.
So that is the most important reason to avoid creating a sentient superintelligence to start with—though I have not exhausted the set.