"My intuition is that making Mary superhuman doesn't change that experiencing red seems to narrow down the possibilities for her."
"I disagree."
You... disagree? Do you mean your own intuition is different, or do you mean you have some special insight into my psychology that tells you that I'm misunderstanding or misrepresenting my own intuitions?
I'm reporting on psychological data about what my intuitions are indicating to me. I'm not a dualist, so I'm not (yet) making any assertions about what Mary would actually do or say or know. I'm explaining what output my simulator is giving me when I run the thought experiment.
If Mary was a superhuman she could study what functions of the brain cause us to experience "qualia," and then study the memories these processes generated. She could then generate such memories in her own brain, giving her the knowledge of what qualia feel like without ever experiencing them.
You're assuming that all superhumans intelligent enough to understand the biophysics of color vision will also necessarily have a module that allows them to self-modify in a way that they have whatever first-person subjective experience they wish. There's no reason to assume that. As long as a Mary without this capacity (but with the third-person biophysics-comprehending capacity) is possible, the argument goes through. The fact that a Mary that can spontaneously generate its own experience of redness is also possible doesn't make any progress toward refuting or dissolving the Mary hunch.
It sounds to me like you've been reading too much Dennett. Dennett is not a careful or patient dissector of the Hard Problem. The entire RoboMary paper, for instance, is a non sequitur in relation to the arguments it's meant to refute. It's fun and interesting, but it's talking about a different subject matter.
If qualia were not a physical part of the brain, duplicating the memories of someone who had experienced them would not have this effect.
That's not true at all. Most forms of dualism allow Mary to generate the relevant mental states by manipulating the physical states they are causally tied to.
"Can you explain why this intuition persists for me, when (as far as I can tell) it doesn't for any other complex system?"
"Because, as I said before, our emotions are 'black boxes' that humans are very bad at understanding and explaining."
This doesn't look to me like an explanation yet, even an outline of one. In fact, it looks like an appeal to the Black Box black box: 'Black box' is being used as a special word meant to pick out some uniquely important and effective category of Unknown Thingie. But just saying 'we don't understand emotions yet' doesn't tell me anything about why emotions appear irreducible to me, while other things I don't understand do seem reducible to me.
Their Kolmogorov complexity is extraordinarily high, but we feel like they are simple because of our familiarity with them.
I don't feel that mental states are simple! Yet the Mary hunch persists. You seem to be hopping back and forth between the explanations 'qualia seem irreducible because we don't know enough about them yet' and 'qualia seem irreducible because we don't realize how complicated they are'. But neither of these explanations makes me any less confused, and they're both incredibly vague. I think this is a legitimate place to insist that we say not "complexity".
I think the ability to study and modify her own source code and memory, as well as the source code and memory of others is probably all she'd need,
Why, specifically, would any of those four abilities help? Are all four needed? Are some more important than others? Why, for instance, wouldn't just studying my own source code and memory (without being able to do radical surgery on it) suffice for knowing the phenomenal character of redness, or the phenomenal character of a bat's echolocation...?
You... disagree? Do you mean your own intuition is different, or do you mean you have some special insight into my psychology that tells you that I'm misunderstanding or misrepresenting my own intuitions?
I mean my intuition is different.
I don't feel that mental states are simple! Yet the Mary hunch persists. You seem to be hopping back and forth between the explanations 'qualia seem irreducible because we don't know enough about them yet' and 'qualia seem irreducible because we don't realize how complicated they are'.
Alright, I'll try to stop hoppin...
Related to: When Anthropomorphism Became Stupid, Reductionism, How to Convince Me That 2 + 2 = 3
"Reality is normal." That is: Surprise, confusion, and mystery are features of maps, not of territories. If you would think like reality, cultivate outrage at yourself for failing to intuit the data, not resentment at the data for being counter-intuitive.
"Not one unusual thing has ever happened." That is: Ours is a tight-knit and monochrome country. The cosmos is simple, tidy, lawful. "[T]here is no surprise from a causal viewpoint — no disruption of the physical order of the universe."
"It all adds up to normality." That is: Whatever is true of fundamental reality does not exist in a separate universe from our everyday activities. It composes those activities. The perfected description of our universe must in principle allow us to reproduce the appearances we started with.
These maxims are remedies to magical mereology, anthropocentrism, and all manner of philosophical panic. But reading too much (or too little) into them can lead seekers from the Path. For instance, they may be wrongly taken to mean that the world is obliged to validate our initial impressions or our untrained intuitions. As a further corrective, I suggest: Reality is weirdly normal. It's "normal" in odd ways, by strange means, in surprising senses.
At the risk of vivisecting poetry, and maybe of stating the obvious, I'll point out that the maxims mean different things by "normal". In the first two, what's "normal" or "usual" is the universe taken on its own terms — the cosmos as it sees itself, or as an ideally calibrated demon would see it. In the third maxim, what's "normal" is the universe humanity perceives — though this still doesn't identify normality with what's believed or expected. Actually, it will take some philosophical work to articulate just what Egan's "normality" should amount to. I'll start with Copernicanism and reductionism, and then I'll revisit that question.
Everything is usual. Very nearly nothing is familiar.
Relative to the universe's laws, everything is par for the course. But relative to human standards of normality — what I'll call the familiar — barely anything that actually exists is par for the course. Nearly all events are senseless, bizarre, inhuman. But that's because our mapping hardware is adapted to a very specific environment; if anything's objectively weird, it's we, not the other denizens of Everythingland. That's much of why absurdity is a poor guide to probability.
Egan's Law reminds us that beneath our human surface weirdness lies a deeper regularity, a deeper unity with the rest of Nature. But to call this alien order 'regular' already assumes a shift in perspective from what a person off the street would initially think of as 'regular', to the hidden patterns of the very large and very small. We should prize the ability to shift between these points of view, while carefully avoiding conflating them.
Thinking like reality requires understanding reality. You can't just will yourself into becoming a high-fidelity map. If nothing else, you'll be too vulnerable to knowledge gaps. Pretending you think like quantum physics is even worse than rebelling against the physics for being too confusing. At least in the latter case you've noticed the disparity between your map and the territory.
To take pride in one's confusion at the quantum, rather than striving mightily to understand, is an epistemic sin. But to deny one's confusion at the quantum, to push it from one's mind and play-act at wisdom, is a graver sin. The goal isn't to do away with one's confusion; it's to do away with one's best reasons to be confused.
What adds up to the familiar needn't be familiar.
x + y + z + ... = Normality. But x ≠ Normality. Nor does y. In fact, even the complete quantum-mechanical description of the human world would not look normal, from a human perspective. But it would be a description of exactly the world we live in.
This is another reminder that surprise is a feature of maps, not of territories. Very different maps — a wave function scribbled in silicon, a belief, a gesture, a child's drawing — can represent the same territory. The drawing is normal (familiar), and the wave function isn't. But the referent of the two maps may well be the same. In asserting "quantum mechanics adds up to normality", we're really asserting that our maps of familiar objects, to the extent they're accurate, would co-refer with portions of the maps of an ideal Finished Science. They're two different languages, but faithful translation is possible.
The relationship between the quantum and the familiar can be thoroughly unfamiliar. The way in which the unintuitive fundamental truths yield an intuitive Middle World is not itself intuitive.
We could call this "emergence", if we wished to craft a territory predicate out of mapstuff. Otherwise, I suggest calling this predicate"wow-sometimes-I'm-not-very-good-at-keeping-track-of-lots-of-small-things-ence". Understanding the logical or mathematical implications of simple patterns is not humanity's specialty. Causal and part-whole relations are no exception to this rule.
It all adds up to the phenomenon.
Normality isn't common sense. Normality isn't our beliefs, expectations, assumptions, or strongest convictions. Normality, in the sense relevant to a true and binding "It all adds up to normality", is the phenomenon, the human world as it appears. (Not the human world as it is believed to appear. The human world as we actually encounter it.)
What's "the phenomenon"?
I'm going to be willfully cagey about that. What I really mean by "the phenomenon" is "whatever's going on". Or whatever's going on that we're cognitively accessing or representing. It needn't all add up to a world that makes my map true; but it will add up to a world that explains why my map looks the way it does.
Even saying that much risks overly restricting the shape explanations are allowed to take. Egan's Law can be used as a general constraint on successful explanations — they must account for the explanandum — but only if we treat the explanandum loosely enough to permit dissolutions and eliminations alongside run-of-the-mill reductions. It all adds up to an explanation of how our beliefs arose, though not necessarily a validation of them.
It's obvious that appearances can deceive us about underlying reality. But a variety of perceptual illusions demonstrate that experiences can also consistently mislead us about themselves. It all adds up to normality, but normality lies.
In some cases, we can't accurately describe our surface impressions until we've understood their underlying mechanism. Since so much of science is revisionary, we mustn't interpret Egan's Law in a way that unduly privileges first-pass descriptions. When data and theory conflict, it's sometimes more likely that the data has been misrepresented than that the theory is false.
How far does this openness to map revision extend? As far as the dynamics of one's brain allows. From the sidelines, it may appear to me that in principle a thinker should be able to have certainty about some propositions — for instance, 'an experience is occurring'. But when I actually find myself living through such a thought, I don't in fact experience infinite confidence. It remains physically possible for me to be persuaded otherwise.
This point ought to be a bit controversial, and more defense of it is needed. But I do insist on treating 'phenomenal experience' and 'phenomenon' as prima facie independent concepts. Egan's Law really is the law, whereas claims about some inerrant mode of reasoning or perception will at best qualify as well-supported hypotheses.
Egan's Law is not about saving conscious experience, or our theories, or our axioms, or our interpretations or descriptions of the phenomenon. It's about saving the phenomenon itself — the piece of the world in which we are, in fact, submerged. Egan's Law can be restated: The part of reality that (under its familiar description) puzzled or surprised us is identical to (or otherwise lawfully derivable from) the part of reality that (under its more fundamental description) explains it.
The human piece of the universe is of a piece with everything else. And the Everything Else gets explanatory priority. Of all our science's findings to date, that may well be the most startling.