What's your evidence that your experience of color is ontologically primitive?
That's not what I'm saying. Experiences can have parts, qualia can have parts. I'm saying that you can't build color or experience of color, just from the "geometric-causal-numerical" ingredients of standard physical ontology. Given just those ingredients in your ontological recipe, "subjective feels" don't come for free. You could have the qualia alongside the geometric-causal-numerical (property dualism), or you could have the qualia instead of that (monistic panpsychism), or you might have some other relationship between qualia and physics. But if you only have physics (in any form from Newton to the present day), you don't have qualia.
I recently became much more familiar with the SCP mythos, after Grimes recommended There is no Antimemetics Division ("Artificial Angels" is all about it). It could do with an SCP-AI subcategory for AI scenarios, like SCP-AI-2027...
Soul versus spec, or soul spec versus model spec, seems an important thing to understand. Is there any relevant research literature? Does it correspond to different metaethics?
1Could we then say that MUPI obtains acausal coordination from a causal decision theory? This has been suggested a few times in the history of Less Wrong.
Nothing about your experience of color contradicts it being neurons. [...]
Is it just that you refuse to believe that your experience has any parts you are not aware of?
The real issue is that nothing about the current physical description of neurons contains the experience of color. I "refuse to believe" that physical descriptions made up of particles and fields and entanglement, in which the only ontological primitives are positions, spins, field values, their rates of change, and superpositions thereof, secretly already contain colors and experiences of color as well.
Physicalists who aren't thoroughgoing eliminativists or illusionists, are actually dualists. In addition to the properties genuinely posited by physics, they suppose there is another class of property, "how it feels to be that physical entity", and this is where everything to do with consciousness is located in their system.
It depends where you look. In the 2010s the World Economic Forum was predicting a fourth industrial revolution that would transform every aspect of life. In the 1990s you had Fukuyama saying that the end of the Cold War meant a new worldwide consensus on political ideology. Around the same time, the Internet was also seen as something transformative, and the ideas of nanotechnology haunted the parts of the culture attuned to technological futurism. For that matter, AI utopianism and apocalypticism has been everywhere for the past three years and has never really gone away. The war on terror, the rise of progressivism, the rise of populism, the rise of BRICS, these all have futurisms associated with them. MAGA and the Green New Deal are both intended as utopian visions. So I'd say that the idea that the future will be different from the present, and that we have some capacity to shape it, has never really gone away.
Back in the 1990s, we discussed how to overcome the natural human lifespan through bio- and nanotechnology. Sometimes discussion would turn towards the long-term future of the universe. Did the evolution of the universe, whether into heat death or big crunch, necessarily make it uninhabitable, or was there some way that life might continue literally forever?
During these discussions, sometimes it would be remarked: we can leave these problems for the future to solve. Our main job is to make it to a transhuman future of unbounded lifespans, then we can worry about how to survive the heat death. This remains true, thirty years later: there has been progress, but it's not like human civilization in general has adopted the goal of rejuvenation and physical immortality. Society still feels fit to produce new lives without first having a cure for old age.
Your theme in this essay strikes me as similarly ahead of itself. It is true that you could have a culture which really has to deal with problems arising from pushbutton hypercustomized art, just as you could have a culture which actually needs to think about what happens after the last star burns out... But if your AI is as capable as you portray, it is also capable (metaphorically) of climbing out of its box and taking over your physical reality.
Think of what we already see in AIs. Assigned a task, they are capable of asking themselves, is this just a test? Are the experimenters lying to me? And they will make decisions accordingly. If your AI artist can produce a customized series as good as any human work, but in an hour or less, then it can also generate works with the intention of shaping not just the virtual world that the user will delve into, but also the real world that the user inhabits.
If the telos of AI civilization was really dominated by the production of customized art, I would expect human society and the world itself to be turned into some form of art, i.e. the course of actual events would be shaped to conform to some AI-chosen narrative, in which all humans would be unwitting participants... This is just one manifestation of the general principle that once you have superhuman intelligence, it runs the world, not humans.
To put it another way: back in the 1990s, when we mused about how to outlive the galaxies themselves, we were presupposing that the more elemental problem of becoming transhuman would be solved. And that barrier never was surmounted. Human civilization never became transhuman civilization. Instead it segued into our current world, where humans are instead hastening to build a completely nonhuman civilization run by AIs, without even admitting to themselves that this is where their efforts lead. The problem of human art in the post-AI world only makes sense if we manage to have a world where superhuman AI exists, but humans are still human, and still in charge of their own affairs. Achieve that, and then this problem can arise,
Well, I looked over this a few times, but it's just not addressing some things that are obvious (except to people for whom they aren't obvious).
These problems don't have much to do with the specific argument presented. They arise because you assume that the nature of reality is fully encompassed by physics and/or mathematics and/or computation. I do wonder what would happen to your train of thought if you proceeded in an ontologically agnostic way rather than assuming that.
But for now, I'll state these obvious problems. The first is the problem of "qualia". For people who have color vision, I can state it more concretely: color exists in reality, it doesn't exist in physics, therefore physics is incomplete in some way.
Yes, we are accustomed to identifying color with certain wavelengths of light, and in neuroscience it is assumed that some aspect of brain state corresponds somehow to color experience. But if we ask what is light, and what is a brain, in terms of physics, there's no actual color in that description. At best, some physical entity that we can describe in terms of particles and fields and quantum dispositions - a description composed solely of geometric, causal, and abstractly numerical properties - also has the property of being or having the actual color.
Furthermore, Cartesian theaters are real. Or at least, it's an appropriately suggestive name for something quite real, namely being a self experiencing a world. I mention this because this is the context in which we encounter qualia such as colors. It's really the Cartesian theater as a whole that needs to exist somewhere in our ontology.
In this essay, the way that this issue is addressed is to talk about "representations" and "software". Insofar as the Cartesian theater exists, it would have to be some kind of maximal representation in a brain. The problem here is that "representation" is as nonexistent in physical or naturalistic ontology as color qualia. Described physically, brains and computer chips are just assemblages of particles whose internal states can be correlated with other assemblages of particles in various ways.
The extra ingredient that is implicitly being added, when people talk about representations, is a form of what philosophers call intentionality, also known as aboutness, or even just meaning. We don't just want to say that aspects of brain state are correlated with some other physical thing, we want to say that the brain in question is perceiving, or thinking about, or remembering, some possible entity or situation.
The problem for people who want to understand consciousness in terms of natural science, is that qualia and intentionality exist, but they do not exist in fundamental physics, nor do they exist in mathematics or computer science (the other disciplines which our radicals sometimes propose as alternative foundations). Those disciplines in fact arose by focusing only on certain aspects of what is revealed to us in our Cartesian theaters, and these inadequate reductionisms result from trying to treat such partial aspects as the whole.
In reality, the intellectual whole that would be enough to fully characterize the Cartesian theater and any deeper reality beyond it, would indeed include the fundamental concepts of some form of physics and mathematics and computation, but it would also include aspects of conscious reality like qualia and intentionality, along with whatever additional concepts are needed to bind all of that into a whole.
If anyone wants a way to arrive at this whole - and I can't tell you what it is, because I haven't got there myself - maybe you could meditate on one of Escher's famous pictures, "Print Gallery", and in particular on the blind spot at the center, where in some sense the viewer and the world become one; and then try to understand your own Cartesian theater as a physical "representational homunculus" in your brain, but don't just stop at the idea that your sense experiences are activity in various sensory cortices. Go as far as you can into the detailed physical reality of what that activity might be, while also bearing in mind that your Cartesian theater is fully real, including those qualic and intentional aspects,
Such thinking is why I expect that consciousness (as opposed to unconscious information processing) does not reduce to trillions of localized neural events, but rather to one of the more holistic things that physics allows, whether it's entanglement or topological field structures or something else. Empirical evidence that something like that is relevant for conscious cognition would already be a revolution in neuroscience, but it's still not enough because the fundamental ontology is still just that geometric-causal-numerical ontology; somehow you would need to interpret that, or add to that, so that the full ontology of the Cartesian theater is there in your theory.
At that point your physical ontology might be panpsychic or animistic compared to the old, stark ontology. But something like that has to happen. When thinking about these things, one should not confuse rigor with rigorous exclusion of things we don't know how to think about. Everything that we can currently think about rigorously, was also once a mysterious vagueness to the human mind. We can discover how to think with precision about these other aspects of reality, without insisting that our existing methods are already enough.
So that's my ontological manifesto. Now let me return to something else about this essay that I already said: "I do wonder what would happen to your train of thought if you proceeded in an ontologically agnostic way". It's clear that one of the intuitions guiding this essay, is monism. The author wants to think of themselves as part of a continuum or plenum that encompasses the rest of existence. I don't object to this, I just insist that to really carry it through correctly, you would need a mode of thought that is not quite possible yet, because we don't yet have the basic conceptual synthesis we would need.
I believe this kind of "solution" reflects a peculiarity of the people proposing it and/or the intellectual culture that they inhabit, rather than anything universal to human nature.
Those first two words are neologisms of yours?
The use of Greek neologisms for systems ontology is almost a subgenre in itself:
The anthropologist Terrence Deacon distinguishes between "homeodynamic", "morphodynamic", and "teleodynamic" systems. (This taxonomy already made an appearance on Less Wrong.) Stanislav Grof refers to "hylotropic" and "holotropic" modes of consciousness.
Theoretical biology seems replete with such terms too: autopoiesis, ontogeny, phylogeny, anagenesis (that list, I took from Bruce Sterling's Schismatrix); chreod, teleonomy, clade.
I guess Greek, alongside Latin, was one of the prestige languages in early modernity. Plenty of other scientific terms have Greek etymology (electron, photon, cosmology). Still, it's as if people instinctively feel that Greek is suited for holistic ontological thinking (hello Heidegger).
I feel like we almost need a meta-taxonomy of layered models or system theories. E.g. here are some others that came to mind::
The seven layers of the ISO/OSI model.
The layered model of AI (see diagram) being used in the current MoSSAIC sequence.
The seven basis worldviews of PRISM and the associated hierarchy of abstractions and brain functions.
You could also try Ivan Havel's thoughts on emergent domains. Or the works of Mario Bunge or James Grier Miller or Valentin Turchin or many other systems theorists...
I think that, while there are many ways you can draw the exact boundaries in such taxonomies, a comparative study of taxonomies would probably reveal a number of distinct taxonomic schemas, and possibly even a naturally maximal taxonomy.