Further, why is that possibility tied to the word "soul", which carries all sorts of irrelevant baggage?
It's just the history of some words. It's not that important.
I experience red, and other qualia
People frequently claim this. One thing missing is a mechanism that gets us from an entity experiencing such fundamental mental states or qualia and that being's talking about it. Reductionism offers an account of why they say such things. If, broadly speaking, the reductionist explanation is true, then this isn't a phenomenon that is something to challenge reductionism with. If the reductionist account is not true, then how can these mental states cause people to talk about them? How does something not reducible to physics influence the world, physically? Is this concept better covered by a word other than "magic"? And if these mental states are partly the result of the environment, then the physical world is influencing them too.
I don't see why it's desirable to posit magic; if I type "I see a red marker" because I see a red marker, why hypothesize that the physical light, received by my eyes and sending signals to my brain, was magically transformed into pure mentality, enabling it to interact with ineffable consciousness, and then magicked back into physics to begin a new physical chain of processes that ends with my typing? Wouldn't I be just as justified in claiming that the process has interruptions at other points?
As the physical emanation "I see red people" may be caused by laws of how physical stuff interacts with other physical stuff, we don't guess it isn't caused by that, particularly as we can think of no coherent other way.
We are used to the good habit of not mistaking the limits of our imaginations for the limits of reality, so we won't say we know it impossible. However, if physics is a description of how stuff interacts with stuff, so I don't see how it's logically possible for stuff to do something ontologically indescribable even as randomness. Interactions can either be according to a pattern, or not, and we have the handy description "not in a pattern, indescribable by compression" to pair with "in a pattern, describable by compression", and how matter interacts with matter ought to fall under one of those. So apparent or even actual random "deviation from the laws of physics" would not be unduly troubling. Systematic deviation from the laws of physics, isn't.
Do you think your position is captured by the statement, "matter sometimes interacts with matter neither a) in a pattern according to rules, nor b) not in a pattern, in deviation from rules"?
Photons go into eyes, people react predictably to them (though this is a crude example, too macro)...something bookended by the laws of physics has no warrant to call itself outside of physics, if the output is predictable from the input. That's English, as it's used for communication, no personal definitions allowed.
if I type "I see a red marker" because I see a red marker, why hypothesize that the physical light, received by my eyes and sending signals to my brain, was magically transformed into pure mentality, enabling it to interact with ineffable consciousness
There's a fascinating psychological phenomena called "blindsight" where the conscious mind doesn't register vision - the person is genuinely convinced they are blind, and they cannot verbally describe anything. However, their automatic reflexes will still navigate the world just fine. I...
Anyone who does not believe mental states are ontologically fundamental - ie anyone who denies the reality of something like a soul - has two choices about where to go next. They can try reducing mental states to smaller components, or they can stop talking about them entirely.
In a utility-maximizing AI, mental states can be reduced to smaller components. The AI will have goals, and those goals, upon closer examination, will be lines in a computer program.
But in the blue-minimizing robot, its "goal" isn't even a line in its program. There's nothing that looks remotely like a goal in its programming, and goals appear only when you make rough generalizations from its behavior in limited cases.
Philosophers are still very much arguing about whether this applies to humans; the two schools call themselves reductionists and eliminativists (with a third school of wishy-washy half-and-half people calling themselves revisionists). Reductionists want to reduce things like goals and preferences to the appropriate neurons in the brain; eliminativists want to prove that humans, like the blue-minimizing robot, don't have anything of the sort until you start looking at high level abstractions.
I took a similar tack asking ksvanhorn's question in yesterday's post - how can you get a more accurate picture of what your true preferences are? I said:
A more practical example: when people discuss cryonics or anti-aging, the following argument usually comes up in one form or another: if you were in a burning building, you would try pretty hard to get out. Therefore, you must strongly dislike death and want to avoid it. But if you strongly dislike death and want to avoid it, you must be lying when you say you accept death as a natural part of life and think it's crass and selfish to try to cheat the Reaper. And therefore your reluctance to sign up for cryonics violates your own revealed preferences! You must just be trying to signal conformity or something.
The problem is that not signing up for cryonics is also a "revealed preference". "You wouldn't sign up for cryonics, which means you don't really fear death so much, so why bother running from a burning building?" is an equally good argument, although no one except maybe Marcus Aurelius would take it seriously.
Both these arguments assume that somewhere, deep down, there's a utility function with a single term for "death" in it, and all decisions just call upon this particular level of death or anti-death preference.
More explanatory of the way people actually behave is that there's no unified preference for or against death, but rather a set of behaviors. Being in a burning building activates fleeing behavior; contemplating death from old age does not activate cryonics-buying behavior. People guess at their opinions about death by analyzing these behaviors, usually with a bit of signalling thrown in. If they desire consistency - and most people do - maybe they'll change some of their other behaviors to conform to their hypothesized opinion.
One more example. I've previously brought up the case of a rationalist who knows there's no such thing as ghosts, but is still uncomfortable in a haunted house. So does he believe in ghosts or not? If you insist on there being a variable somewhere in his head marked $belief_in_ghosts = (0,1) then it's going to be pretty mysterious when that variable looks like zero when he's talking to the Skeptics Association, and one when he's running away from a creaky staircase at midnight.
But it's not at all mysterious that the thought "I don't believe in ghosts" gets reinforced because it makes him feel intelligent and modern, and staying around a creaky staircase at midnight gets punished because it makes him afraid.
Behaviorism was one of the first and most successful eliminationist theories. I've so far ignored the most modern and exciting eliminationist theory, connectionism, because it involves a lot of math and is very hard to process on an intuitive level. In the next post, I want to try to explain the very basics of connectionism, why it's so exciting, and why it helps justify discussion of behaviorist principles.