Appreciate the time and care you put into your response -- I find it helpful to work through this a step at a time.
I would similarly say that I can't actually write a program that calculates the square root of 200, because programs can't do any such thing. Rather, it is the program executor that performs the calculation, making use of the program.
One of the big achievements of programming language research, in the 1960s and 1970s, was to let us make precise assertions about the meaning of a program, without any discussion of running it. We can say "this program emits the number that's the best possible approximation to the square root of its input", without running the program, without even having any physical system that can run the program. There are some non-trivial and important properties of programs, as separate from a particular physical execution.
Not all properties are of this form. "This program is currently paged out" is a property of the particular concrete execution, not of an abstract execution history or of the program itself.
I assume there are properties that human minds possess that are of both sorts. For example, I suspect "this is a sad brain" is a meaningful assertion even if the brain is frozen or if we're looking at one time-step of a simulation. However, I think consciousness, the way we usually use the word, is the second sort of property.
And the reason the distinction matters is that I want a sense what aspects of minds are relevant when discussing, e.g., an upload that isn't currently running or a synthesized AGI.
Why do you believe this? It doesn't seem to follow from the above: even if consciousness is a property of program-executors rather than of the programs themselves, it doesn't follow that only biological humans can execute the appropriate programs.
I agree with this. I didn't mean to say "nothing but a biological human can ever be conscious" -- just that "not all physical embodiments would have the right property." I expect that uploaded-human-attached-to-robot would be usefully described as conscious, and that a paper-and-pencil simulation would NOT.
We can say "this program emits the number that's the best possible approximation to the square root of its input", without running the program, without even having any physical system that can run the program. There are some non-trivial and important properties of programs, as separate from a particular physical execution. Not all properties are of this form. "This program is currently paged out" is a property of the particular concrete execution, not of an abstract execution history or of the program itself.
I agree with this as far ...
In Zombies! Zombies? Eliezer mentions that one aspect of consciousness is that it can causally affect the real world, e.g. cause you to say "I feel conscious right now", or result in me typing out these words.
Even if a generally accepted mechanism of consciousness has not been found yet are there any tentative explanations for this "can change world" property? Googling around I was unable to find anything (although Zombies are certainly popular).
I had an idea of how this might work, but just wanted to see if it was worth the effort of writing.