(This post grew out of an old conversation with Wei Dai.)
Imagine a person sitting in a room, communicating with the outside world through a terminal. Further imagine that the person knows some secret fact (e.g. that the Moon landings were a hoax), but is absolutely committed to never revealing their knowledge of it in any way.
Can you, by observing the input-output behavior of the system, distinguish it from a person who doesn't know the secret, or knows some other secret instead?
Clearly the only reasonable answer is "no, not in general".
Now imagine a person in the same situation, claiming to possess some mental skill that's hard for you to verify (e.g. visualizing four-dimensional objects in their mind's eye). Can you, by observing the input-output behavior, distinguish it from someone who is lying about having the skill, but has a good grasp of four-dimensional math otherwise?
Again, clearly, the only reasonable answer is "not in general".
Now imagine a sealed box that behaves exactly like a human, dutifully saying things like "I'm conscious", "I experience red" and so on. Moreover, you know from trustworthy sources that the box was built by scanning a human brain, and then optimizing the resulting program to use less CPU and memory (preserving the same input-output behavior). Would you be willing to trust that the box is in fact conscious, and has the same internal experiences as the human brain it was created from?
A philosopher believing in computationalism would emphatically say yes. But considering the examples above, I would say I'm not sure! Not at all!
This seems like a good comment to illustrate, once again, your abuse of the idea of meaning.
There are two ways to understand this claim: 1) most words refer to things which happen also to be configurations of atoms and stuff. 2) most words mean certain configurations of atoms.
The first interpretation would be fairly sensible. In practice you are adopting the second interpretation. This second interpretation is utterly false.
Consider the word "chair." Does the word chair mean a configuration of atoms that has a particular shape that we happen to consider chairlike?
Suppose someone approached a chair in your house with an atomic microscope and discovered that it was not made of atoms, but was a continuous substance without any boundaries in it. Would you suddenly say that it was not a chair? Not at all. You would say "this chair is not made of atoms." This proves conclusively that the meaning of the word chair has nothing whatsoever to do with "a configuration of atoms." A chair is in fact a configuration of atoms; but this is a description of a thing, not a description of a word.
This could be true, if you mean this as a factual statement. It is utterly false, if you mean it as an explanation of the word "pain," which refers to a certain subjective experience. The word "pain" is not about brain activity in the same way that the word "chair" is not about atoms, as explained above.
I would just note that "are tables also chairs" has a definite answer, and is quite meaningful.
I would say that being a chair (according to the meaning of the word) is correlated with being made of atoms. It may be perfectly correlated in fact; there may be no chair which is not made of atoms, and it may be factually impossible to find or make a chair which is not. But this is a matter for empirical investigation; it is not a matter of the meaning of the word. The meaning of the word is quite open to the possibility that there is a chair not made of atoms. In the same, the meaning of the word "consciousness" refers to a subjective experience, not to any objective description, and consequently in principle the meaning of the word is open to application to a consciousness which does not satisfy any particular objective description, as long as the subjective experience is present.
I explicitly added "other stuff" to my sentence to avoid this sort of argument. I don't want or need to be tied to current understanding of physics here.
But even if I had only said "atoms", this would not be a problem. After seeing a chair that I previously thought was impossible, I can update what I mean by "chair". In the same, but more mundane way, I can go to a chair expo, see a radical new design of ch... (read more)