(This post grew out of an old conversation with Wei Dai.)
Imagine a person sitting in a room, communicating with the outside world through a terminal. Further imagine that the person knows some secret fact (e.g. that the Moon landings were a hoax), but is absolutely committed to never revealing their knowledge of it in any way.
Can you, by observing the input-output behavior of the system, distinguish it from a person who doesn't know the secret, or knows some other secret instead?
Clearly the only reasonable answer is "no, not in general".
Now imagine a person in the same situation, claiming to possess some mental skill that's hard for you to verify (e.g. visualizing four-dimensional objects in their mind's eye). Can you, by observing the input-output behavior, distinguish it from someone who is lying about having the skill, but has a good grasp of four-dimensional math otherwise?
Again, clearly, the only reasonable answer is "not in general".
Now imagine a sealed box that behaves exactly like a human, dutifully saying things like "I'm conscious", "I experience red" and so on. Moreover, you know from trustworthy sources that the box was built by scanning a human brain, and then optimizing the resulting program to use less CPU and memory (preserving the same input-output behavior). Would you be willing to trust that the box is in fact conscious, and has the same internal experiences as the human brain it was created from?
A philosopher believing in computationalism would emphatically say yes. But considering the examples above, I would say I'm not sure! Not at all!
Sure, but what does that have to do with anything? Does "objective" mean "well understood" to you?
There are multiple representations of pain the same way that there are multiple representations of chair.
It is ridiculous how much of this debate is about the basic problem of classification, rather than anything to do with brains. Flawed reasoning starts with a postulate that "Pain" exists and then asks, what physical states correspond to it. And when told that "pain is the activity in region X", it somehow feels that "activity in Y could also be described as pain", is a counter argument. Good reasoning starts with noticing that people say "ouch" when they stub their toes, or that subbing a toe has a very distinct feeling, and then asks, what causes/predicts these actions/differences, and then wonders, how could we best classify these.
Calling my reasoning, even if not fully formal, "subjective intuitions" seems rude. I'm not sure if there is some point you're trying to express with that.
Not sure where you see me talking about intelligence. But intelligence is far more well defined and measurable than consciousness. Multiple realizability has nothing to do with that.
We do, on the other hand, know subjecively what pain feels like..
That's not the point. The point is that if we have words referring to subjective sensations, like "purple" and "bitter", we can distinguish them subjectively. But if we discard out subjective insight into them, as you are proposing, and replace ... (read more)