(This post grew out of an old conversation with Wei Dai.)
Imagine a person sitting in a room, communicating with the outside world through a terminal. Further imagine that the person knows some secret fact (e.g. that the Moon landings were a hoax), but is absolutely committed to never revealing their knowledge of it in any way.
Can you, by observing the input-output behavior of the system, distinguish it from a person who doesn't know the secret, or knows some other secret instead?
Clearly the only reasonable answer is "no, not in general".
Now imagine a person in the same situation, claiming to possess some mental skill that's hard for you to verify (e.g. visualizing four-dimensional objects in their mind's eye). Can you, by observing the input-output behavior, distinguish it from someone who is lying about having the skill, but has a good grasp of four-dimensional math otherwise?
Again, clearly, the only reasonable answer is "not in general".
Now imagine a sealed box that behaves exactly like a human, dutifully saying things like "I'm conscious", "I experience red" and so on. Moreover, you know from trustworthy sources that the box was built by scanning a human brain, and then optimizing the resulting program to use less CPU and memory (preserving the same input-output behavior). Would you be willing to trust that the box is in fact conscious, and has the same internal experiences as the human brain it was created from?
A philosopher believing in computationalism would emphatically say yes. But considering the examples above, I would say I'm not sure! Not at all!
Again you confirm that you don't understand what the game taboo is (rationalist or not). "Yellow bent fruit" is not a synonym of "banana".
My criticism is that this description obviously matches a roomba. It can definitely perceive walls (it can become aware of them through sensors) and I don't see why this perception wouldn't be personal (it happens completely withing the roomba), although I suspect that this word might mean something special for you. Now, as I say this, I assume that you don't consider roomba conscious. If you do, then maybe I have not criticisms.
Is that the criticism you anticipated?
I don't know what sort of scale of incompleteness you have. Actually, there could be an agent who can recognize bananas exactly as well as you, without actually knowing whether they grow on plants or are made in factories. A banana has many distinctive properties, growing on plants is not the most important one.
How does it feel? It feels bad, of course, but what else?
"Perception" includes subjectively noticing something, not just being affected by it. I don't think that a roomba notices or perceives anything.
Among other things, it usually feels a bit like heat. Why do you ask?