(This post grew out of an old conversation with Wei Dai.)
Imagine a person sitting in a room, communicating with the outside world through a terminal. Further imagine that the person knows some secret fact (e.g. that the Moon landings were a hoax), but is absolutely committed to never revealing their knowledge of it in any way.
Can you, by observing the input-output behavior of the system, distinguish it from a person who doesn't know the secret, or knows some other secret instead?
Clearly the only reasonable answer is "no, not in general".
Now imagine a person in the same situation, claiming to possess some mental skill that's hard for you to verify (e.g. visualizing four-dimensional objects in their mind's eye). Can you, by observing the input-output behavior, distinguish it from someone who is lying about having the skill, but has a good grasp of four-dimensional math otherwise?
Again, clearly, the only reasonable answer is "not in general".
Now imagine a sealed box that behaves exactly like a human, dutifully saying things like "I'm conscious", "I experience red" and so on. Moreover, you know from trustworthy sources that the box was built by scanning a human brain, and then optimizing the resulting program to use less CPU and memory (preserving the same input-output behavior). Would you be willing to trust that the box is in fact conscious, and has the same internal experiences as the human brain it was created from?
A philosopher believing in computationalism would emphatically say yes. But considering the examples above, I would say I'm not sure! Not at all!
If you don't doubt you are conscious, I'm not sure why you would need to figure out whether you are conscious - it seems to me that you already know based on direct experience.
That these things are difficult to describe is not in dispute; that is what I meant when I said "consciousness seems to defy precise definitions". But, we can still talk about them as there seems to be a shared understanding of the concepts.
One need not have a precise definition of a thing to discuss and believe in that thing or to know that one is effected by that thing. For example, consider someone unschooled in physics beyond a grade-school level. He/she knows about gravity, knows that he/she is subject to the effects of gravity and can make (qualitative) predictions about the effects of gravity, even if he/she cannot say whether gravity is a force, a warping of spacetime, both of these things, neither of these things, or even understand the distinction. Similarly, there is enough of a common understanding of consciousness and first person experiences for a person to be confident that she/he is conscious and has first person experiences.
I do agree that the lack of precise definition (and, more importantly, the lack of measurable or externally observable manifestations) makes it impossible (at the present) for an observer to know whether some other entity is conscious.