Perhaps I might not know the difference, but I am not the only observer here. Would the people replaced know the difference?
Well, presumably, the original people who were replace would indeed know the difference, as they watch helplessly from within the bubbling storage tanks where the evil aliens / wizards / whomever had put them prior to replacing them with the automatons.
The more interesting question is, would the automatons believe that they were the originals ? My claim is that, in order to emulate the originals perfectly with 100% accuracy -- which is what this thought experiment requires -- the automatons would have to believe that they were, in fact, original; and thus they would have to be conscious.
You could probably say, "ah-hah, sure the automatons may believe that they are the originals, but they're wrong ! The originals are back on the mothership inside the storage vats !" This doesn't sound like a very fruitful objection to me, however, since it doesn't help you prove that the automatons are not conscious -- merely that they aren't composed of the same atoms as some other conscious beings (the ones inside the vats). So what, everyone is made of different atoms, you and I included.
I encounter many intelligent people (not usually LWers, though) who say that despite our recent scientific advances, human consciousness remains a mystery and currently intractable to science. This is wrong. Empirically distinguishable theories of consciousness have been around for at least 15 years, and the data are beginning to favor some theories over others. For a recent example, see this August 2011 article from Lau & Rosenthal in Trends in Cognitive Sciences, one of my favorite journals. (Review articles, yay!)
Abstract: