Okay, I think my bright dilettante answer to this is the following: The key is what allows you to prove that the FHE is conscious. It is not, itself, the FHE's consciousness, which is probably still silently running (although that can no longer be proven). Proof of consciousness and consciousness are different things, although they clearly are related, and something may or may not have proved it's consciousness in the past before losing its ability to do so in the future.
I used the following thought experiment while thinking about this:
The thought experiment that occurs to me is simply looking at someone's brain while they do something stereotypically consciousness-indicating! An outside observer watching a brain might say, "oh, that just looks like a wet, wobbly lump of meat, I can't even remotely tell how it's supposed to operate just by looking at it, why would I think it's generating consciousness?" The analogue to FHE here would be a lack of knowledge about neuroscience & such.
Yet another exceptionally interesting blog post by Scott Aaronson, describing his talk at the Quantum Foundations of a Classical Universe workshop, videos of which should be posted soon. Despite the disclaimer "My talk is for entertainment purposes only; it should not be taken seriously by anyone", it raises several serious and semi-serious points about the nature of conscious experience and related paradoxes, which are generally overlooked by the philosophers, including Eliezer, because they have no relevant CS/QC expertise. For example:
Scott also suggests a model of consciousness which sort-of resolves the issues of cloning, identity and such, by introducing what he calls a "digital abstraction layer" (again, read the blog post to understand what he means by that). Our brains might be lacking such a layer and so be "fundamentally unclonable".
Another interesting observation is that you never actually kill the cat in the Schroedinger's cat experiment, for a reasonable definition of "kill".
There are several more mind-blowing insights in this "entertainment purposes" post/talk, related to the existence of p-zombies, consciousness of Boltzmann brains, the observed large-scale structure of the Universe and the "reality" of Tegmark IV.
I certainly got the humbling experience that Scott is the level above mine, and I would like to know if other people did, too.
Finally, the standard bright dilettante caveat applies: if you think up a quick objection to what an expert in the area argues, and you yourself are not such an expert, the odds are extremely heavy that this objection is either silly or has been considered and addressed by the expert already.