Permutation City is an awesome novel that was written in 1994. Even if the author, Greg Egan, used a caricature of this community as a bad guy in a more recent novel, his work is still a major influence on a lot of people around these parts who have read it. It dissolves so many questions around uploading and simulation that it's hard for someone who has read the book to talk about simulationist metaphysics without wanting to reference the novel... but doing that runs into constraints imposed by spoiler etiquette.
So go read Permutation City if you haven't read it already because it's philosophically important and a reasonably fun read.
In the meantime, if you haven't then you should also read A Fire Upon The Deep by Vernor Vinge (of "singularity" coining fame) and then read Eliezer's fan fic The Finale of the Ultimate Meta Mega Crossover which references both of them in interesting ways to make substantive philosophical points and doesn't take too long to read.
In the comments below there will be discussion that has spoilers for all three works.
With thanks to HonoreDB, yes, the structure must have a source. And also, as with the Chinese Room, there is a sleight-of-concept going on where something that looks like a human (Searle's paper manipulator and Egan's Durham) is not the actual "brains" of the system (which are really the symbol manipulation rules with Searle, or the dust/translator combination with Egan) that we're truly analyzing.
I agree with you that if there is not stateful process to worry about, but merely the instantiation of a trivially predictable "movie-like image of counting the number 8" then the dust hypothesis might make sense... but I suspect that very few of the phenomena that we care about are like this, nor do I think that such phenomena are going to be interesting to us post-uploading. I can't fully and succinctly explain the intuition I have here, but the core of the objection it is connected to reversible computing, computational irreducibility, and their relation to entropy and hence the expenditure of energy.
From these inspirations, it seems likely to me that "the dust" can only be said to contain structure that I care about if the energy used to identify/extract/observe that structure is less than what would have been required for an optimally efficient computational process to invent that structure from scratch. Thus, there is probably a mechanically rigorous way to distinguish between hearing a sound versus imagining that same sound, that grows out of the way that hearing requires fewer joules than imagining. If a dust interpretation system requires too much energy, I would guess either than it is mediating a scientifically astonishing real signal (in a grossly inefficent way)... or you're dealing with a sort of clever hans effect where the interpretation system plus its battery is the real source of the "detected patterns", not the dust.
Using this vocabulary to speak directly to the issues raised in the article on strong substrate independence, the problem with other quantum narratives (or the bits of platospace mathematicians spend their time "exploring") is that the laws of physical computation seem such that our brains can never hear anything from those "places", our brains can only imagine them.
Yes, that seems like a reasonable way to state more rigorously the distinction between systems I might care about and systems I categorically don't care about.
Though, thinking about Permutation City a bit more... we, as readers of the novel, have access to the frame in which Peer's consciousness manifests. The residents of PC don't have access to it; Peer is no easier for them to access than the infinite number of other consciousnesses they could in principle "detect" within their architecture.
So we care about Peer, and they don't, and neither ... (read more)