On page 208-210 of The Rediscovery of the Mind, Searle writes:
On the standard textbook definition of computation, it is hard to see how to avoid the following results:
For any object there is some description of that object such that under that description the object is a digital computer.
For any program and for any sufficiently complex object, there is some description of the object under which it is implementing the program. Thus for example the wall behind my back is right now implementing the Wordstar program, because there is some pattern of molecule movements that is isomorphic with the formal structure of Wordstar. But if the wall is implementing Wordstar, then if it is a big enough wall it is implementing any program, including any program implemented in the brain. [...]
I do not think that the problem of universal realizability is a serious one. I think it is possible to block the result of universal realizability by tightening up our definition of computation. Certainly we ought to respect the fact that programmers and engineers regard it as a quirk of Turing's original definitions and not as a real feature of computation. Unpublished works by Brian Smith, Vinod Goel, and John Batali all suggest that a more realistic definition of computation will emphasize such features as the causal relations among program states, programmability and controllability of the mechanism, and situatedness in the real world. All these will produce the result that the pattern is not enough. There must be a causal structure sufficient to warrant counterfactuals. But these further restrictions on the definition of computation are no help in the present discussion because the really deep problem is that syntax is essentially an observer-relative notion. The multiple realizability of computationally equivalent processes in different physical media is not just a sign that the processes are abstract, but that they are not intrinsic to the system at all. They depend on an interpretation from outside. We were looking for some facts of the matter that would make brain processes computational; but given the way we have defined computation, there never could be any such facts of the matter. We can't, on the one hand, say that anything is a digital computer if we can assign a syntax to it, and then suppose there is a factual question intrinsic to its physical operation whether or not a natural system such as the brain is a digital computer.
And if the word "syntax" seems puzzling, the same point can be stated without it. That is, someone might claim that the notions of "syntax" and "symbols" are just a manner of speaking and that what we are really interested in is the existence of systems with discrete physical phenomena and state transitions between them. On this view, we don't really need 0's and l's; they are just a convenient shorthand. But, I believe, this move is no help. A physical state of a system is a computational state only relative to the assignment to that state of some computational role, function, or interpretation. The same problem arises without 0's and l's because notions such as computation, algorithm, and program do not name intrinsic physical features of systems. Computational states are not discovered within the physics, they are assigned to the physics.
I've been thinking about ethics and brain emulations for a while and now have realized I am confused. Here are five scenarios. I am pretty sure the first is morally problematic, and pretty sure the last is completely innocuous. But I can't find a clean way to partition the intermediate cases.
A) We grab John Smith off the street, scan his brain, torture him, and then by some means, restore him to a mental and physical state as though the torture never happened.
B) We scan John Smith's brain, and then run a detailed simulation of the brain being tortured for ten seconds, and over again. If we attached appropriate hardware to the appropriate simulated neurons, we would hear the simulation screaming.
C) We store, on disk, each timestep of the simulation in scenario B. Then we sequentially load each timestep into memory, and overwrite it.
D) The same as C, except that each timestep is encrypted with a secure symmetric cipher, say, AES. The key used for encryption has been lost. (Edit: The key length is much smaller than the size of the stored state and there's only one possible valid decryption.)
E) The same as D, except we have encrypted each timestep with a one time pad.
I take for granted that scenario A is bad: one oughtn't be inflicting pain, even if there's no permanent record or consequence of the pain. And I can't think of any moral reason to distinguish a supercomputer simulation of a brain from the traditional implementation made of neurons and synapses. So that says that B should be equally immoral.
Scenario C is just B with an implementation tweak -- instead of _calculating_ each subsequent step, we're just playing it back from storage. The simulated brain has the same sequence of states as in B and the same outputs.
Scenario D is just C with a different data format.
Scenario E is just D with a different encryption.
Now here I am confused. Scenario E is just repeatedly writing random bytes to memory. This cannot possibly have any moral significance! D and E are indistinguishable to any practical algorithm. (By definition, secure encryption produces bytes that "look random" to any adversary that doesn't know the key).
Either torture in case A is actually not immoral or two of these adjacent scenarios are morally distinct. But none of those options seem appealing. I don't see a simple clean way to resolve the paradox here. Thoughts?
As an aside: Scenarios C,D and E aren't so far beyond current technology as you might expect. Wikipedia tells me that the brain has ~120 trillion synapses. Most of the storage cost will be the per-timestep data, not the underlying topology. If we need one byte per synapse per timestep, that's 120TB/timestep. If we have a timestep every millisecond, that's 120 PB/second. That's a lot of data, but it's not unthinkably beyond what's commercially available today, So this isn't a Chinese-Room case where the premise can't possibly be realized, physically.