Need to use complexity theory to try and see how powerful the super-intelligence has to be. It may end up that it will need to be substantially bigger than Jupiter Brain (cubed, squared, to 100th power?) to construct anything relevant to self preservation, from such a coarse data set. As intuition pump, and I said that before, a computing system that is to mankind as mankind is to 1 amoeba should be expected to predict the weather for about two times longer than mankind can - that's given perfect information - given limited information the gain may be quite small and entirely unimportant. For the coarse brain scans to detailed brain state, one has to somehow run simulation in reverse (or worse yet, bruteforce the state), which I think explodes even worse than forward butterfly effect. I'd say we don't know enough about brain to be able to tell what it takes to do this kind of thing, but what we do know about chaotic non-linear systems in general, does not inspire optimism.
The jump that if might be possible in principle, it would therefore be doable by super-intelligence, only works if you think theological thoughts about AI.
It may end up that it will need to be substantially bigger than Jupiter Brain (cubed, squared, to 100th power?) to construct anything relevant to self preservation, from such a coarse data set.
Yeah, I'm aware of this and said as much in a previous comment. I appreciate you giving a more detailed explanation, but wish you hadn't also included a sentence implying that I "think theological thoughts about AI".
As you say, this needs to be investigated more using complexity theory, but my guess is that we won't reach any definitive conclusions due ...
Paul Christiano recently suggested that we can use neuroimaging to form a complete mathematical characterization of a human brain, which a sufficiently powerful superintelligence would be able to reconstruct into a working mind, and the neuroimaging part is already possible today, or close to being possible.
Paul was using this idea as part of an FAI design proposal, but I'm highlighting it here since it seems to have independent value as an alternative or supplement to cryonics. That is, instead of (or in addition to) trying to get your body to be frozen and then preserved in liquid nitrogen after you die, you periodically take neuroimaging scans of your brain and save them to multiple backup locations (1010 bits is only about 1 gigabyte), in the hope that a friendly AI or posthuman will eventually use the scans to reconstruct your mind.
Are there any neuroimaging experts around who can tell us how feasible this really is, and how much such a scan might cost, now or in the near future?
ETA: Given the presence of thermal noise and the fact that a set of neuroimaging data may contain redundant or irrelevant information, 1010 bits ought to be regarded as just a rough lower bound on how much data needs to be collected and stored. Thanks to commenters who pointed this out.