How is sleep, unconsciousness, deep anesthesia any different, though?...But further, why is continuity important?
Those two questions are two sides of the same coin to me. Those examples preserve continuity in the form of synapses and other neural connections. In none of those cases does the brain actually stop running, just the consciousness program. You can't just pull out someone's heart while they're anesthetized--if the brain's cells die from lack of fuel, you're destroying the hardware that the consciousness program needs to reboot from.
If intelligence can be simulated on a computer, and it seems likely that intelligence sophisticated enough to ponder it's own consciousness probably really is conscious, why would a reboot have any effect on its identity?
Assuming that you have programmed it to care about its own consciousness, not just to ponder it, the first boot would die, and the reboot would wake up thinking it was the first boot.
...cryonics is probably unnecessary if I can instead do a molecule-level brain-image upload before death (assuming that turns out to be possible). But if that's so, don't we also need to reject the idea of a personal future?
When you upload your brain-image, please make the most of your life after that, because it would be the same as with the computer. You will die in fear and loneliness, and your copy will wake up convinced he is you. (That would make a great fortune-cookie message!) In both cryonic preservation, and brain upload, the original quantum system which is you is being shut down--no splitting realities are involved here (except the usual ones)-- you are going to experience death, and it was my understanding that the point of cryonics and mind transfer was to avoid experiencing death. (By "experience death", I mean that your mind-pattern ceases to function.) Anyone deriving comfort from those two methods should seriously consider this concrete downside to them.
Assuming that you have programmed it to care about its own consciousness, not just to ponder it, the first boot would die, and the reboot would wake up thinking it was the first boot.
But if a consciousness can be simulated on a computer running at multiple GHz, would not a simulation on a computer running at one cycle per hour also be consciousness? And then if you removed power from the computer for the hour between each cycle, is there any reason to think that would affect the simulation?
Most people, given the option to halt aging and continue in good heath for centuries, would. Anti-aging research is popular, but medicine is only minimally increasing lifespan for healthy adults. You, I, and everyone we know have bodies that are incredibly unlikely to make it past 120. They're just not built to last.
But what are you, really? Your personality, your memories, they don't leave you when you lose a leg. Lose most parts of your body and you're still you. Lose your brain and that's it. [1] You are a pattern, instantiated in the neurons of your brain. That pattern is sustained by your body, growing and changing as you learn and experience the world. Your body supports you for years, but it deteriorates and eventually isn't up to the task any more. Is that 'game over'?
Perhaps we could scan people's brains at extremely high detail so we could run them in some sort of human emulator. This requires a thorough understanding of the brain, huge amounts of storage, unbelievably fast computers, and very detailed scanning. If it's even possible, it may be several hundred years away.
Our bodies aren't going to last that long, but what if we could figure out how to preserve our brains so that the information didn't decay? Then, if the future turned out to be one in which we had advanced brain emulation and scanning technology, we could be revived. I don't know if people in the future would want to spend the time or money to revive us, but in a future with technology this advanced, reviving a preserved brain as a computer simulation could be really cheap.
The most advanced technology for long-term tissue preservation today [2] is cryonics: freezing with vitrification. You add something to the blood that keeps ice crystals from forming and then freeze it. This is pretty much the same thing frogs do, hibernating frozen through the winter. The biggest organs that have been successfully brought back to working order after vitrification are rabbit kidneys, and the brain is a lot bigger and much more complex. While there are people applying this technique to human brains after death, it's very much a one way street; we can't revive them with current technology.
How much should it worry us that we can't reverse this freezing process? If we're already talking about revival via high-detail scanning and emulation, which is only practical after hundreds of years of technological development, does it matter that we can't currently reverse it? The real question in determining whether vitrification is sufficient is whether we're preserving all the information in your brain. If something critical is missing, or if something about our current freezing process loses information, the brains we think are properly preserved might be damaged or deteriorated beyond repair. Without a round trip test where we freeze and then revive a brain, we don't know whether what we're doing will work.
Another issue is that once you've frozen the brain you need to keep it cold for a few centuries at least. Liquid nitrogen is pretty cheap, but providing it constantly over such a long time is hard. Organizations fall apart: very few stay in business for even 100 years, and those that do often have departed from their original missions. Current cryonics organizations seem no different from others, with financial difficulties and imperfect management, so I don't think 200+ years of full functioning is very likely.
Even if nothing goes wrong with the organization itself, will our society last that long? Nuclear war, 'ordinary' war, bioterrorism, global warming, plagues, and future technologies all pose major risks. Even if these don't kill everyone, they might disrupt the cryonics organizations or stop technological development such that revival technology is never developed.
Taking all these potential problems and risks into account, it's unlikely that you can get around death by signing up for cryonics. In attempts to calculate overall odds for success from estimated chances of each step I've seen various numbers: 1:3, 1:4, 1:7, 1:15 and 1:400. I'm even more pessimistic: I calculated 1:600 when I first posted to lesswrong and have since revised down to 1:1000. To some people the probability doesn't matter, but because it's expensive and there are plenty of other things one can do with money, I don't think it's obviously the sensible thing to do.
(I also posted this on my blog.)
[1] Well, lose your heart and you're gone too. Except that we can make mechanical hearts and you stay the same person on receiving one. Not so much with a mechanical brain.
[2] Plastination is also an option, but it's not yet to a point where we can do it on even a mouse brain.