To be honest, part of the reason I waited on making the post was because I was confused about it myself :P. But nevertheless. The following questions probably need to be either answered or dissolved by a complete theory of identity/consciousness; the first is somewhat optional, but refusing to answer it shunts it onto physics where it becomes much stranger. I'm sure there are others questions, too - if nobody responds to this comment I'll probably make a new post regardless.
Why aren't I a Boltzmann Brain? - this one's long, so I'm breaking it off.
A Boltzmann brain is a response to the argument that the entire universe might just be a momentary patch of order forming by pure chance on a sea of random events, and that therefore our memories never happened and the universe will probably fall apart in the next few seconds. The response is that it is far more likely that, rather than the entire universe coming together, only your brain spontaneously is created (and then dies moments later, as the physics it relies on to function doesn't exist or are significantly different.) The response can be further generalized - the original argument requires something like a Tegmark IV multiverse that contains all universes that are mathematically consistent, but even in a Tegmark I multiverse (simply an infinite universe with random matter patterns in all directions) you would occasionally expect to see copies of your brain forming in empty space before dying in vacuum, and further that there would be many more of these Boltzmann brains than coherent versions of yourself.
And yet, it seems ludicrous to predict that in the next second, down will become purple and my liver will sprout legs and fly away by flapping them. Or that I will simply die in vacuum, for that matter. So... why aren't I a Boltzmann brain?
Why the Born probabilities?
I see no reason why a non-conscious machine, say a bayesian superintelligence, would not encounter the Born probabilities. As such, consciousness seems unlikely to be related to them - it's too high-level to be related to quantum effects.
Continuity of consciousness. Part of the hard problem of consciousness - why do I wake up tomorrow as myself and not, oh, EY?
How do you define "I" that you can credibly imagine waking up as Eliezer? What difference do you expect in the experience of that Eliezer? I think it's a b...
Although Elizier has dealt with personal identity questions (in terms of ruling out the body theory), he has not actually, as far as I know, "solved" the problem of Personal Identity as it is understood in philosophy. Nor, as far as I know, has any thinker (Robin Hanson, Yvain, etc) broadly in the same school of thought.
Why do I think it worth solving? One- Lesswrong has a tradition of trying to solve all of philosophy through thinking better than philosophers do. Even when I don't agree with it, the result is often enlightening. Two- What counts as 'same person' could easily have significant implications for large numbers of ethical dilemnas, and thus for Lesswrongian ethics.
Three- most importantly of all, the correct theory has practical implications for cryonics. I don't know enough to assert any theory as actually true, but if, say, Identity as Continuity of Form rather than of Matter were the true theory it would mean that preserving only the mental data would not be enough. What kind of preservation is necessary also varies somewhat- the difference in requirement based on a Continuity of Consciousness v.s a Continuity of Psyche theory, for example should be obvious.
I'm curious what people here think. What is the correct answer? No-self theory? Psyche theory? Derek Parfit's theory in some manner? Or if there is a correct way to dissolve the question, what is that correct way?