I updated it again. Admittedly the post needs a lot of work, it was only planned as a discussion after this problem came to my mind the previous night. I added two paragraphs, edited others, added headings and accounted for some comments. I'll update it again as I learn more about this issue.
Just go with the simple truth. Use your intuitions, even if you don't understand them: it's better than not using them at all, instead relying on abstractions of dubious relevance. Or wait for the abstractions to grow up.
Just go with the simple truth. Use your intuitions, even if you don't understand them...
That's what I have basically been doing before LW. I let my intuition decide when not to trust my intuition. But I noticed that my intuitions disagree with some of the eventual conclusions being made here. And I perceive the consensus to be that if your intuition does disagree then you should better distrust your intuition, not what has been written here. Since I am unable to figure out when not to trust my intuitions I can either trust all of them or none.
Since I am unable to figure out when not to trust my intuitions I can either trust all of them or none.
You trust the best tool you have, to the extent you believe (based on what the best tools you have tell you) it should be trusted. If trusting a judgment of perfect emptiness is better than trusting your intuitions, stop trusting your intuitions, but it's rarely like that. When you stop trusting your intuitions, you don't automatically get a better alternative. Unless you have such an alternative, by all means use the intuitions.
Relevant posts: The Simple Truth, ...What's a bias, again?, Where Recursive Justification Hits Bottom.
(Also, it's not actually possible to stop trusting all of your intuitions. You can overrule only a small number of individual nudges, since even your conscious deliberation consists of smaller intuitions.)
What I do is be excessively slow to take on new ideas and chew on them in a sandbox. There is no such thing as strong memetic sandboxing, but it slows the ingress of bad ideas. And good ones, but hey.
If you find you're too susceptible to incoming memes, which appears to be what you're describing, then consciously trusting your intuitions further seems likely to lead to better results than what you're doing now.
This may lead to some degree of inconsistency and compartmentalisation (memetic sandboxing being akin to deliberate compartmentalisation), but the results will, I think, still be less worse than what you're getting now.
If you have to choose at least for the moment between blanket trust of your intuition versus blanket trust of stuff some guys wrote on the Internet, go with the former. Your intuition has at least managed to keep you alive so far.
Especially since its a question of values. "Is it worthwhile for parts x, y, and z to go on existing with the rest gone?" Can be mostly answered by asking yourself if you'd be interested in the prospect.
It can't be reliably so answered, but it's often the best tool we have, and the thing to do when dissatisfied with quality of the best available option is not to discard it.
There are ways of improving the intuitions though, patching particular problems in them, and some even appear in this post. Continuity of experience is one such confused notion, which gives a good argument against immediate intuition in the relevant situations.
"Is it worthwhile for parts x, y, and z to go on existing with the rest gone?"
I am trying to figure out what it is that defines us, what it means to survive. That Sarah Palin and you both drink coffee or that some being with your name will be maximizing paperclips isn't it. Self-preservation seems to be an important part of rationality, or what is it that instrumental rationality is for? And if you can't recognize yourself then what about epistemic rationality? Of course, there is only the universe that is unfolding and no self, but that could be said about values as well.
Admittedly our DNA is the earliest version of our selfs, but if we don't care about the temporal parts of our selfs but only about a static algorithmic description of a certain spatiotemporal position, then what's wrong with that?
Under your logic should identical twins self-identify as each other?
Under your logic should identical twins self-identify as each other?
No? I guess I didn't make that clear enough but I was actually arguing against it when I further wrote, "It seems a lot, we stop caring about past reifications of our selfs...". At the end I conclude that the only time-consistent and reliable definition of identity can be based upon an agents utility function and therefore any agent that does not exhibit goal-stability will be prone to make inconsistent decisions.
Under your logic should identical twins self-identify as each other?
A second thought, yes if human values and goals are genetically encoded then we should identify with our twins to the extent that it matters much more if two agents share the same utility-function than if they share a causal history. But I believe that human values are largely a part of our upbringing, culture and education. I am not sure if humans exhibit stable goals but believe it would be important to find out what goals are universal and unmodifiable.
I don't disagree with any of this, as long as we don't forget that value is complex.
That is, I can value preserving the continuity of my memory in the short term, even while acknowledging that in the long term all the bits that are currently in my memory will be more-or-less lost even if "I survive indefinitely" (in square quotes because I don't believe that "I" means something coherent in the context of indefinite survival, which makes the phrase intrinsically misleading).
And yeah, there's a point along that continuum where I stop taking it personally, and am just as happy to have "someone else" replace "me." But until that point, I do take it personally.
And there are other values that can be optimized for without reference to my identity at all, and I'm happy to have those maximized as well... and in some cases, would prefer that to my own identity being preserved.
That is, I can value preserving the continuity of my memory in the short term, even while acknowledging that in the long term all the bits that are currently in my memory will be more-or-less lost even if "I survive indefinitely" (in square quotes because I don't believe that "I" means something coherent in the context of indefinite survival, which makes the phrase intrinsically misleading).
Well, there's no obvious incoherence in caring about the ancestral of memory-connectedness instead of, or in addition to, connectedness itself. (Parfit talks about this in Reasons and Persons.) You could even define "yourself" via the ancestral, in which case "you" could indeed survive indefinitely, provided that in each year you have strong memories of the previous year.
Not that I find this idea particularly attractive. Instead, looking in a cold clear light at what I care about when it comes to survival, there are two strands. One is as XiXiDu mentions: I want (an) agent(s) to continue pursuit of my goals. But the deeper emotional attraction comes from anticipating good experiences. And the identity constraint on anticipating good experiences is: absolutely none. I can anticipate your good future just as sensibly as I can anticipate mine, provided that I can understand what would be a good experience for you - or for that matter, provided I can understand what would be a good experience for my future self.
"would we don't mind to be alive ever after because we died when the computation halted? This doesn't seem to be the case."
Typo.
Why are you assuming that the intuition corresponds to somehting well defined and real at all? I'm not saying it doesn't but it seems far from obvious.
I don't see why we can't value continuity of consciousness and still go under anesthesia. Temporary states are extremely different than the permanent state of death
I don't see why we can't value continuity of consciousness and still go under anesthesia.
We can do that of course. What I tried to fathom with my post was time-consistent self-referencing. We can value our memories, the continuity of experience and even our DNA. But the only reliable marker for agents seems to be a stable utility function. Rational agents with an identical utility function will to some extent converge to exhibit similar behavior and are therefore able to cooperate. We can more consistently identify with our values and goals than with our past and future memories, digitized backups or causal history.
We expect to come back from general anaesthesia. People consider the dangers of being operated on, but don't seem to think of anaesthesia as being one of them.
(That said, general anaesthesia is way less safe than people seem to regard it, which is why anaesthetists are fully-trained doctors who have specialised, rather than just technicians. And why your anaesthetist can turn you away on the morning of your operation just like that if they're at all unsure.)
In this post I try to fathom an informal definition of Self, the "essential qualities that constitute a person's uniqueness". I assume that the most important requirement for a definition of self is time-consistency. A reliable definition of identity needs to allow for time-consistent self-referencing since any agent that is unable to identify itself over time will be prone to make inconsistent decisions.
Data Loss
Obviously most humans don't want to die, but what does that mean? What is it that humans try to preserve when they sign up for Cryonics? It seems that an explanation must account and allow for some sort of data loss.
The Continuity of Consciousness
It can't be about the continuity of consciousness as we would have to refuse general anesthesia due to the risk of "dying" and most of us will agree that there is something more important than the continuity of consciousness that makes us accept a general anesthesia when necessary.
Computation
If the continuity of consciousness isn't the most important detail about the self then it very likely isn't the continuity of computation either. Imagine that for some reason the process evoked when "we" act on our inputs under the control of an algorithm halts for a second and then continues otherwise unaffected, would we don't mind to be alive ever after because we died when the computation halted? This doesn't seem to be the case.
Static Algorithmic Descriptions
Although we are not partly software and partly hardware we could, in theory, come up with an algorithmic description of the human machine, of our selfs. Might it be that algorithm that we care about? If we were to digitize our self we would end up with a description of our spatial parts, our self at a certain time. Yet we forget that all of us possess such an algorithmic description of our selfs and we're already able back it up. It is our DNA.
Temporal Parts
Admittedly our DNA is the earliest version of our selfs, but if we don't care about the temporal parts of our selfs but only about a static algorithmic description of a certain spatiotemporal position, then what's wrong with that? It seems a lot, we stop caring about past reifications of our selfs, at some point our backups become obsolete and having to fall back on them would equal death. But what is it that we lost, what information is it that we value more than all of the previously mentioned possibilities? One might think that it must be our memories, the data that represents what we learnt and experienced. But even if this is the case, would it be a reasonable choice?
Indentity and Memory
Let's just disregard the possibility that we often might not value our future selfs and so do not value our past selfs either for that we lost or updated important information, e.g. if we became religious or have been able to overcome religion.
If we had perfect memory and only ever improved upon our past knowledge and experiences we wouldn't be able to do so for very long, at least not given our human body. The upper limit on the information that can be contained within a human body is 2.5072178×1038 megabytes, if it was used as a perfect data storage. Given that we gather much more than 1 megabyte of information per year, it is foreseeable that if we equate our memories with our self we'll die long before the heat death of the universe. We might overcome this by growing in size, by achieving a posthuman form, yet if we in turn also become much smarter we'll also produce and gather more information. We are not alone either and the resources are limited. One way or the other we'll die rather quickly.
Does this mean we shouldn't even bother about the far future or is there maybe something else we value even more than our memories? After all we don't really mind much if we forget what we have done a few years ago.
Time-Consistency and Self-Reference
It seems that there is something even more important than our causal history. I think that more than everything we care about our values and goals. Indeed, we value the preservation of our values. As long as we want the same we are the same. Our goal system seems to be the critical part of our implicit definition of self, that which we want to protect and preserve. Our values and goals seem to be the missing temporal parts that allow us to consistently refer to us, to identify our selfs at different spatiotempiral positions.
Using our values and goals as identifiers also resolves the problem of how we should treat copies of our self that are featuring alternating histories and memories, copies with different causal histories. Any agent that does feature a copy of our utility function ought to be incorporated into our decisions as an instance, as a reification of our selfs. We should identify with our utility-function regardless of its instantiation.
Stable Utility-Functions
To recapitulate, we can value our memories, the continuity of experience and even our DNA, but the only reliable marker for the self identity of goal-oriented agents seems to be a stable utility function. Rational agents with an identical utility function will to some extent converge to exhibit similar behavior and are therefore able to cooperate. We can more consistently identify with our values and goals than with our past and future memories, digitized backups or causal history.
But even if this is true there is one problem, humans might not exhibit goal-stability.