You have probably read about the idea of quantum immortality before. The basic idea seems to be that, as anything that can happen does happen (assuming either that the many-worlds interpretation of quantum theory is true, or that there is an infinite number of parallel universes wherein "you" exist) and it is impossible to remember your own death, every living thing is immortal.

Take a game of russian roulette as an example: In those universes in which you die, you are no longer alive enough to care about that fact, leaving only those universes relevant in which you survive. This would make playing russian roulette for money a valid financial strategy, by the way.

However, I think that this view ignores a very important fact: death is not binary. You are not either alive or dead, but may exist in various intermediate forms of suffering and reduced cognitive abilities. This means that what actually happens when you play russian roulette is the following:

In those universes in which you win, everything is fine. In those in which you lose, however, you now have a gaping head wound. I assume that this hurts a lot, at least in those instances where you still have enough mental capacity to actually feel anything. Due to some fluke however (remember that absolutely all possible scenarios happen), you may still be alive and in a lot of pain. Most instances of you will then die from bloodloss or something, but for every timestep afterwards there will alway be an infinite number of universes wherein you continue to live, in most of them in complete agony.

The instances of you in the other worlds that were never shot will be blissfully unaware of this fact.

Now consider that you will also reach such a state of perpetual-agony-close-to-death-but-never-quite-reaching-it in everyday live. In fact, an infinite number of alternate "you"s, having split off from your everett branch just a second ago, are now suffering through this.

The ratio of "you"s in your current state to those in the one described above is very high, as the probability of continued survival in such a state for any amount of time is infinitesimal, but it does exist. Consider however, that this ratio decreases massively as you age and that virtually all instances of you will be in such a state 200 years from now unless immortality is achieved.

There is one bright spot, however:

As time marches on, the continuous elongation of your suffering/death-that-will-not-come is going to become increasingly unlikely. Therefore, it will eventually be overtaken by the probability that the universes wherein you still persist also contains an entity (an AI?) that is both capable and willing to rescue you. Assuming you still care and haven't gone insane already.

Another interesting thing: The above does not apply if your potential death is very sudden (so you won't feel it) and thorough (so there is an extremely low chance of survival). This means that while plaing russian roulette for money is unreasonable, playing russian roulette for money, using nukes instead of a revolver, is entirely reasonable and recommended :-)

edit: I think I have phrased this really poorly and that this has been misinterpreted. See my comment below for clarification.

A lot of thought has been put into the discussion of how one would need to define the goals of an AI so that it won't find any "loopholes" and act in an unintended way.

Assuming one already had an AI that is capable of understanding human psychology, which seems necessary to me to define the AI's goals anyway, wouldn't it be reasonable to assume that the AI would have an understanding of what humans want?

If that is the case, would the following approach work to make the AI friendly?

-give it the temporary goal to always answer questions thruthfully as far as possible while admitting uncertainty

-also give it the goal to not alter reality in any way besides answering questions.

-ask it what it thinks would be the optimal definition of the goal of a friendly AI, from the point of view of humanity, accounting for things that humans are too stupid to see coming.

-have a discussion between it and a group of ethicists/philosophers wherein both parties are encouraged to point out any flaws in the definition.

-have this go on for a long time until everyone (especially the AI, seeing as it is smarter than anyone else) is certain that there is no flaw in the definition and that it accounts for all kinds of ethical contingencies that might arise after the singularity.

-implement the result as the new goal of the AI.

What do you think of this approach?