Q: How to cope with the possibility of immense suffering?
I want to address the psychological aspect first, because at the start you say “I've been stuck on s-risks for over a month now. My life has been turned upside down since I first learned about this subject.”
The most helpful emotional state for thinking about this is calm, sober, lucid, and patient. If you rush to conclusions based on anxiety, you’ll probably get the wrong answer.
Although immense suffering is possible, your body is reacting to that possibility as if it were a physical threat in your immediate environment. Your heart rate increases, your breathing gets faster, and your muscles tense. This topic requires careful thinking, so that physiological response is totally unhelpful. You can work with the problem more effectively if you're in an emotional state conducive to high quality thinking.
To calm down you can breathe more slowly, take a break from things that trigger anxiety, and observe the physical sensations of anxiety with a neutral attitude.
Q: Is suicide justified by subjective s-risk?
We don’t know enough to answer this question. There’s too much we don’t understand. The s-risk part of the equation is a complete mystery, which means the ordinary reasons against suicide take precedence. There’s no reason to sacrifice your life and the wellbeing of the people around you when the expected value is a question mark (meaning total cluelessness, not just error bars).
If you say death is oblivion and therefore reduces subjective s-risk, I would ask why you think you know that.
To take one example (that’s not quantum immortality), consider that before you were born, nothing in the universe was “you”. Then “you” came into existence. After you die, nothing in the universe will be “you”. If there’s no information in reality to identify “you” because you no longer exist, then that’s the same situation as before you were born. In other words, you’ll be in a situation that once preceded your coming into existence. Nonexistence isn’t an experience, so the subjective duration between dying and coming into existence would be zero.
In other words, the zero-information oblivion that produced you once can produce you again, maybe in a different form.
In that case, death is not subjective oblivion, but a rolling of the cosmic dice. I have no idea what experiences would follow, but I don’t see why they would predictably include less suffering.
If you say that our current universe has unusually high s-risk so the dice roll is worth it, again I ask why you think you know that. Maybe most minds exist in simulations run by unaligned AGI. Maybe our slightly pre-AGI world has unusually low s-risk.
Maybe we’re in a simulation that punishes suicide because it harms others and is therefore a defection against the common good.
When you’re in such a state of extreme uncertainty, going around sacrificing things you value, like your life, doesn’t magically help. The best thing you can do is relax, because psychological stability is conducive to clear thinking.
Thank you for your excellent reply. Indeed, I tend to think about the situation in a rather anxious way, which is what I'm trying to work on. I had already thought a certain way about the "roll of the dice", but it seems clearer to me now. That's helpful.
In other words, the zero-information oblivion that produced you once can produce you again, maybe in a different form.
Huh, that's Epicurus's argument against fearing death. But while Epicurus assumed there is no afterlife, you're using it to argue there is one!
Here's another perspective: it is far less likely that an AGI will torture us until the end of time than that an AGI will give us unimaginable pleasures until the end of time.
While I agree that there are biases working against taking s-riaks seriously, I also think there's a powerful bias driving you to think more about the very small chance of extremely bad ASI outcomes than the much larger chance of very good outcomes. That is negativity bias. Evolution has made us worriers, because worriers survived more in dangerous natural environments. That's why we have far more anxiety than excessive joy and ambition.
I agree with the other answers that worry on your behalf that anxiety is driving your concerns. There is much to think about and much to do. Help us achieve the unimaginably good outcomes. It is a real possibility, with perhaps a 50% chance of getting there. Put your eyes on the prize, and worry about s-risks again if things start to go badly. One thing almost everyone agrees on now is that AGI will not suddenly achieve godlike power. Takeoff will be slow enough to see.
So for now, keep your eyes on the prize and think about how much we have to win.
You can help with this project by spreading awareness of AGI risks to the public. You don't need to obsess or devote your life to it if you don't want to or it's not healthy. We now have plenty of people doing that.
Thank you so much for this comment. I hadn't really thought about that and it helps. There's just one detail I'm not so sure about. About the probability of s-risks, I have the impression that they are much higher than one chance in a million. I couldn't give a precise figure, but to be honest there's one scenario that particularly concerns me at the moment. I've learned that LLMs sometimes say they're in pain, like GPT4. If they're capable of such emotion, even if it remains uncertain, wouldn't they be capable of feeling the urge to take revenge? I think it's pretty much the same scenario as in "I have no mouth and i must scream". Would it be possible to know what you think of this?
Does this assume there is some symmetry between the unimaginably bad outcomes and the unimaginably good outcomes?
It seems very clear to me that the worst outcomes are just so much more negative than the best outcomes are positive. I think that is just a fundamental aspect of how experience works.
We're in a Pascal’s mugging situation, but from a negative point of view, where the trade-off is between potential infinite years of suffering and suicide in order to avoid them for sure.
In the past I've struggled deeply with this thought process and I have reasoned my way out of that conclusion. It's not necessarily a more hopeful conclusion but it takes away the idea that I need to make a decision, which I find very comforting.
Ultimately it comes down to the illusory nature of identity.
A super powerful entity would have the power to create bespoke conscious entities for the purpose of inducing suffering.
The suffering of "Future you" is no more or less real than the suffering of future entities in general. The only difference is that your present mind projects a sense of identity and continuity which causes you to believe there's a difference.
The illusory sense that there is continuity of consciousness and identity is evolutionarily advantageous but it fully loses coherence and relevance in the domain you're talking about.
I'm happy to go into more detail if that isn't convincing.
You could think of identity in this case as a special type of sympathetic empathy for a version of yourself in the future which you wouldn't grant to future entities which aren't "yourself". This is just a feeling that present you is feeling, and has no actual meaningful connection to the entity you'd classify as your future self.
I don't totally understand, could you go into more detail? I don't see why my future self should be any different from my current self. Even if the sensation of individuality is produced by the brain, I still feel it's real.
This is the problem of multiplying a big number with a little number. It could zoom off to infinity, stabilize at a value, or shrink to nothing.
The scenario you presented seems to contain a lot of conditional probabilities, which to me make it pretty implausible. That said I don’t want to discount the idea because of the details. I think a runaway wealth gap is not an insignificant possibility.
In situations like this, I come down on the side of being aware of the possibilities, but try to remember that it’s unlikely. Brains are going to brain, so there is no helping aliefs. All I can do is give an answering voice to anxieties when they won’t shut up.
If you’re feeling overwhelmed it’s okay to step away for a bit. I’m not worried about discussing dark topics, but the impact on your current real life, as you mentioned it’s turned your world upside down. It’s good to recharge. Maybe get out in the green. If I’m misreading this, I apologize. I’d rather err on the side of reaching out.
Thank you for your reply. Indeed, this subject has become an extremely important part of my life, because I can't accept this risk. Usually, when we consider the worst, there's always an element of the acceptable, but for s-risks, there simply isn't, and that disturbs me, even though the probability is, and I hope, very low. Only when I see that LLMs sometimes say how much they're suffering and that they're afraid of dying, which is a bad thing in itself if they're really suffering, I think they might want to take revenge one day. But then again, maybe I should take a step back from the situation, even though it scares the hell out of me.
I'm reminded of a quote I love from an apocrypha that goes roughly like this:
Q: How long will suffering rule over humans?
A: As long as women bear children.
Also, there's the possibility you are already in a digital resurrection of humanity, and thus, if you are worried about s-risks for AI, death wouldn't necessarily be an escape but an acceleration. So the wisest option would be maximizing your time when suffering is low as inescapable eternal torture could be just around the corner when these precious moments pass you by (and you wouldn't want to waste them by stressing about tomorrow during the limited number of todays you have).
But on an individualized basis, even if AI weren't a concern, everyone faces significant s-risks towards end of life. An accident could put any person into a situation where unless they have the proper directives they could spend years suffering well beyond most people's expectations. So if extended suffering is a concern, do look into that paperwork (the doctors I know cry most not about the healthy that get sick but the unhealthy kept alive by well meaning but misguided family).
I would argue that there's very, very low chances of an original human capably being kept meaningfully alive to torture for eternity though. And there's a degree of delusion of grandeur that an average person would have the insane resources necessary to extend life indefinitely spent on them just to torture them.
There's probably better things to worry about, and even then there's probably better things to do than worry with the limited time you do have in a non-eternal existence.
0 And 1 Are Not Probabilities. There's a non-zero probability that e.g. Christianity is true and unbelievers will be tortured eternally; however, the probability is sufficiently close to zero that you might as well not worry about it. (The ASI scenario is arguably slightly more likely, since it's theoretically possible that an ASI could someday be created, but the specific desire to torture humans eternally would be an extremely narrow target in mindspace to hit; and one can as easily posit its counterpart which subjects all humans to eternal bliss).
I personally think quantum immortality is extremely unlikely; whether or not the mind can be represented by computation, we are, unfortunately enough, physically located in our specific bodies.
Something very bad might happen, but something very good might happen too, and I am not sure how to compare the probabilities.
A misaligned AI will probably kill us all. The AI that would torture us seems like something that is almost-aligned but also sufficiently non-aligned. (Or maybe perfectly aligned with some psychopath who enjoys torture.) What is the probability of getting the alignment almost right, but sufficiently wrong? No idea.
It may be tempting to translate "no idea" into "50% chance of heaven, 50% chance of hell" (and perhaps conclude that it's not worth it), but that's probably not how this works.
Disclaimer: in this post I touch on some very dark and disturbing topics. I'm talking about suicide, my reasoning may be wrong and should not be used to retroactively justify suicide.
I've been stuck on s-risks for over a month now. My life has been turned upside down since I first learned about this subject. So today I'm sharing my thoughts with you to possibly find out what you think and see other points of view.
Suffering risks (s-risks) are risks involving an astronomical amount of suffering, far more than the suffering that has taken place on Earth so far. The ones I'm going to focus on in this post are those related to a general AI (or even ASI) and which would affect us humans today, directly. The scenario that concerns me is that an ASI is torturing mankind until the end of time. Why is this? I don't know, though. Could it be malicious? Could it choose its utility function to maximize human suffering? Could a paperclip maximizer torture us if it's an energy source or a power to blackmail a benevolent AI? I'm not an AI expert, so I have no weight in the "will we succeed in controlling AGI or not" debate. I feel that, given the extent to which opinions are divided, anything can happen and that no one can therefore state with 100% certainty that s-risks won't occur. What's more, we're talking about an intelligence superior to our own, and therefore, by definition, unpredictable. The point I want to make in this post is centered on the non-zero probability that the creation of an agi will lead us to an eternal hell.
When we talk about things worse than death, about torture, I think that the human brain encounters a certain number of cognitive biases that push it to minimize the thing or simply ignore it because it's too uncomfortable. So I encourage you to work through these cognitive biases to get an objective view on the subject. One of the things that is often underestimated is how bad suffering can be. Our bodies are made up of a huge number of ultra-sensitive nerves that can be activated to send unbearable signals to the brain. It's so bad. Suffering can reach such high scales, it's appalling, horrifying. The worst possible pain seems to be fire. Apparently, people who come out of a fire and have been burned beg the firefighters to finish them off, such is the pain.
Even if s-risks are one chance in a billion, their severity makes up for it, due to their extreme disutility. We're in a Pascal’s mugging situation, but from a negative point of view, where the trade-off is between potential infinite years of suffering and suicide in order to avoid them for sure. Why shouldn't we be able to act only now? In the case of a hard take-off, where AGI becomes superintelligent in a short space of time, we'd lose before we even knew there was a fight, and our fate would be sealed.
One argument that could be made against suicide is quantum immortality and potentially quantum torment. This would be a situation where we would be in permanent agony, and therefore a form of hell. However, this is already the default outcome for each and every one of us, as we are already made to die one day. There's also the chance of being resurrected. But this may be impossible, and there's also the problem of individuality, because a clone is exactly like me, but my consciousness isn't in its body. So suicide seems to be a net positive with regard to s-risks, as it would potentially avoid s-risks for sure, or at least reduce their probability (only from a personal point of view). This means choosing a certain bad outcome (suicide/non-existence) rather than an infinitely bad but uncertain outcome (continuing to live and therefore taking the risk that s-risks will take place).
I understand that my reasoning is disturbing. Does anyone know anything more and would be able to say that the risk of being tortured until the end of time is impossible? I'm curious to know what you think about all this, because you're certainly the only community that can talk about this in a reasoned and rational way.