Thank you for posting this.

In the context of AI safety, I often hear statements to the effect of

This is something we should worry about.

There's a very important, fundamental mistake being made there that can be easy to miss: worrying doesn't help you accomplish any goal, including a very grand one. It's just a waste of time and energy. Terrible habit. If it's important to you that you suffer, then worrying is a good tactic. If AI safety is what's important, then by all means analyze it, strategize about it, reflect on it, communicate about it. Work on it.

Don't worry about it. When you're not working on it, you're not supposed to be worrying about it. You're not supposed to be worrying about something else either. Think a different thought, and both your cognitive work and emotional health will improve. It's pure upside with no opportunity cost. Deliberately change the pattern.

To all those who work on AI safety, thank you! It's extremely important work. May you be happy and peaceful for as long as your life or this world system may persist, the periods of which are finite, unknown to us, and ultimately outside of our control despite our best intentions and efforts.

Very insightful post. Here are personal thoughts with low epistemic status and high rambling potential:

These all feel to me like corollaries to the belief "AGI is so important that I can't gauge the value of anything else except in regards to how it affects AGI". Hence: "everything else is meaningless because AGI will change everything soon" or "nobody around me is looking up at the meteor about to hit us and that makes me feel kind of insane. (*Cough* so I hang out with rationalists, whose entire shtick is learning how not to be insane)". 

As for other non-obvious effects: I personally feel some sort of perceived fragility around the whole field. There are arguments on this site for why AGI alignment should not be discussed in politics or why attempting to convince OpenAI or DeepMind employees to switch jobs can easily backfire (eg this post for caution advice). These make any outreach at all seem risky. There are also people I know wondering whether they should attempt to do anything at all relative to alignment, because they perceive themselves as probable dead weights. The relatively short timelines, the sheer scope, and the aura of impossibility around alignment seem to make people more cautious than they otherwise should be. Obviously the whole point of the field is to be cautious; but while it's true that the tried-and-tested scientific method isn't safe for AGI in general I'm not sure stressing the rationalist-tools solve-problems-before-you-experiment approach is healthy everywhere. So, caution is right there in the description of the field, but you have to make sure you contain it well so that it doesn't infect places where you would do good to be reckless and use trial-and-error. I am probably quite wrong about this but I don't see many people talking about it, so if there's any reasonable doubt we should figure it out. 

Alignment work should probably be perceived as less fragile. Unlike the AI field in general, alignment projects specifically don't pose much of a risk to the world. So we can probably afford to be more loose here than elsewhere. In my experience alignment feels like a pack of delicate butterflies flying together, with every flap of wings sending dozens of comrades spiraling out of the sky, which might or might not set off a domino/Rube Goldberg machine that blows up the world. 

The alienation is something I felt for a bit, until I started working on my project and working with folk, talking to folk, etc. Also, been very pleasantly surprised how receptive non AI/non-tech folk are when talking to them about AI risk, as long as it's framed in a down to earth, relatable manner, introduced organically, etc.

Bella is meeting a psychotherapist, but they treat her fear as something irrational. This doesn't help, and only makes Bella more anxious. She feels like even her therapist doesn't understand her.

How would one find a therapist in their local area who's aware of what's going on in the EA/rat circles such that they wouldn't find statements about, say, x-risks as being schizophrenic/paranoid?

I feel like Ezra. I've also gotten various sources of feedback that make me think I might not be cut out for "top level" alignment research... but I find myself making progress on my beliefs about the problem, just slower than others.

Thoughts? Advice?

Patterns

AGI will either cause doom or create a utopia. Everything else seems unimportant and meaningless.

Alex is an ML engineer working in a startup that fights with aging. He believes that AGI will either destroy humanity or bring a utopia, and among other things it will stop aging, so Alex thinks that his job is meaningless, and quits it. He also sometimes asks himself "Should I invest? Should I exercise? Should I even floss my teeth? This all seems meaningless." 

No one knows how the post-AGI world will look like. All predictions are wild speculations, and it's very hard to tell whether any actions unrelated to AI safety are meaningful. This uncertainty can cause anxiety and depression

These problems are an exacerbated version of existential problem of meaninglessness of life, and the way to mitigate them is to rediscover meaning in the world that ultimately doesn't have meaning.

 

I feel like this is an instance of a more general issue: In general, we are bad at rescaling utility when we encounter new situations, and our non-utilitarian way of evaluating outcomes can lead us into very large amounts of pain. The issue is basically that utopia and doom/dystopia are the limiting cases of the problem of information that appears to change their utility calculations vastly, especially negative utility, so psychological problems appear like denialism or guilt.

Essentially, the way to handle this problem is to do 2 things:

1. Reset the 0 point, such that the new information means that your 0 point is the way the world works now.
2. Rescale utilities such that instead of treating vastly important problems as things where you treat them as having massive disutility or utility, instead go in the opposite direction. Rescale utilities such that other problems have less utility than this problem and maintain something approximating a normal utility for even the most important problems.

philip_b has the gory details on that process, and it's worth taking a look at it:

I suggest not only shifting the zero point, but also scaling utilities when you update on information about what's achievable and what's not. For example, suppose you thought that saving 1-10 people in poor countries was the best you could do with your life, and you felt like every life saved was +1 utility. But then you learned about longtermism and figured out that if you try, then in expectation you can save 1kk lives in the far future. In such situation it doesn't make sense to continue caring about saving an individual life as much as you cared before this insight - your system 1 feeling for how good thing can be won't be able to do its epistemological job then. It's better to scale utility of saving lives down, so that +1kk lives is +10 utility, and +1 life is +1/100000 utility. This is related to Caring less.

In general, I kinda wished rationalists would make their pitches, at least later on as essentially about caring less about certain problems, rather than caring more about x cause.

Typo nitpicks (suggestions): "humanity might extinct" --> "go extinct" / "everything else seem unimportant and meaningless" --> "everything else seems" / "non-AI safety company" --> "a non-AI safety company" / "he feels demotivated, he also needs" --> "he feels demotivated, but he also needs" /  "version of existential problem" --> version of the existential problem" 

I wonder to what extent you meant it when you said those were specific to AI safety? I’m not at all involved in that (but on the other hand I’m still probably on LW far too much), and I liter have all of them. Or did you mean ‘here is how some common-ish psychological issues manifest themselves in an AI safety context’ ?