This is a linkpost for https://www.transhumanaxiology.com/p/the-dissolution-of-ai-safety
The problem of "humans hostile to humans" has two heavy tails: nuclear war and biological terrorism, which could kill all humans. A similar problem is the main AI risk: AI killing everyone for paperclips.
The central (and not often discussed) claim of AI safety is that the second situation is much more likely: it is more probable that AI will kill all humans than that humans will kill all humans. For example, by advocating for pausing AI development, we assume that the risks of nuclear war causing extinction are less than AI extinction risks.
If AI is used to kill humans as just one more weapon, it doesn't change anything stated above until AI evolves into an existential weapon (like a billion-drone swarm).
These aren't the only heavy tails, just the ones with highest potential to happen quickly. You could also have e.g. people regulating themselves to extinction.
I think this is a temporary situation because no sufficiently powerful entity has invested sufficiently much in AI-based defence. If this situation persists without any major shift in power for long enough, then it will be because the US and/or China have made an AI system to automatically suppress AI-powered gangs, and maybe also to automatically defend against AI-powered militaries. But the traditional alignment problem would to a great degree apply to such defensive systems.
LLMs have almost completely negated the original reasons people had to believe in “AI Risk”