For my MLSS project, I created this distillation of “Is power-seeking AI an existential risk?” which was originally written in 2021 by Joe Carlsmith, an analyst at Open Philanthropy. After a while, Stephen McAleese contacted me and proposed to re-edit it and upload it as a post. Here is the result.
The report estimates the level of existential risk from AI by breaking the possibility of an existential catastrophe from AI down into several enabling steps, assigning probabilities to each one, and estimating the total probability by multiplying the probabilities. The report concludes that advanced AI is a potential source of existential risk to humanity and has a roughly 5% chance of causing an... (read 2525 more words →)