Your project might be failing without you even knowing it.
It’s hard to save the world. If you’re launching a new AI Safety project, this sequence helps you avoid common pitfalls.
Your most likely failure modes along the way:
You never get started. Entrepreneurship is uncomfortable, and AI Safety is complex. There are many failure modes. It’s hard to figure out how to do something useful, so many people never try. Simultaneously, many things in AI Safety that obviously should be done are not done yet. You might mistakenly assume they’re already being done and therefore not try them.
You move slowly, fail to gain traction, and get stuck. You launch a tool – six months later, you have 12 users. You keep adding features, hoping something will change. Nothing does.
You satisfice. You figure that what you’re doing must be high impact, just because it’s part of AI Safety. In reality, you could be having 10x more impact than you are.
You fail without realizing it. You have users, citations, participants. But you're not actually reducing x-risk, and you don't realize it because you're not tracking your impact or not tracking the right things.
To set yourself up for success, you need two broad skillsets: entrepreneurial skills and impact-specific skills, which include impact estimation and strategic understanding of AI Safety. Mastering both skillsets is rare – we for sure know we haven’t.[1]
Think of it roughly as: Impact = Adoption × Effectiveness[3]
You'll require competent execution to build anything at all. But that's not enough. You need to find the right thing to build. Something that leads to positive impact.
The Two Impact Multipliers
These two skillsets help you create something impactful:
1. Entrepreneurial skills help you iterate toward something people actually engage with. 2. Impact skills guide you to the effectiveness that translates adoption into impact. This is tricky because you can’t directly measure whether you reduced AI x-risk until it’s too late to act on that information. Nevertheless, we’ll introduce the skills that will help you increase your odds.
Together, these help you understand the problems in AI Safety, prioritize between them, and develop solutions that work.
Most Projects Won’t Matter. Yours Could.
Your work isn’t automatically high-impact just because it’s part of AI Safety.
Impact follows something like a power law.[4]A small number of projects will create most of the impact.
So by default, your project will probably have little to none, or even negative, impact. But it also means your project could be one of the few making a real difference if you optimize deliberately.[5]
Would you help kill every single human on earth?
Of course not! You’re not a terrorist.
And yet we fear that this sequence of posts will enable you to accidentally move in that direction. There are several reasons for this:
Very negative and very positive AI Safety projects are pretty close together in the input space (we’ll discuss that in post 3).
We will tell you to bias to action, but that bias can be a really bad idea because of the Unilateralist’s Curse (we’ll discuss that in the final post).
For now, we want to leave you with a consideration about your emotions and motivated reasoning:
When really not wanting to do something bad makes you more likely to do that bad thing
We know you don’t want to be responsible for the extinction of humanity, the end of nature, and everything we ever cared about. It’s so trivially true that it’s almost a silly thing to point out.
But we wonder: perhaps you so badly don’t want to be responsible for such terrible things that your brain won’t even allow you to entertain the possibility that, unknowingly and inadvertently, you could be contributing to such bad outcomes.
Perhaps then, faced with such an unfathomably scary proposition, your brain preemptively decides that your work is good. Without having honestly assessed it. That’s motivated reasoning.[6]
Basically: You really don’t want to destroy the world → Your brain won’t allow you to honestly assess how your work might contribute to destroying the world → You can’t make fully-informed decisions to prevent unintended outcomes → You’re more likely to be contributing to the end of the world.
One method to overcome such pre-emptive opinion-making is by using visualizations to process the emotions of a bad outcome before deciding whether the outcome is likely, as proposed in Leave a Line of Retreat.
You can do this
Nobody has really figured out AI Safety yet – whether that’s technical safety, governance, or fieldbuilding. That means you can get up to speed quite quickly, bring new ideas, and move the field forward. You’re not late. You can make a real change.
Funding for AI Safety nonprofits may also increase soon. Anthropic’s cofounders and a number of employees have pledged to donate large portions of their wealth, which is expected to become available at an Anthropic IPO later in 2026. Part of that money may already come available sooner due to an ongoing pre-IPO share sale.[7]
Next Up: Post 2 – Entrepreneurial Skills
Many AI Safety projects fail by getting stuck. Post 2 shows you how successful entrepreneurs iterate toward adoption. Adapted to the context of AI Safety.
Already know how to build things people love? Skip to post 3 about Impact.
Edit: Post 2 is out now and the other posts will be released in the weeks to come. Add your email below to get notified when the next post comes out:
This sequence recombines existing ideas and creates some scaffolding around it. We hope/expect this will be useful for many people, and a similar style is used in many non-fiction books. This sequence is the result of ~80 hours of thinking, reading, and writing, and 30 conversations with experts and peers, though the resulting text remains a distillation of our own understanding rather than the result of quantitative study. If you disagree, you may have good reasons and we'd love to hear them. Moreover, if something isn’t useful to you in your specific situation, don’t use it. ↩︎
Read this as "Effectiveness ≡ Impact per unit of adoption". If you can think of a better word to describe this than 'effectiveness', let us know and we will change it. ↩︎
This is somewhat related to the framing of "Impact = Magnitude x Direction" that's sometimes used. You could probably think of “adoption” having mainly a “magnitude” component, and “effectiveness” having both a “magnitude” and a “direction” component. ↩︎
This seems quite clear for nonprofits in other EA cause areas and for startups and companies in the for-profit world, so we expect it to hold for AI Safety too. ↩︎
Or maybe the very idea of “there could exist things that would destroy the world" is so scary that your brain preemptively decides that nothing will be that bad. A sort of self-comforting heuristic of “I admit we theoretically might die from AIx but it’s unlikely and we’ll probably be fine, because I find it too scary to actually consider the possibility and think through the arguments.” But that’s beyond the scope of this sequence. ↩︎
Of course, it’s uncertain when exactly this IPO will happen, how quickly money will start flowing, and to where. But all in all, it seems like a decent time to start an AI Safety nonprofit, build some track-record, and position yourself to give the field a major boost by channeling the Anthropic IPO donations to something worthwhile. Besides, regardless of funding: the world could really use your help right now. ↩︎
Your project might be failing without you even knowing it.
It’s hard to save the world. If you’re launching a new AI Safety project, this sequence helps you avoid common pitfalls.
Your most likely failure modes along the way:
You never get started. Entrepreneurship is uncomfortable, and AI Safety is complex. There are many failure modes. It’s hard to figure out how to do something useful, so many people never try. Simultaneously, many things in AI Safety that obviously should be done are not done yet. You might mistakenly assume they’re already being done and therefore not try them.
You move slowly, fail to gain traction, and get stuck. You launch a tool – six months later, you have 12 users. You keep adding features, hoping something will change. Nothing does.
You satisfice. You figure that what you’re doing must be high impact, just because it’s part of AI Safety. In reality, you could be having 10x more impact than you are.
You fail without realizing it. You have users, citations, participants. But you're not actually reducing x-risk, and you don't realize it because you're not tracking your impact or not tracking the right things.
To set yourself up for success, you need two broad skillsets: entrepreneurial skills and impact-specific skills, which include impact estimation and strategic understanding of AI Safety. Mastering both skillsets is rare – we for sure know we haven’t.[1]
Impact = Adoption × Effectiveness
To have impact, you need to:
Think of it roughly as: Impact = Adoption × Effectiveness[3]
You'll require competent execution to build anything at all. But that's not enough. You need to find the right thing to build. Something that leads to positive impact.
The Two Impact Multipliers
These two skillsets help you create something impactful:
1. Entrepreneurial skills help you iterate toward something people actually engage with.
2. Impact skills guide you to the effectiveness that translates adoption into impact. This is tricky because you can’t directly measure whether you reduced AI x-risk until it’s too late to act on that information. Nevertheless, we’ll introduce the skills that will help you increase your odds.
Together, these help you understand the problems in AI Safety, prioritize between them, and develop solutions that work.
Most Projects Won’t Matter. Yours Could.
Your work isn’t automatically high-impact just because it’s part of AI Safety.
Impact follows something like a power law.[4]A small number of projects will create most of the impact.
So by default, your project will probably have little to none, or even negative, impact. But it also means your project could be one of the few making a real difference if you optimize deliberately.[5]
Would you help kill every single human on earth?
Of course not! You’re not a terrorist.
And yet we fear that this sequence of posts will enable you to accidentally move in that direction. There are several reasons for this:
For now, we want to leave you with a consideration about your emotions and motivated reasoning:
When really not wanting to do something bad makes you more likely to do that bad thing
We know you don’t want to be responsible for the extinction of humanity, the end of nature, and everything we ever cared about. It’s so trivially true that it’s almost a silly thing to point out.
But we wonder: perhaps you so badly don’t want to be responsible for such terrible things that your brain won’t even allow you to entertain the possibility that, unknowingly and inadvertently, you could be contributing to such bad outcomes.
Perhaps then, faced with such an unfathomably scary proposition, your brain preemptively decides that your work is good. Without having honestly assessed it. That’s motivated reasoning.[6]
Basically: You really don’t want to destroy the world → Your brain won’t allow you to honestly assess how your work might contribute to destroying the world → You can’t make fully-informed decisions to prevent unintended outcomes → You’re more likely to be contributing to the end of the world.
One method to overcome such pre-emptive opinion-making is by using visualizations to process the emotions of a bad outcome before deciding whether the outcome is likely, as proposed in Leave a Line of Retreat.
You can do this
Nobody has really figured out AI Safety yet – whether that’s technical safety, governance, or fieldbuilding. That means you can get up to speed quite quickly, bring new ideas, and move the field forward. You’re not late. You can make a real change.
Funding for AI Safety nonprofits may also increase soon. Anthropic’s cofounders and a number of employees have pledged to donate large portions of their wealth, which is expected to become available at an Anthropic IPO later in 2026. Part of that money may already come available sooner due to an ongoing pre-IPO share sale.[7]
Next Up: Post 2 – Entrepreneurial Skills
Many AI Safety projects fail by getting stuck. Post 2 shows you how successful entrepreneurs iterate toward adoption. Adapted to the context of AI Safety.
Already know how to build things people love? Skip to post 3 about Impact.
Edit: Post 2 is out now and the other posts will be released in the weeks to come. Add your email below to get notified when the next post comes out:
This sequence recombines existing ideas and creates some scaffolding around it. We hope/expect this will be useful for many people, and a similar style is used in many non-fiction books. This sequence is the result of ~80 hours of thinking, reading, and writing, and 30 conversations with experts and peers, though the resulting text remains a distillation of our own understanding rather than the result of quantitative study. If you disagree, you may have good reasons and we'd love to hear them. Moreover, if something isn’t useful to you in your specific situation, don’t use it. ↩︎
Read this as "Effectiveness ≡ Impact per unit of adoption". If you can think of a better word to describe this than 'effectiveness', let us know and we will change it. ↩︎
This is somewhat related to the framing of "Impact = Magnitude x Direction" that's sometimes used. You could probably think of “adoption” having mainly a “magnitude” component, and “effectiveness” having both a “magnitude” and a “direction” component. ↩︎
This seems quite clear for nonprofits in other EA cause areas and for startups and companies in the for-profit world, so we expect it to hold for AI Safety too. ↩︎
You'll need some luck, too. ↩︎
Or maybe the very idea of “there could exist things that would destroy the world" is so scary that your brain preemptively decides that nothing will be that bad. A sort of self-comforting heuristic of “I admit we theoretically might die from AIx but it’s unlikely and we’ll probably be fine, because I find it too scary to actually consider the possibility and think through the arguments.” But that’s beyond the scope of this sequence. ↩︎
Of course, it’s uncertain when exactly this IPO will happen, how quickly money will start flowing, and to where. But all in all, it seems like a decent time to start an AI Safety nonprofit, build some track-record, and position yourself to give the field a major boost by channeling the Anthropic IPO donations to something worthwhile. Besides, regardless of funding: the world could really use your help right now. ↩︎