When I was 19, I decided I was going to save the world.
Now, 25 years later, I’ve decided to give up.
To explain why and what giving up means, we first need to go back to how I ever got into the world-saving business. As a kid, I loved computers, and I started teaching myself to program them when I was 8. By 16 I could write real programs that did interesting things, and it was also around this time I got into Linux. And since I was into Linux, that meant I read Slashdot.
If you weren’t into Linux in the late 90s, you might not even know what Slashdot was. It still exists, but it’s a shell of its former self. Back then it was a vibrant, community-driven news site that in many important ways felt to me like LessWrong feels now. And it was from Slashdot that I learned about nanotech and grey goo, which led me to the Foresight Institute, which led me to the Extropians, which ultimately led to me being a very early member of the SL4 mailing list.
And, again, if you’re not familiar with SL4, it’s approximately how all this AI safety and rationalist stuff got started. And so it was sometime over the summer of 2001, discussing Friendly AI with Eliezer Yudkowsky on IRC, that I decided to orient my life around saving the world from the risks of superintelligent AI.
The only problem was, I wasn’t up to the task yet. In fact, nobody was. In that first decade I, along with everyone else, was just trying to figure out how to think well enough to be able to think usefully about AI at all. It proved a surprisingly hard task, and one that required the founding of the rationalist community.
While all this was happening, I was also living my life. I stayed in school for a while trying for a math PhD, then dropped out, joined a startup, and got married. When the startup and the marriage failed, I took the chance to move to the Bay Area. This was in 2013, and the rationalist community was starting to come together in person. I moved into a rationalist group house in Berkeley, took a job with another startup, and on nights and weekends worked on my “hobby” of figuring out how to save the world.
Over time, I came up with some ideas. I wrote a lot about them. I still think some of them are pretty good ideas that are worth exploring more. They might even contribute to world saving. But by 2020 it was clear to me I was less interested in AI safety engineering (at least as it existed at that time) and more interested in philosophy, so I pivoted to writing Fundamental Uncertainty, a book about all the philosophy I’d had to work out to think clearly about AI. My hope was still that I’d save the world, but I’d do it by enabling other people to work effectively in AI safety. I finished that book earlier this year, and then the question became, what should I do next?
Life conspired for a while to keep me from answering that question. I suffered my fourth concussion in July and have had post-concussion syndrome since then. I’m close to being back to normal, but still not completely there. I also took on a larger role at work, and while I think what we’re doing is important, it’s obviously not saving the world from AI-induced catastrophe. So between concussion recovery and work, I’ve not had a lot of time to do anything extra, and in that time the world has also changed.
The timeline to superintelligence is looking short. At the moment, the most likely world-saving activities are in policy and politics, two areas I’ve long shied away from. I do what I can. I write and call my representatives. I engage on Twitter. I do my best to be a link in preference cascade chains that may lead to effective policy outcomes.
But between the changes in my life and changes in the world, I’ve had to accept that I’m not going to be the one to save the world, and it’s time I stopped trying. And that’s okay, because it’s not up to one person to save it. Reality isn’t like a Marvel movie or a fairytale. The future of the world doesn’t hang—or only does very rarely—on the actions of a single person. Instead, it’s the actions of many people working together that have saved the world. That’s how we managed to keep the ozone layer, stop acid rain, prevent nuclear war, engineer our way out of Y2K, and much else.
And so I think I can finally say, after spending more than half my life trying to save the world, that I’ve given up. Not on the world being saved, but on saving the world on my own.
I want to be clear that I do still think we can save the world, and that the world needs saving, specifically from the threat posed by superintelligent AI. It’s just that such saving is unlikely to happen because one person did one thing, but instead because many people did many things to avert myriad potential catastrophes. So although I’m giving up on saving the world, I’m not giving up on the world being saved, and I’ll do what I can to contribute to that effort.
My own spin on fatalism (in various other situations as well) is that if nothing I can do matters[1], it also doesn't matter if I give up. So I might as well keep taking the steps in the right direction (without expecting an effect), rather than in some other direction. The main distinction is doing something full time vs. persistently in the background (or at the prompting of external events), because of the effects on motivation and the frictions of not being slotted into a bigger organized effort.
The first filter is when the things I can do don't matter to any emotionally relevant degree. At that point, some hair-splitting impact estimates might still persist. When they also fail, some deontological principles can persist, gesturing at idealized equilibria where everyone acts similarly, even though in actuality they clearly don't. ↩︎
When I was 19, I decided I was going to save the world.
Now, 25 years later, I’ve decided to give up.
To explain why and what giving up means, we first need to go back to how I ever got into the world-saving business. As a kid, I loved computers, and I started teaching myself to program them when I was 8. By 16 I could write real programs that did interesting things, and it was also around this time I got into Linux. And since I was into Linux, that meant I read Slashdot.
If you weren’t into Linux in the late 90s, you might not even know what Slashdot was. It still exists, but it’s a shell of its former self. Back then it was a vibrant, community-driven news site that in many important ways felt to me like LessWrong feels now. And it was from Slashdot that I learned about nanotech and grey goo, which led me to the Foresight Institute, which led me to the Extropians, which ultimately led to me being a very early member of the SL4 mailing list.
And, again, if you’re not familiar with SL4, it’s approximately how all this AI safety and rationalist stuff got started. And so it was sometime over the summer of 2001, discussing Friendly AI with Eliezer Yudkowsky on IRC, that I decided to orient my life around saving the world from the risks of superintelligent AI.
The only problem was, I wasn’t up to the task yet. In fact, nobody was. In that first decade I, along with everyone else, was just trying to figure out how to think well enough to be able to think usefully about AI at all. It proved a surprisingly hard task, and one that required the founding of the rationalist community.
While all this was happening, I was also living my life. I stayed in school for a while trying for a math PhD, then dropped out, joined a startup, and got married. When the startup and the marriage failed, I took the chance to move to the Bay Area. This was in 2013, and the rationalist community was starting to come together in person. I moved into a rationalist group house in Berkeley, took a job with another startup, and on nights and weekends worked on my “hobby” of figuring out how to save the world.
Over time, I came up with some ideas. I wrote a lot about them. I still think some of them are pretty good ideas that are worth exploring more. They might even contribute to world saving. But by 2020 it was clear to me I was less interested in AI safety engineering (at least as it existed at that time) and more interested in philosophy, so I pivoted to writing Fundamental Uncertainty, a book about all the philosophy I’d had to work out to think clearly about AI. My hope was still that I’d save the world, but I’d do it by enabling other people to work effectively in AI safety. I finished that book earlier this year, and then the question became, what should I do next?
Life conspired for a while to keep me from answering that question. I suffered my fourth concussion in July and have had post-concussion syndrome since then. I’m close to being back to normal, but still not completely there. I also took on a larger role at work, and while I think what we’re doing is important, it’s obviously not saving the world from AI-induced catastrophe. So between concussion recovery and work, I’ve not had a lot of time to do anything extra, and in that time the world has also changed.
The timeline to superintelligence is looking short. At the moment, the most likely world-saving activities are in policy and politics, two areas I’ve long shied away from. I do what I can. I write and call my representatives. I engage on Twitter. I do my best to be a link in preference cascade chains that may lead to effective policy outcomes.
But between the changes in my life and changes in the world, I’ve had to accept that I’m not going to be the one to save the world, and it’s time I stopped trying. And that’s okay, because it’s not up to one person to save it. Reality isn’t like a Marvel movie or a fairytale. The future of the world doesn’t hang—or only does very rarely—on the actions of a single person. Instead, it’s the actions of many people working together that have saved the world. That’s how we managed to keep the ozone layer, stop acid rain, prevent nuclear war, engineer our way out of Y2K, and much else.
And so I think I can finally say, after spending more than half my life trying to save the world, that I’ve given up. Not on the world being saved, but on saving the world on my own.
I want to be clear that I do still think we can save the world, and that the world needs saving, specifically from the threat posed by superintelligent AI. It’s just that such saving is unlikely to happen because one person did one thing, but instead because many people did many things to avert myriad potential catastrophes. So although I’m giving up on saving the world, I’m not giving up on the world being saved, and I’ll do what I can to contribute to that effort.