Fantastic post, particularly interesting to me as since last December I decided to pivot and work on AI safety (it seems the most useful thing to do). In the spirit of sharing different perspectives, I will also share what I tried and failed at doing since then, so someone else might avoid my failures and recognize more easily if your suggested plan (hobbies while in stable job) might not work for them (as it didn't for me).
The TL-DR: My job drains too much energy from me to be productive on AI safety in my spare time (I tried and failed), nor can I easily get another job with money and worklife-balance. Thus, I prefer quitting my job and doing AI safety full time with no funding, until either I can get paid doing that, or I have to return to a job (but at least I will have produced something instead of nothing).
More context : I'm a software engineer doing backend development (code interacting with databases) in a startup. My strongest passions are game theory, psychology, education, ethics. If there was no urgency to AGI as x-risk, I would work on improving education and ethics (same vibe as the sequences wanting to train rationalists). I'm recently graduated from engineering school and have been working for close to 2 years with a decent salary. I discovered late in my studies that I don't like working as a programmer (because https://www.lesswrong.com/posts/x6Kv7nxKHfLGtPJej/the-topic-is-not-the-content). I have a personality where it seems I can only do good work on things that interest me (this is normal to an extent for everyone, I'm pointing it's stronger than normal for me to the point I often cannot do mental work on something which doesn't interest me (even though I can force myself to do physical work)).
So my diploma and experience are for a job I don't enjoy and which drains me. The usual circumstances of working for a company (working around certain time slots, optimizing your work for company profit over utility) are also a burden. It was in this context I decided last December to try pivoting into AI alignment research and I was also fortunate to have regular discussions with one AI alignment researcher on my topic, what I wanted to learn and write (he deserves credit for that effort and I'm only not mentioning him because my failure is my own). I found a couple papers I enjoyed, chose one to analyze in detail, and chose the goal of publishing a short LW article doing some further work on that paper. I totally failed doing that. The full list of reasons is of course diverse and complicated, but the main one is I put little time into it, and the time I had was low energy. I could follow that rate for a year and produce almost no value. Other reason for failure : I mistakenly chose a paper which wasn't centrally about AI safety, so the work I produced was pretty useless for solving the actual problems of AI safety.
So I abandoned that plan. Instead, I negotiated a raise in salary, kept my living expenses low and saved money. I'm now at the point where I can quit my job (and am planning to do so soon) and actually try AI safety research in suitable conditions, with the only caveat being limited time. Doing this, I hope to know much better how good a personal fit I have to alignment research and to other related work (governance, pedagogy) and choose the rest of my path accordingly.
In general, I would advise trying the hobby path first, while staying aware it won't work for everyone (a failure in being productive in something as a hobby doesn't mean you won't be good at it as a job). If it doesn't work, you have several others options as presented in the post, and I contribute the one I am currently doing : gain enough money to get some real free time (probably at least 3 months, 6 should be enough, probably no point to more than 12) and try doing that thing you wanted to do.
Formulating actionable research projects with predictably legible results takes some fluency in a topic. This means a mentor who would do this for you (adjusting as you progress), unusual luck, unusual talent, or studying for at least a few years to figure out what's going on. In hindsight, both mentor-guided research projects and self-directed study projects can turn out to be appallingly inefficient uses of time, so some care in picking these might go a long way.
It seems to me that "hobbies" of the kind you're describing are similar to the "founding" role that is the vaunted holy grail of EA's call for direct work. You're recognizing that part of "founding" is creating the slack for yourself to explore neglected issues and ideas, to turn them into a concrete project. The necessity of investing your own time and money to do this until somebody else is willing to pay you to continue is a big reason why these issues and ideas are neglected in the first place.
Maybe. But a person following up on threads in their leisure time, and letting the threads slowly congeal until they turn out to turn into a hobby, is usually letting their interests lead them initially without worrying too much about "whether it's going anywhere," whereas when people try to "found" something they're often trying to make it big, trying to make it something that will be scalable and defensible. I like that this post is giving credit to the first process, which IMO has been historically pretty useful pretty often. I'd also point to the old tradition of "gentlemen scientists" back before the era of publicly funded science, who performed very well per capita; I would guess that high performance was at least partly because there was more low-hanging fruit back then, but my personal guess is that that wasn't the only cause.
Many, though not all, of the "gentlemen scientists" were an intensely competitive bunch. They didn't typically have to to scale and defend their discoveries by building large organizations, because they were producing scientific knowledge. Their interests were guided by the interests of their contemporaries, or by the pressing issues of their day, as well as their own enthusiasms.
For example, Joseph Montgolfier started building parachutes around age 35. About 7 years later:
... he was watching a fire one evening while contemplating one of the great military issues of the day—an assault on the fortress of Gibraltar, which had proved impregnable from both sea and land. Joseph mused on the possibility of an air assault using troops lifted by the same force that was lifting the embers from the fire. He believed that the smoke itself was the buoyant part and contained within it a special gas, which he called "Montgolfier Gas", with a special property he called levity, which is why he preferred smoldering fuel.
Joseph then built a box-like chamber 1×1×1.3 m (3 ft by 3 ft (0.91 m) by 4 ft) out of very thin wood, and covered the sides and top with lightweight taffeta cloth. He crumpled and lit some paper under the bottom of the box. The contraption quickly lifted off its stand and collided with the ceiling.
Joseph recruited his brother to balloon building by writing, "Get in a supply of taffeta and of cordage, quickly, and you will see one of the most astonishing sights in the world." The two brothers built a similar device, scaled up by three (so 27 times greater in volume). On 14 December 1782 they did their very first test flight, lighting with wool and hay, and the lifting force was so great, that they lost control of their craft. The device floated nearly two kilometers (about 1.2 mi) and was destroyed after landing by the "indiscretion" of passersby.
I think the example of the Montgolfier brothers is one in which a combination of slack-fueled hobbies, pursued openly, but with consideration for how they might make a big difference in the world, leads organically to scaling up if and when the time is right.
Darwin's voyage on the HMS Beagle likewise had a mixture of open-ended curiosity and ambition:
He read John Herschel's new book, Preliminary Discourse on the Study of Natural Philosophy (1831), which described the highest aim of natural philosophy as understanding such laws through inductive reasoning based on observation, and Alexander von Humboldt's Personal Narrative of scientific travels in 1799–1804. Inspired with "a burning zeal" to contribute, Darwin planned to visit Tenerife with some classmates after graduation to study natural history in the tropics. In preparation, he joined Adam Sedgwick's geology course, then on 4 August travelled with him to spend a fortnight mapping strata in Wales.
After leaving Sedgwick in Wales, Darwin spent a few days with student friends at Barmouth. He returned home on 29 August to find a letter from Henslow proposing him as a suitable (if unfinished) naturalist for a self-funded supernumerary place on HMS Beagle with captain Robert FitzRoy, a position for a gentleman rather than "a mere collector". The ship was to leave in four weeks on an expedition to chart the coastline of South America. Robert Darwin objected to his son's planned two-year voyage, regarding it as a waste of time, but was persuaded by his brother-in-law, Josiah Wedgwood II, to agree to (and fund) his son's participation. Darwin took care to remain in a private capacity to retain control over his collection, intending it for a major scientific institution.
Darwin couldn't have anticipated exactly what he would discover, but it seems like he was laying the groundwork to accomplish something big with his time on Beagle, and was connecting with the major scientific and philosophical isssues of his day.
What seems crucial here is that Joseph Montgolfier and Charles Darwin weren't hurrying to scale up the first or most obvious idea they had. They weren't in a big hurry to "found." Instead, it seems like they recognized a theme (flight, naturalism) where they had some interest, aptitude, and connections, and explored it in an open-ended way. They thought about the mechanics (how to float a box to the ceiling with heat or slow down your fall with a parachute, or naturalistic observations), but also what their passion might mean for the world (military adventures in Gibraltar, or natural and theological theories).
As it happened, Montgolfier and Darwin both eventually hit on an intersection between the mechanics of their discipline and the interests and issues of the time. When that happened, they did put a lot of their energy into a "founding" effort - Darwin's book, Montgolfier's balloon.
So if we're thinking about "gentlemen scientists," we're not talking about a solitary exploration of one's private and idiosyncratic interests. We're talking about a passionate engagement with the issues of the day, which heavily informed what those interests were in the first place. This in combination with the natural enthusiasms and resources available to the person in question seems to have guided their path.
Great post! I've been mentioning for years that volunteering can be an effective way of making a contribution. Though many people think of volunteering as for a specific organization, I don't think it has to be, so a hobby could be an example. I think there are not enough volunteer opportunities in EA, and we've worked hard at ALLFED on our volunteer program. Not only have we had dozens of volunteers skill up, but they have also made significant contributions, often co-authoring journal articles and becoming full time staff. Thanks for the shout out! I'm actually still volunteering for ALLFED (and donating).
This seems to nudge people in a generally good direction.
But the emphasis on slack seems somewhat overdone.
My impression is that people who accomplish the most typically have had small to moderate amounts of slack. They made good use of their time by prioritizing their exploration of neglected questions well. That might create the impression of much slack, but I don't see slack as a good description of the cause.
One of my earliest memories of Eliezer is him writing something to the effect that he didn't have time to be a teenager (probably on the Extropians list, but I haven't found it).
I don't like the way you classify your approach as an alternative to direct work. I prefer to think of it as a typical way to get into direct work.
I've heard a couple of people mention recently that AI safety is constrained by the shortage of mentors for PhD theses. That seems wrong. I hope people don't treat a PhD as a standard path to direct work.
I also endorse Anna's related comments here.
There are investments you can’t make from a structured, nine-to-five, narrowly teleological environment. ... The best search strategies for complex problems like life generally don’t seek out particular homogeneous objectives, but interesting novelty. The search space is too complicated and unknown for linear objective-chasing to work. ... you cannot pursue interesting novelty—things that no one else is doing or which you have never seen before, or the little threads of nagging curiosity or doubt—by chasing along known direct value gradients. But that’s where the treasure is.
+1, I think this is a great perspective. Everything I've written on Less Wrong was because I decided it was better to work full time in software engineering and do academic-like stuff on the side rather than the other way around. As a result I did a bunch of work on AI safety among other things (whether it amounts to much is a separate question). I think this is a pretty important and viable career path, and I'm sort of surprised I hadn't seen a post calling it out explicitly before, especially not one written by me since I've been doing the same for so long!
Curated. This is a worthwhile message to get out there. Many people are feeling urgency these days with regards to AI, and if not directly themselves, the social community applies a lot of pressure on people to be "doing something", and I suspect this pressure detracts from the mental Slack to think, which I in turn suspect is important for people noticing new angles on the problems. Just being able to think about things freely without requirement to "do your job" or "justify a grant" seems pretty valuable, so I hope more people consider this.
Idea - set up a “slack Slack” where people doing (or wanting to do) impactful / interesting side projects can find collaborators / share ideas and advice / keep each other accountable
a guy who spent 15 years building up a top-notch physics expertise that is now completely irrelevant for my life.
By the way, I think that the physics perspective ("thinking like a physicist") is under-represented and under-appreciated in the AGI safety community.
Edit: to extend this thought a little further: I think the philosophical perspective is over-represented and adequately appreciated. The mathematical perspective is either under-represented and adequately appreciated, or adequately represented and over-appreciated (I'm not sure). The engineering/empirical/interpretability perspective is adequately or under-represented (we probably should experiment more with the SoTA models that are already available and try to interpret them better, even though this might give the sense to their creators that they did something useful for alignment in publishing these models and their act was not harmful on balance) and adequately appreciated.
Thank you for a valuable post and for the interesting links provided. I am sure you are right that much value comes from individuals pursuing their passion for EA in their own time.
My experience has been that the time available for independent intellectual or altruistic pursuits varies over a lifetime. When demands from children and career peak, it is hard, but I recommend keeping jotting down ideas, using commuting time for reading or listening and trying to stop work absorbing all your thinking capacity. Later, career changes or (as in my case) early retirement can provide slack and the opportunity to do more.
This post has been recorded as part of the LessWrong Curated Podcast, and can be listened to on Spotify, Apple Podcasts, and Libsyn.
(Also posted on EA Forum)
Introduction
In EA orthodoxy, if you're really serious about EA, the three alternatives that people most often seem to talk about are
(1) “direct work” in a job that furthers a very important cause;
(2) “earning to give”;
(3) earning “career capital” that will help you do those things in the future, e.g. by getting a PhD or teaching yourself ML.
By contrast, there’s not much talk of:
(4) being in a job / situation where you have extra time and energy and freedom to explore things that seem interesting and important.
But that last one is really important!
Examples
For example, here are a bunch of things off the top of my head that look like neither “direct work” nor “earning-to-give” nor “earning career capital”:
Three ingredients to a transformative hobby
Ingredient 1: Extra time / energy / slack
Honestly, I wasn’t really sure whether to put it on the list at all. Scott Alexander famously did some of his best writing during a medical residency—not exactly a stage of life where one has a lot of extra free time. (See his discussion here.) Another excellent blogger / thinker, Zvi Mowshowitz, has been squeezing his blogging / thinking into his life as a pre-launch startup founder and parent.
Or maybe those examples just illustrate that, within the “time / energy / slack” entry, “time” is a less important component than one might think. As they say, “if you want something done, ask a busy person to do it”. (Well, within limits—obviously, as free time approaches literally zero, hobbies approach zero as well.)
Note a surprising corollary to this ingredient: “direct work” (in the EA sense) and transformative hobbies can potentially work at cross-purposes! For example, at my last job, I was sometimes working on lidar for self-driving cars, and sometimes working on military navigation algorithms,[1] and meanwhile I was working on AGI safety as a hobby (more on which below). Now, I really want there to be self-driving cars ASAP. I think they’re going to save lots of lives. They’ll certainly save me a lot of anguish as a parent! And we had a really great technical approach to automobile lidar—better than anything else out there, I still think. And (at certain times) I felt that the project would live or die depending on how hard I worked to come up with brilliant solutions to our various technical challenges. So during the periods when I was working on the lidar project, and I had extra time at night, or was thinking in the shower, I was thinking about lidar. And thus my AGI safety hobby progressed slower. By contrast—well, I have complicated opinions about military navigation algorithms, but let’s just say that they have never aroused in me a great passion. So during the periods when I was working on military navigation algorithms at my day job, and I had free time at night, or was thinking in the shower, I was thinking about AGI safety instead, and I made faster progress! (See Paul Graham’s essay “The Top Idea in your Mind”.)
Ingredient 2: ???
Here I’m referring to the fact that lots of people have extra time / energy / slack, and don’t use it for any world-changing hobbies. Instead, umm, I don’t know, maybe they watch a lot of TV, or argue about politics online, or host fancy parties, or build model ships, or whatever. (I'm not criticizing people who do those things. Your time is your time. Spend it as you wish!)
I don’t know what accounts for the difference. Certain types of motivation and skills and interests, I guess?
(I’m reminded of the book quote: “When you are older, you will learn that the first and foremost thing which any ordinary person does is nothing.”)
Ingredient 3: Willingness to pivot
As I understand it, Eliezer Yudkowsky was really into thinking about nanotechnology before deciding that actually thinking about AGI was a much better use of his free time, and then later pivoted again to thinking more specifically about AGI safety / alignment. Nate Soares likewise describes pivoting to AGI safety after a decade thinking about governance and institutions. I don’t know what David Denkenberger was thinking about in his free time before he came upon the question of feeding the world through nuclear winter, but I bet it was very interesting and different! For my part, AGI safety was the 5th (!!) long-term (i.e. multi-year) intense ambitious hobby of my life. (And now it’s my job.)
I received the following comment on a draft version of this post:
Here’s my message to “people who are devoting huge amounts of time+effort to side projects that are going nowhere”: Don’t do that, OK? I mean, jeez. One of the key selling points of hobbies is that if you think maybe you should pivot and start from scratch on something totally different, you can just up and do it! There’s nothing stopping you! You don’t owe anyone anything! You have to kill your darlings! Maybe you learned something along the way, or at least had fun, and life is long, go start something new, etc. etc. That’s my motivational speech, thanks for listening. Take it from me, a guy who spent 15 years building up a top-notch physics expertise that is now completely irrelevant for my life.
How can hobbies compete at all with jobs? My theory: compromises are terrible
Naïvely, if we’re going to compare “doing stuff as one’s job” versus “doing stuff as one’s hobby”, the former seems to have overwhelming advantages: jobs brings to bear heavy artillery in the form of time, energy, money to spend on resources, multiple people coordinating, and so on. And don’t get me wrong, jobs make the world go round. Nobody is manufacturing integrated circuits in their free time.
It’s no coincidence that David Denkenberger and Eliezer Yudkowsky and Holden Karnofsky (and me!) transitioned their hobbies into proper jobs approximately as soon as possible (more on which just below).
But we still have to explain how important things get done at first as hobbies, not jobs. I think the answer is: compromises are terrible.
If you need to simultaneously satisfy two criteria, i.e. “this is an important thing to do” and “this is something I can immediately get paid to do”, then there’s a tradeoff. You need to compromise on both.
I said above that jobs bring to bear “heavy artillery” that hobbies can’t. But hobbies can make up for that deficiency with better aim.
(For more on the theme of compromises when trying to satisfy multiple criteria at once, see for example Eliezer Yudkowsky’s Purchase Fuzzies and Utilons Separately or Alex Lawsen’s Know What You’re Optimizing For. For more on the advantages of doing things unconstrained by institutional incentives and inertia, see Paul Graham’s The Power of the Marginal, and Eliezer Yudkowsky’s book Inadequate Equilibria.)
Hobbies can eventually develop into jobs
A hopeful possibility, of course, is that the hobby can eventually lay the groundwork for the creation of a new job, one whose money- and status-related incentives are all perfectly aligned with the thing that you have now figured out is what you want to do. Then you get the best of both worlds—the aim of the hobby, and the heavy artillery (time, resources, etc.) of a job.
But you can’t get there until you know what you’re aiming for.
My own two stories
These are a couple anecdotes from my own life; feel free to skip down to the Conclusion if you're uninterested.
Story #1: Me & solar cells
Back in my youth, I was passionate about inventing better solar cells. It was the reason I went into physics in the first place as an undergrad in 2003, and then I deliberately went to a physics grad school where there was an unusually high concentration of solar cell research happening.
How'd it go? I had some outputs in the "direct work" category (the thing I was “supposed to be doing” as a grad student), and also I had some outputs in the “slack / hobby” category (stuff I did informally, independent of my grad school advisor, unrelated to my eventual dissertation, etc.)
Shall we compare?
In the “direct work” category:
This category is where I was trying to compromise between
Basically, I wound up doing decently at the latter and pretty much entirely failing at the former.
I worked in four labs:
In the “hobbies” category:
Winner: Hobbies
I can say with some confidence that I have contributed a little bit to help bring cheaper and better solar cells into the world, exclusively via projects that I did in my free time, that my grad-school PI didn’t even know about, and that probably would have done either nothing or almost-nothing to help me advance in academia.
Also, among my more standard legible grad-student-y projects, I had one near-miss which I think could have been practically useful (a priori). And wouldn’t you know it, it was the one project that I initiated and developed in my free time (before eventually looping in my PI).
Story #2: Me & AGI safety
In 2019, having finished some other hobby, I decided my next hobby would be AGI safety. So I started reading and writing posts and comments on LessWrong, in my little bits of time squeezed between my full-time job and two young kids.
Of course, I found it very frustrating how little time I had, and I immediately started brainstorming how I could get much more time by doing AGI safety as a job.
The normal advice would be to apply for existing AGI safety jobs, or go back to grad school etc. But I had high living expenses and didn’t want to move out of Boston. Alternatively, I managed to come up with a handful of possible projects that I might do within my existing job, that seemed at least slightly relevant to AGI safety. For example, I could have tried to talk my way onto my coworkers’ existing DARPA-funded project related to machine learning uncertainty, or various other things. But I wound up dropping those ideas too.
So nothing came of any of that, and I’m glad it didn’t, because my current belief is that I did dramatically better by just blogging and commenting on LessWrong in my tiny amount of free time, than I would have done if I had successfully found a way to do AGI safety in an institutional setting right off the bat. I wound up going off in a weird intriguing direction, and then losing interest, and then going off in a different weird intriguing direction, which incidentally involved teaching myself neuroscience, and I wound up making enough progress to get a grant to do full-time independent research, and here I am! I think I’m doing something important and maybe a bit idiosyncratic, and it’s all thanks to having spent a significant amount of time just reading and writing about whatever the hell I wanted.
Conclusion
I was inspired to write this post right after talking to (what felt like) 3000 super-enthusiastic undergrads at EAGxBoston a couple months ago (I love you all!), who all wanted to get into AGI safety / alignment.
Among the people I talked to, there seemed to be an unspoken assumption that, for example, if they go to grad school at all, they should try to maneuver into doing AGI-safety-related projects with an AGI-safety-concerned PI, and if they did so, victory!! Let’s be clear: I’m not opposed to doing that! Some people should definitely be doing that! Great work has been done that way, e.g. Alex Turner’s dissertation. But let’s not pretend that this isn’t a compromise. You’ll wind up in a project which is simultaneously optimizing for both advancing AGI safety and easy consumption by peer-reviewers, by your PI’s future grant review committees, by your own dissertation committee, by future people reading your CV, etc.
(It’s no coincidence that the “weird sci-fi stuff” side of AGI safety is almost entirely absent from academia.)
Apart from grad school, other frequently-proposed plans were: joining existing AGI safety nonprofits, joining existing mentorship programs, and getting good at ML (or neuroscience or whatever) in normal legible ways like “taking online courses” or “getting jobs in that field”. Those are all good things too! I’m a huge fan of all those things!
But not many people brought up the idea of just having any job whatsoever, ideally a pleasant, invigorating job that you’re really good at and which has good work-life balance,[2] and meanwhile making a plan in your free time for how to solve AGI safety. (Or if that’s too ambitious, maybe “write intelligent comments on people’s AGI-safety-related blog posts”, and/or “create AGI-safety-related pedagogy”, and work your way up from there—certainly that’s all I was hoping for at first!) And only then, when and if you come up with a plan, you can apply for funding, either for independent research, or starting a new organization, or joining up with other people who share your vision, etc.[3]
Granted, that approach has its own issues. In some ways, it’s a terrible plan! But I think it at least deserves a modicum of consideration.
(As usual, consider reversing any advice!)
Thanks Alex Lawsen, Justis Mills, and Adam Shimi for critical comments on a draft.
(among dozens of other projects, but these two can serve as prototypical examples)
Or in the case of undergrads, I sometimes noticed perhaps a bit too much interest in filling up their résumé with the maximum number of research internships and good grades in hard classes etc., and perhaps a bit too little interest in having the mental space and energy to get sucked into thinking about interesting and important questions on their own time. [… Says the guy who spent his undergrad years filling up his résumé with the maximum number of research internships and good grades in hard classes etc. …]
Or maybe you’ll wind up feeling that AGI technical safety research isn’t the right thing for you to do at all, and off you go in some other direction!