I ran an AGISF (technical alignment track) program at the University of Michigan last semester. At the end of the program, I shared a "next steps" document full of ideas for actions to take after finishing the program, and I've refined the document into a public version that you can find here. Below are some relevant parts for people in a similar position.
Getting Feedback
I encourage you to apply for advising at 80,000 Hours (80k), a career advising service for people who want to find career paths that best improve the world. 80k views risks from advanced AI as one of the world’s most pressing problems. The advisors aren’t professional AI researchers, but they know a lot about AI safety and can help you think through your plans.
For general advice on technical alignment careers, including links to other career guides, see the Careers in alignment document (by Richard Ngo of OpenAI) from Week 7.
“All skill depends on practice. So too, with memory. If you want to remember something, you need to practice remembering it, not just looking at it.
Retrieval practice—where you shut the book and try to recall what you’ve learned without looking at it—is one of the most effective studying techniques.1 It beat fancier techniques like concept mapping in head-to-head trials.2 Retrieval also works even if you can’t get feedback.3”
You might be able to build skills through traditional routes like taking classes. I took Deep Learning for Computer Vision with Justin Johnson and it was the best EECS course I have ever taken. I highly recommend taking this class as your intro to machine learning.
You might also be able to build skills outside of school (e.g. with online courses, internships, textbooks, research programs, and other projects). Sometimes this is actually a more efficient way to learn advanced things you need. Here are some guides with suggestions:
You do not need to gain comprehensive background knowledge before you can start working on technical AI safety projects. One extreme plan is to learn everything before doing anything, and another extreme plan is to do something before learning anything (and learn the background knowledge you need as it comes up). You probably want to balance somewhere between these two extremes. Experiment to see what works for you!
If you want an even more regular digest than that, check out this link! It updates daily with new ML papers, although many of them won’t be directly related to AI safety.
Import AI is a weekly newsletter covering the significance of recent AI progress
aisafety.community tracks many online groups you can join to connect with people from around the world who care about AI safety. Email me if any of the links are dead and I’ll get you a link that works.
aisafety.video has many channels and podcasts you can follow for more AI safety content.
AI safety organizations
In my view, humanity could use a lot more time to prepare for advanced AI. As such, I oppose most work that significantly reduces the time we have for solving problems, including some work at top AI labs.Even some jobs that contribute to AI safety can substantially reduce the time we have (and, in general, I think that any field can have some seemingly benign jobs that actually cause substantial harm), so it’s important to be cautious and think carefully about the impact of a given role. There are, of course, tradeoffs to consider: see the “Safety Capabilities Balance” section of “X-Risk Analysis for AI Research” and this anonymous advice from 80k.
Furthermore, some technical safety research is not very useful for AI safety, even if the researchers have good intentions. The organizations listed below are trying to reduce catastrophic risks from AI, but they are not necessarily accomplishing this. Again, it’s important to think critically about the effects and contributions of any role you’re considering. Don’t blindly apply to every job you see with “AI safety” in the description.
Here is a big (but not comprehensive) list of organizations doing AI safety work, especially technical research. Inclusion on this list does not imply my endorsement. See also aisafety.world for more, as well as aisafety.world/map (still a work in progress; to discuss additions and improvements you can join the Alignment Ecosystem Development Discord server).
“Our mission is to reduce catastrophic and existential risks from artificial intelligence through technical research and advocacy of machine learning safety in the broader research community.” Also hosts the Intro to ML Safety program.
OpenAI and DeepMind are two of the leading groups for advancing AI capabilities, so I oppose some of the work they’re doing (see above). However, they also have people working on alignment/safety and on governance, and I do recommend those roles.
Anthropic “is an AI safety and research company that’s working to build reliable, interpretable, and steerable AI systems.” They are a top research lab, but some of their projects (like the Claude chatbot) seem to be accelerating progress towards advanced AI, so again, the situation is complicated and this is not a blanket recommendation for all roles at Anthropic.
I ran an AGISF (technical alignment track) program at the University of Michigan last semester. At the end of the program, I shared a "next steps" document full of ideas for actions to take after finishing the program, and I've refined the document into a public version that you can find here. Below are some relevant parts for people in a similar position.
Getting Feedback
I encourage you to apply for advising at 80,000 Hours (80k), a career advising service for people who want to find career paths that best improve the world. 80k views risks from advanced AI as one of the world’s most pressing problems. The advisors aren’t professional AI researchers, but they know a lot about AI safety and can help you think through your plans.
One more option is career coaching from AI Safety Support.
Advice and guides
For general advice on technical alignment careers, including links to other career guides, see the Careers in alignment document (by Richard Ngo of OpenAI) from Week 7.
Career opportunities
The AGI Safety Fundamentals Opportunities Board is an excellent resource for tracking jobs, internships, and other programs.
Getting started with academic research
The Student ML Safety Research Stipend Opportunity provides stipends for students doing empirical ML safety research.
Personal projects and exercises
“All skill depends on practice. So too, with memory. If you want to remember something, you need to practice remembering it, not just looking at it.
Retrieval practice—where you shut the book and try to recall what you’ve learned without looking at it—is one of the most effective studying techniques.1 It beat fancier techniques like concept mapping in head-to-head trials.2 Retrieval also works even if you can’t get feedback.3”
– Scott Young in The 10 Essential Strategies for Deeper Learning
See Technical AI safety exercises and projects for a list of possible projects and exercises to try. Also consider working on a project with a friend!
Upskilling
You might be able to build skills through traditional routes like taking classes. I took Deep Learning for Computer Vision with Justin Johnson and it was the best EECS course I have ever taken. I highly recommend taking this class as your intro to machine learning.
You might also be able to build skills outside of school (e.g. with online courses, internships, textbooks, research programs, and other projects). Sometimes this is actually a more efficient way to learn advanced things you need. Here are some guides with suggestions:
All these guides have some recommended online courses. I don’t think Andrew Ng's deep learning course is included anywhere, but it’s another good option. Also consider Coursera’s new course on Mathematics for Machine Learning and Data Science Specialization.
You do not need to gain comprehensive background knowledge before you can start working on technical AI safety projects. One extreme plan is to learn everything before doing anything, and another extreme plan is to do something before learning anything (and learn the background knowledge you need as it comes up). You probably want to balance somewhere between these two extremes. Experiment to see what works for you!
Following research + news
AI safety organizations
In my view, humanity could use a lot more time to prepare for advanced AI. As such, I oppose most work that significantly reduces the time we have for solving problems, including some work at top AI labs. Even some jobs that contribute to AI safety can substantially reduce the time we have (and, in general, I think that any field can have some seemingly benign jobs that actually cause substantial harm), so it’s important to be cautious and think carefully about the impact of a given role. There are, of course, tradeoffs to consider: see the “Safety Capabilities Balance” section of “X-Risk Analysis for AI Research” and this anonymous advice from 80k.
Furthermore, some technical safety research is not very useful for AI safety, even if the researchers have good intentions. The organizations listed below are trying to reduce catastrophic risks from AI, but they are not necessarily accomplishing this. Again, it’s important to think critically about the effects and contributions of any role you’re considering. Don’t blindly apply to every job you see with “AI safety” in the description.
Here is a big (but not comprehensive) list of organizations doing AI safety work, especially technical research. Inclusion on this list does not imply my endorsement. See also aisafety.world for more, as well as aisafety.world/map (still a work in progress; to discuss additions and improvements you can join the Alignment Ecosystem Development Discord server).