Alignment research careers: a first-year plan

Alignment research careers are open to more people than most guides suggest, but the roles differ a lot in what they ask of you. Some need a PhD and strong research taste, some need strong programming, and many need neither. This guide sets out the main roles, gives you a first-year plan in four quarters, and covers an open debate about how to start.

Everything below about roles and skills comes from guides written by people in the field, so read it as their view, dated, not as a rule.

What alignment research careers look like

Charlie Rogers-Smith's guide, How to pursue a career in technical AI alignment (June 2022), sorts technical alignment work into a few kinds of role:

• Research lead, theoretical. Sets the agenda for conceptual or mathematical work. He writes that a PhD is not required but is helpful, and that it takes "extremely strong epistemics and research taste".
• Research lead, empirical. Proposes and runs experiments on current machine learning systems. A PhD is not strictly required, but in practice most have one.
• Research contributor. Works on a team carrying out projects others propose. For empirical contributors, a PhD is not required and the key skill is "strong skill at programming". Theoretical contributor roles were, in his words, "pretty rare" when he wrote it.
• Professor. A research lead who also mentors graduate students and teaches. Here a PhD is required.

He is also clear that technical research is not the only path. You can have a large impact through strategy, governance, policy, security, forecasting, support roles, field-building or grant-making, and he argues it is probably better to do great work in one of those areas than mediocre technical work, "because impact is heavy-tailed". The aisafety.com jobs board makes the same point in one line: many roles don't require technical skills. If you are unsure how alignment and the wider field relate, our post on AI alignment vs AI safety sets out the difference.

Why engineers are in demand

In AI Safety Needs Great Engineers (November 2021), Andy Jones, then at Anthropic, wrote that their safety work was limited by "a shortage of great engineers", and that several other safety organisations felt the same. His reason: once models like GPT-3 existed, alignment gained an empirical side, and running experiments on real models needs a large stack of custom software.

He also wrote that at Anthropic there was no hard line between researchers and engineers. If you come from software, that matters: the contributor path may fit you better than trying to become a research lead first. The post is from 2021, so check current job listings before relying on it.

A first-year plan, quarter by quarter

Here is one way to spend your first year, built from the steps these guides recommend. Each quarter ends with something you can show another person, because shown work is what gets you useful feedback.

1. Months 1 to 3: the ideas. Learn the core problems and the main arguments for and against them. Start with what AI alignment is, then follow our AI safety self-study path. By the end, pick two topic groups that pull you in, such as interpretability or evaluations, and write a page of notes on each in your own words.
2. Months 4 to 6: the tools. If empirical work interests you, Rogers-Smith suggests learning basic deep learning, partly to test whether you enjoy it. If theory pulls you, build the maths for your topic group instead. Show: working code or worked proofs.
3. Months 7 to 9: one replication. Reproduce one foundational paper in your topic group. Rogers-Smith calls paper replications "essential for contributor roles, and useful for lead roles", and suggests showing your work by open-sourcing your code and maybe writing a blog post. He quotes DeepMind's rough test for its Research Engineer role: if you can reproduce a typical ML paper in a few hundred hours, they are probably interested in interviewing you. He also notes you can apply for funding to do replications.
4. Months 10 to 12: feedback and applications. Share the replication. Ask people to judge honestly whether you are on track, and, as he puts it, make it easy for them to tell you that you're not. Then apply. The aisafety.com pages for training programs, advisors and funding list fellowships, free guidance calls and grants, and its communities page recommends joining a few groups.

If your interest is governance or policy, keep the same shape and swap the replication for a written analysis of one policy question.

How to test your fit honestly

A first year is mostly an experiment on yourself. A few signals from the guides:

• Enjoyment. Do you keep going on the deep learning or the proofs when nobody is checking? That is the point of months 4 to 6.
• Output. Could you finish a replication, and how long did it take compared with the "few hundred hours" yardstick?
• Outside view. What do people further along say about your work? Rogers-Smith adds that "grades matter less than people think".

If the answers point away from research, that is useful too. As Rogers-Smith argues, great work on another path probably beats mediocre technical work.

The debate: take a general AI job first?

A common question is whether to build skills in general AI work, including work that makes AI more capable, before moving to safety. People in the field disagree.

• Be wary of it. In a 2018 note shared by Rob Bensinger, Andrew Critch says he is concerned about telling people who care about safety to go into capabilities research. His worry is that they will "start pretending" their work is relevant to safety, creating a false sense of security. He calls "do AI yourself, so you're around to help make it safe" a really bad meme for general career advice.
• It can be a stage. Critch makes an exception for very junior researchers using general AI research to improve their skills until they can contribute to safety. Rogers-Smith says it is fine, and maybe preferable, to do non-alignment research as an undergraduate.
• The line itself is contested. Rogers-Smith writes that it might not be good to become excellent if it means advancing AI capabilities, but notes that Anthropic believe staying at the frontier of capabilities is necessary for good alignment work. He says he does not know how coherent the capabilities-safety split is, and calls it "an active area of debate".

Frequently asked questions

Do I need a PhD for alignment research?

Not for every role. In Rogers-Smith's guide, empirical research contributors do not need one, theoretical research leads find it helpful but not required, most empirical research leads have one, and professors need one.

Can I work in AI safety without technical skills?

Yes. The aisafety.com jobs board says many roles don't require technical skills, and the guides list governance, policy, strategy, support roles and field-building as paths with real impact.

What is the best first project?

The guides point to replicating a foundational paper in a topic you care about. It builds skills, gives you something to show, and tests whether you enjoy the work.

How long does it take to get ready?

No source here gives a fixed time. The plan above uses a year so that you test your fit with real work before you apply.

Get started

Months 1 to 3 are where Learn AI Alignment Theory fits. It has 22 courses and 69 lessons of about 8 minutes, in three levels: Basic for the core ideas with no background needed, Intermediate for how today's training methods can go right or wrong, and Advanced for open research problems and live debates, including a course called Research Agendas, Theory and Careers. Its topics map follows the aisafety.com self-study topics, every lesson lists its sources, and you sign in with Google or an emailed code. The about page has the full overview.

Start Learn AI Alignment Theory: short hands-on lessons from first principles to open research, with every side of every debate and the sources to read next.