AI safety has a salary overhang. A bunch of people are paid way below their market rate. (Also true of EA, but I’ll lump them under “AI safety” for this piece).
This has been getting a bit better, very recently — I hear about eg Redwood Research technical talent making ~$800k/y in salary. But when you compare this to the $10-100m/y comp packages for talent in labs, it still seems kind of low. And CG’s money cannon has fired very few shots to date — maybe ~15 orgs hit, see Trace — so the raises are still very unequally distributed.
It’s not just rank-and-file employees, by the way. Even the leadership is grossly underpaid. The numbers I hear range from $150k - $600k/y, for execs at top orgs like METR, MATS, FAR.AI, Constellation. Like, c’mon, I made $250k/y total comp as a undistinguished new grad software engineer in 2016 Google. For the leaders of preeminent safety orgs, surely we can do better than that.
If I were a conspiracy theorist, I might say that “accepting lower salaries” is a psyop from funders.
Low salaries mean that employees and execs remain beholden to funders, rather than having the independence to work on what makes their heart sing. For me, hitting FIRE early in my tech career allowed me to quit my job and work on side projects and startups. To this day, I feel more independent from the “AI safety hivemind” because my paycheck doesn’t depend on it.
Paying low salaries is also penny-wise, pound-foolish from the perspective of funders & orgs. It means that great talent ends up choosing not to work in safety roles. A lot of the safety brain drain is because the labs are bidding much higher.
Also, funder <> org <> employee values are much more aligned within AI safety than in the wider world. Much of what these employees might buy with extra money are things that would be good by the lights of the funder & the org — eg donating to charities or other important causes. (Orgs could offer large donation matches in lieu of cash to enforce this, though I think donation matches have their own problems.)
In practice, one big problem might be that the employees themselves don’t really know what they’re worth. My kingdom for a levels.fyi of AI safety.
AI safety people like to point out that frontier lab employees currently have some leverage over their leaders, and should use this leverage to advocate for good policies (eg pace), before they’re all automated away by AI. I’d point out that AI safety employees themselves are in similarly precarious positions. How long before automated AI safety researchers?
One scheme I’m fond of is “lab employee union (but, good)”. Similarly, it might be interesting to have an “AI safety employee union (but, good)”.
… or are they fine, actually?
One pushback against higher salaries is they’re already pretty high, for the reference class of nonprofit work. Like $100k/y might be enough to cover living costs and help individuals build up some savings, even in high cost of living locations like SF/DC/NYC/London. But, idk, I want safety people to feel safe, and thrive, and have babies.
Some argue that offering lower salaries leads to more mission-oriented employees. I think there’s some truth to this, but also some amount of cope. And this kind of reasoning can let an org undercount the opportunity cost of its people, causing internal & ecosystem-wide misallocation of talent. So put your money where your mouth is!
Maybe it looks bad somehow, if nonprofit employees & execs are making $1m+/y? To that, I’d say: a) stop creating nonprofits, b) inform your critics about how much the labs are pay, and c) maybe just don’t worry so much about looking bad.
IMO the strongest pushback for safety executives, specifically, is that they already have a large amount of steering power, from their ability to direct the org budget & hire people. (This is partly why I’ve never taken a salary at Manifund. And I do appreciate that Sam Altman roughly does the same.)
Gigabrained compensation
Impact equity would solve many of these salary considerations. (Apologies to the choir for the preaching.) Equity allocation is what allowed the startup ecosystem flourish — and now, the frontier lab ecosystem too. The “do good” space is immensely hobbled by nonprofit charity norms.
BTW, one reason we love regranting is that it corrects for some of the problems introduced by low nonprofit salaries. Great AI safety people get the funds and the mandate to go around “angel investing” into new good opportunities.
Beyond higher salaries, I’m tentatively in favor of a UBI or pension for AI safety people. Maybe: $50k/y paid out to forever, to anyone who meets certain objective thresholds, eg 6+ months of fulltime safety employment or funding. I’d hope for a renaissance where individuals feel empowered to move, quit, fund weird things, explore and start little projects.
This wouldn’t be that expensive! I’d guess 10k people might qualify currently, so this would cost $500M/y — peanuts, compared to Anthropic wealth. Obviously there’s room to do design this better, but I think UBI should be one baseline for “how to distribute Anthropic wealth”. (I don’t know that current funding structures even beat this baseline.)
The salaries are too damn low
… or are they fine, actually?
Gigabrained compensation
This wouldn’t be that expensive! I’d guess 10k people might qualify currently, so this would cost $500M/y — peanuts, compared to Anthropic wealth. Obviously there’s room to do design this better, but I think UBI should be one baseline for “how to distribute Anthropic wealth”. (I don’t know that current funding structures even beat this baseline.)