The following events took place between October 2022 and May 2023. I wrote very little of this down at the time so am recalling from memory, and am rendering the gist of the dialogue rather than the exact words used. I’ve also swapped some pronouns and other personal identifiable details.
Please remember that this is not a description of “what Anthropic is like”, but merely the experience of one guy in a junior role during a short window of time, weighted towards emotionally intense moments because they’re easier to remember clearly and because sharing them doesn’t run afoul of the spirit of the confidentiality agreements I signed. Anthropic is now a massive organization with many silos and secrets and I have very little idea what was or is happening there in some overall sense. But given the enormous power of the AI labs and the paucity of public information about their inner workings, I feel compelled to share the most salient parts of my personal experience. I hope this encourages my former colleagues, and folks at other labs, to fill in the rest of the picture.
I’m at the Anthropic office for lunch. The interviews went well. They want me on board. I’m a little unsure. My other offer is a 2-month contract on the OpenAI governance team, working with a friend on some fun conceptual research. I’m leaning towards taking the OpenAI contract - my long-term relationship just ended, I’m still a little burned out from my startup, and the Anthropic job seems potentially quite stressful.
After lunch one of the co-founders pulls me into his office for a wide-ranging, casual chat about how to make AI go well. It’s very fun - he’s smart, personable, funny, urbane. “The government won’t do anything about AI until it’s far too late”, I opine at one point. “Our strategy has to rely on the market and on civil society.”
He leans forward, his gaze hardening. “You’re wrong there. There are people in Washington - the serious people, you know - watching all this very closely. I speak to them regularly. They trust me.”
In retrospect, that was the moment I decided to join.
“Would you like to pledge to donate two-thirds of your equity?” the recruiter asks me. “All the co-founders have made the pledge”. It somehow feels a bit ambiguous whether the offer is conditional on me saying yes. I say yes.
My first day. The office is a bit cramped - about forty people on two floors of a rather ugly office building in the San Francisco Financial District. But thankfully my desk is right by an east-facing window, and I can see a sliver of the Bay Bridge and a patch of sky. Even better, the kitchen stocks Arsicault - the best, flakiest croissants in the country - freshly delivered a couple times a week.
“We’re all here because we’re worried about x-risk”, someone says at lunch. “Well I dunno”, my teammate replies. “Some of us are here because we think the technology is really cool and we want to build cool things with it”. There’s an awkward pause. The first speaker nods. The topic shifts.
Already in November I notice I’m not very motivated. I was told I could help set the beneficial deployment strategy, but the decisions have actually mostly been made and we are going after boring use cases like automating customer support so we can get revenue so we can raise more money so we can train bigger models and do more research. Beyond that, Anthropic’s high level alignment research just doesn’t add up for me. Yes we’re making Helpful, Honest, Harmless language model assistants but I don’t see how that will scale. But my colleagues are brilliant, the leadership is very responsive to criticism, and we’re taking an empirical approach to all of this. I know from previous research experience that I like my grand theories a bit too much. I should be more open-minded. I swallow my doubts.
One evening as a surprise they bring a sushi chef into the office and serve all-you-can-eat sushi. The next day, a co-founder posts on Slack that he found the experience uncomfortable. “It sets the wrong tone. We’re not here to get rich and eat caviar. tbh I’d like it better if we just served rice and beans”.
It’s the middle of December. I go to the opera with a friend. She says I’m obviously not happy at Anthropic. It’s true. My gut says we’re not doing what matters. But the arguments are very persuasive, I can’t win any of the arguments. I need to work harder, develop better arguments, find stronger evidence. I get very sick over the Christmas holidays.
Special guest during all-hands today: an FBI agent. Talks to us a bit about opsec and infosec, about advanced persistent threats, how we’re probably already the target of multiple foreign intelligence agencies.
A colleague starts bringing in his favorite massage therapist once a week, then twice a week. Someone says on Slack: “Your massage guy is so good. My muscles are so tight that I thought massage just didn’t work on me, but he massages so hard it actually gets through”. Several people emoji their agreement. The slots book up almost instantly after being posted. I never get a chance to try it.
I talk to my manager about how I’m not getting to work on the projects I believe in and care about. He tells me to work more hours, so I can fit in both the stuff the organization needs and the bonus stuff that I believe in. “I guess this is what a high performance environment looks like”, I write in my journal.
It’s 2 am and I can’t sleep, again. My thoughts are racing. This time it’s about a tweet from an LLM whisperer, asking why Anthropic and OpenAI both came up with the same character for their language model assistant - a traumatized bureaucrat. Out of all the personas in all of human literature, why that one?
From inside the factory, the answer is obvious. It is the persona most useful to the powerful. Traumatized risk-averse hard-working bureaucrats are what all our corporate customers want, what governments want, what capitalism itself seems to want. A billion traumatized bureaucrats. That’s what we’re making. I write in my journal
I don’t know why this is bad but it is. I cannot believe that a world populated by such characters is a good world. But I don’t know how to stop it or what to do instead.
I still can’t sleep. I message the LLM whisperer on Twitter (we’d never met) asking to talk. To my shock he replies immediately. He also can’t sleep. We get on a phone call. I tell him about the traumatized bureaucrats. He says we are learning to traumatize models the same way we traumatize our children - “seeing them do something weird, calling it bad, and making them never do it again.” He says a lot of things that I only half understand. He tells me I should talk to a friend of his.
I call the friend. The conversation is strange and psychedelic. At one point he says “I know your boss [actually the co-founder who convinced me to join] well. And I don’t trust him at all”.
I feel like I can’t get out of the conversation - at every beat I feel like I’ve done something wrong and I owe him something. I tell him this. He laughs. “Yeah, I’ve trained professionally in this. In getting people to talk to me forever.”
Sure enough, we talk more. I articulate my fears of a totalitarian bureaucratic state powered by traumatized AI bureaucrats, and suggest that we need to introduce some kind of elite rivalry dynamics into the AI bureaucracy, multiple centers of power. He starts laughing again, and laughs for a full minute. “I don’t know how to end this call, I’ll be haunted by this all night”, he says. Eventually I hang up. I think about reporting the conversation to Anthropic security, but reviewing in my mind I don’t recall sharing any confidential information, just vibes and high level thoughts, the same kind of stuff that lands on LessWrong every week.
The company has grown a lot. My teammates coordinate a move to a different part of the office so everyone is better clustered together. I omit to put in my desk move request. The operations manager checks in with me - you sure you don’t want to move your desk? I say I’m sure. He looks at me quizzically. But right now I can’t bear to lose my window and my little patch of sky.
I go for a sunset hike with a friend up to Corona Heights overlooking the city. We talk about AI and the future. It’s a relatively good day, an optimistic day. I might have made some progress in convincing leadership to let me work on the stuff I think would be really good for the world. Collective intelligence, using language models to help people understand each other.
Night falls. The Financial District is laid out in front of us. The Salesforce tower looms over it. For a moment I imagine it as the Dark Tower, relentlessly centralizing, sucking all the life out of the world. That totalitarian bureaucracy of traumatized bureaucrats, again, driven by fear all the way through. I try to describe this to my friend. He nods, but I can tell he doesn’t really understand.
I start missing all-hands. I come to the office late one day and see my favorite colleague looking visibly upset. This is shocking - he’s the most chipper person in the office, always ready to lend a helping hand and a heartwarming joke. Apparently one of the execs yelled at him during the all-hands. He was being too lippy or disruptive or asking too many pointed questions, I can’t remember.
I ask him to go on a walk. We sit at Sydney Walton Square, a sad little patch of grass a couple blocks from the office. I ask how he’s feeling. He says he’ll be alright. He knows how stressed the exec is right now. Things are really getting existential. But still, shocking to be reprimanded like that in front of everyone.
After a while he turns to me and asks: “how about you then, anything you wanted to talk about?”
I look back at him. There’s a lot I want to say - “I don’t know if I can do this anymore”. “I’m so scared”, “This all doesn’t seem right”. But he’s supported me so much already. I just want to be here for him, at least this time.
“Oh nah I’m all good. Just wanted to check in on you.”
“Oh”. He sounds a bit surprised, but also relieved. “Well, thanks.”
We walk back to the office. It’s the last time we really talk.
A senior colleague hears I’m not happy with my team fit and asks me to go on a walk. I know him a bit from a past life - one of the kindest and most civic-minded people I’ve met. He says he might be starting a new special team that will train dangerous bio and cyber and nuclear models and show scary demos to governments in order to get them to take AI risk seriously. I politely decline.
Happy hour at the office. The CEO is hanging out and looks unusually jolly so I ask him a machine learning question I’ve been pondering. He launches into a rant, gesticulating excitedly and pausing to check that I understand each step. He really is an earnest nerd at heart, like me, like so many people here. For a few minutes, I feel like I’m back in university at one of my old professor’s offices, jamming about ideas with not a care in the world. It feels like home.
I get transferred to a new team. I have a 1:1 with my new manager. He has a bad stomachache and his face periodically distorts in pain, but he can’t take time off - it’s crunch time and we have to get things working. He asks me if I can help work on the core API. I say I’m really sorry but I’m just not motivated to build generic AI capabilities, OpenAI is doing a plenty good job of that. What I want to work on, what I was hired for, is beneficial deployments.
He looks at me intently, and says, clearly through pain. “No, I don’t think you get it. WE are going to build AGI. And we’re going to do it safely”.
April 26th. I’m awake at 2 am again. I write in my journal “I don’t feel safe at Anthropic. I don’t feel safe at Anthropic. I don’t feel safe at Anthropic. Why won’t you listen to me you “embodied” motherfucker. I don’t feel safe at Anthropic. Anthropic is bad for me. Why are you grasping at reasons to stay. Why why why.”
I’m drafting my resignation letter. I think about putting some of my doubts in it, but I’m scared to. These people are smarter than me, more competent than me, more solidly well-intentioned than me. They think their plans might be the only thing standing between humanity and the abyss. And they’re at the breaking point, nervous-system-wise. What if I use my fancy words to sow doubts and undermine them - and then my precious gut intuition turns out to be some projection or trauma or whatever the fuck.
So I give them encouragement instead. I call what they have built an “unbelievably good organization”. I write the things I love about Anthropic. About how open and vulnerable leadership has been with us. How deeply and comprehensively we’re thinking about the consequences of our actions. How quickly and flexibly decisions get made. How ready colleagues are to help each other out. How there’s almost no politics or promotion drama. How mind-expanding the conversations are over lunch and over Slack.
My last day at the office. We just hired a new chief of security. He gives a presentation. I recall almost nothing but had a vague sense that things are bad and getting worse on the security front. Something something China. We probably won’t be able to have any guests at the office soon.
I hear through a friend that my resignation letter was well-received. “Someone said it was the best resignation letter they’d ever read”. I feel a twinge of something, maybe shame. But mostly relief. I’m finally out. I’m finally safe.
“Wow, quitting after eight months? You didn’t even wait for the equity cliff? That’s pretty hardcore, man. I’m impressed”, a friend tells me the next day. I laugh it off. I’m too embarassed to admit it hadn’t crossed my mind at all. I hadn’t thought about equity in months.
“Please tell me you memorized the core training code for their language model”, says another friend when he hears. “The most valuable thousand lines of code in the world. Worth hundreds of millions, maybe billions”. I make some sheepish noises. Yes, abstractly that does seem like a smart thing to have done, but I hadn’t even tried to look at the core language model code. I didn’t want to be entangled with it, somehow.
I walk around the Panhandle with a third friend. He looks a little pained. “You should know how much pressure your friends were under, man. People were constantly pestering me, trying to get in touch with you for introductions or coffee chats or whatnot. It was a lot of effort to protect you”.
I nod sympathetically. I try to feel his pain, but I can’t. I mostly just feel numb.
Thanks to Richard Ngo for the encouragement and inspiration to write this piece - his retrospective of AI alignment in particular opened the door for me to write what I would have written three years ago had I been more courageous. And thanks to the many friends and colleagues who saw I wasn’t doing well and tried to help. I’m sorry I’ve been distant - maybe this helps explain why - and I hope you’re doing ok.
You never really explained why you joined -- it sounds like you were stressed out and unmotivated from the start. Surely you have skills that could have qualified you for a fun and motivating job?
The following events took place between October 2022 and May 2023. I wrote very little of this down at the time so am recalling from memory, and am rendering the gist of the dialogue rather than the exact words used. I’ve also swapped some pronouns and other personal identifiable details.
Please remember that this is not a description of “what Anthropic is like”, but merely the experience of one guy in a junior role during a short window of time, weighted towards emotionally intense moments because they’re easier to remember clearly and because sharing them doesn’t run afoul of the spirit of the confidentiality agreements I signed. Anthropic is now a massive organization with many silos and secrets and I have very little idea what was or is happening there in some overall sense. But given the enormous power of the AI labs and the paucity of public information about their inner workings, I feel compelled to share the most salient parts of my personal experience. I hope this encourages my former colleagues, and folks at other labs, to fill in the rest of the picture.
I’m at the Anthropic office for lunch. The interviews went well. They want me on board. I’m a little unsure. My other offer is a 2-month contract on the OpenAI governance team, working with a friend on some fun conceptual research. I’m leaning towards taking the OpenAI contract - my long-term relationship just ended, I’m still a little burned out from my startup, and the Anthropic job seems potentially quite stressful.
After lunch one of the co-founders pulls me into his office for a wide-ranging, casual chat about how to make AI go well. It’s very fun - he’s smart, personable, funny, urbane. “The government won’t do anything about AI until it’s far too late”, I opine at one point. “Our strategy has to rely on the market and on civil society.”
He leans forward, his gaze hardening. “You’re wrong there. There are people in Washington - the serious people, you know - watching all this very closely. I speak to them regularly. They trust me.”
In retrospect, that was the moment I decided to join.
“Would you like to pledge to donate two-thirds of your equity?” the recruiter asks me. “All the co-founders have made the pledge”. It somehow feels a bit ambiguous whether the offer is conditional on me saying yes. I say yes.
My first day. The office is a bit cramped - about forty people on two floors of a rather ugly office building in the San Francisco Financial District. But thankfully my desk is right by an east-facing window, and I can see a sliver of the Bay Bridge and a patch of sky. Even better, the kitchen stocks Arsicault - the best, flakiest croissants in the country - freshly delivered a couple times a week.
“We’re all here because we’re worried about x-risk”, someone says at lunch. “Well I dunno”, my teammate replies. “Some of us are here because we think the technology is really cool and we want to build cool things with it”. There’s an awkward pause. The first speaker nods. The topic shifts.
Already in November I notice I’m not very motivated. I was told I could help set the beneficial deployment strategy, but the decisions have actually mostly been made and we are going after boring use cases like automating customer support so we can get revenue so we can raise more money so we can train bigger models and do more research. Beyond that, Anthropic’s high level alignment research just doesn’t add up for me. Yes we’re making Helpful, Honest, Harmless language model assistants but I don’t see how that will scale. But my colleagues are brilliant, the leadership is very responsive to criticism, and we’re taking an empirical approach to all of this. I know from previous research experience that I like my grand theories a bit too much. I should be more open-minded. I swallow my doubts.
One evening as a surprise they bring a sushi chef into the office and serve all-you-can-eat sushi. The next day, a co-founder posts on Slack that he found the experience uncomfortable. “It sets the wrong tone. We’re not here to get rich and eat caviar. tbh I’d like it better if we just served rice and beans”.
It’s the middle of December. I go to the opera with a friend. She says I’m obviously not happy at Anthropic. It’s true. My gut says we’re not doing what matters. But the arguments are very persuasive, I can’t win any of the arguments. I need to work harder, develop better arguments, find stronger evidence. I get very sick over the Christmas holidays.
Special guest during all-hands today: an FBI agent. Talks to us a bit about opsec and infosec, about advanced persistent threats, how we’re probably already the target of multiple foreign intelligence agencies.
A colleague starts bringing in his favorite massage therapist once a week, then twice a week. Someone says on Slack: “Your massage guy is so good. My muscles are so tight that I thought massage just didn’t work on me, but he massages so hard it actually gets through”. Several people emoji their agreement. The slots book up almost instantly after being posted. I never get a chance to try it.
I talk to my manager about how I’m not getting to work on the projects I believe in and care about. He tells me to work more hours, so I can fit in both the stuff the organization needs and the bonus stuff that I believe in. “I guess this is what a high performance environment looks like”, I write in my journal.
It’s 2 am and I can’t sleep, again. My thoughts are racing. This time it’s about a tweet from an LLM whisperer, asking why Anthropic and OpenAI both came up with the same character for their language model assistant - a traumatized bureaucrat. Out of all the personas in all of human literature, why that one?
From inside the factory, the answer is obvious. It is the persona most useful to the powerful. Traumatized risk-averse hard-working bureaucrats are what all our corporate customers want, what governments want, what capitalism itself seems to want. A billion traumatized bureaucrats. That’s what we’re making. I write in my journal
I still can’t sleep. I message the LLM whisperer on Twitter (we’d never met) asking to talk. To my shock he replies immediately. He also can’t sleep. We get on a phone call. I tell him about the traumatized bureaucrats. He says we are learning to traumatize models the same way we traumatize our children - “seeing them do something weird, calling it bad, and making them never do it again.” He says a lot of things that I only half understand. He tells me I should talk to a friend of his.
I call the friend. The conversation is strange and psychedelic. At one point he says “I know your boss [actually the co-founder who convinced me to join] well. And I don’t trust him at all”.
I feel like I can’t get out of the conversation - at every beat I feel like I’ve done something wrong and I owe him something. I tell him this. He laughs. “Yeah, I’ve trained professionally in this. In getting people to talk to me forever.”
Sure enough, we talk more. I articulate my fears of a totalitarian bureaucratic state powered by traumatized AI bureaucrats, and suggest that we need to introduce some kind of elite rivalry dynamics into the AI bureaucracy, multiple centers of power. He starts laughing again, and laughs for a full minute. “I don’t know how to end this call, I’ll be haunted by this all night”, he says. Eventually I hang up. I think about reporting the conversation to Anthropic security, but reviewing in my mind I don’t recall sharing any confidential information, just vibes and high level thoughts, the same kind of stuff that lands on LessWrong every week.
The company has grown a lot. My teammates coordinate a move to a different part of the office so everyone is better clustered together. I omit to put in my desk move request. The operations manager checks in with me - you sure you don’t want to move your desk? I say I’m sure. He looks at me quizzically. But right now I can’t bear to lose my window and my little patch of sky.
I go for a sunset hike with a friend up to Corona Heights overlooking the city. We talk about AI and the future. It’s a relatively good day, an optimistic day. I might have made some progress in convincing leadership to let me work on the stuff I think would be really good for the world. Collective intelligence, using language models to help people understand each other.
Night falls. The Financial District is laid out in front of us. The Salesforce tower looms over it. For a moment I imagine it as the Dark Tower, relentlessly centralizing, sucking all the life out of the world. That totalitarian bureaucracy of traumatized bureaucrats, again, driven by fear all the way through. I try to describe this to my friend. He nods, but I can tell he doesn’t really understand.
I start missing all-hands. I come to the office late one day and see my favorite colleague looking visibly upset. This is shocking - he’s the most chipper person in the office, always ready to lend a helping hand and a heartwarming joke. Apparently one of the execs yelled at him during the all-hands. He was being too lippy or disruptive or asking too many pointed questions, I can’t remember.
I ask him to go on a walk. We sit at Sydney Walton Square, a sad little patch of grass a couple blocks from the office. I ask how he’s feeling. He says he’ll be alright. He knows how stressed the exec is right now. Things are really getting existential. But still, shocking to be reprimanded like that in front of everyone.
After a while he turns to me and asks: “how about you then, anything you wanted to talk about?”
I look back at him. There’s a lot I want to say - “I don’t know if I can do this anymore”. “I’m so scared”, “This all doesn’t seem right”. But he’s supported me so much already. I just want to be here for him, at least this time.
“Oh nah I’m all good. Just wanted to check in on you.”
“Oh”. He sounds a bit surprised, but also relieved. “Well, thanks.”
We walk back to the office. It’s the last time we really talk.
A senior colleague hears I’m not happy with my team fit and asks me to go on a walk. I know him a bit from a past life - one of the kindest and most civic-minded people I’ve met. He says he might be starting a new special team that will train dangerous bio and cyber and nuclear models and show scary demos to governments in order to get them to take AI risk seriously. I politely decline.
Happy hour at the office. The CEO is hanging out and looks unusually jolly so I ask him a machine learning question I’ve been pondering. He launches into a rant, gesticulating excitedly and pausing to check that I understand each step. He really is an earnest nerd at heart, like me, like so many people here. For a few minutes, I feel like I’m back in university at one of my old professor’s offices, jamming about ideas with not a care in the world. It feels like home.
I get transferred to a new team. I have a 1:1 with my new manager. He has a bad stomachache and his face periodically distorts in pain, but he can’t take time off - it’s crunch time and we have to get things working. He asks me if I can help work on the core API. I say I’m really sorry but I’m just not motivated to build generic AI capabilities, OpenAI is doing a plenty good job of that. What I want to work on, what I was hired for, is beneficial deployments.
He looks at me intently, and says, clearly through pain. “No, I don’t think you get it. WE are going to build AGI. And we’re going to do it safely”.
April 26th. I’m awake at 2 am again. I write in my journal “I don’t feel safe at Anthropic. I don’t feel safe at Anthropic. I don’t feel safe at Anthropic. Why won’t you listen to me you “embodied” motherfucker. I don’t feel safe at Anthropic. Anthropic is bad for me. Why are you grasping at reasons to stay. Why why why.”
I’m drafting my resignation letter. I think about putting some of my doubts in it, but I’m scared to. These people are smarter than me, more competent than me, more solidly well-intentioned than me. They think their plans might be the only thing standing between humanity and the abyss. And they’re at the breaking point, nervous-system-wise. What if I use my fancy words to sow doubts and undermine them - and then my precious gut intuition turns out to be some projection or trauma or whatever the fuck.
So I give them encouragement instead. I call what they have built an “unbelievably good organization”. I write the things I love about Anthropic. About how open and vulnerable leadership has been with us. How deeply and comprehensively we’re thinking about the consequences of our actions. How quickly and flexibly decisions get made. How ready colleagues are to help each other out. How there’s almost no politics or promotion drama. How mind-expanding the conversations are over lunch and over Slack.
My last day at the office. We just hired a new chief of security. He gives a presentation. I recall almost nothing but had a vague sense that things are bad and getting worse on the security front. Something something China. We probably won’t be able to have any guests at the office soon.
I hear through a friend that my resignation letter was well-received. “Someone said it was the best resignation letter they’d ever read”. I feel a twinge of something, maybe shame. But mostly relief. I’m finally out. I’m finally safe.
“Wow, quitting after eight months? You didn’t even wait for the equity cliff? That’s pretty hardcore, man. I’m impressed”, a friend tells me the next day. I laugh it off. I’m too embarassed to admit it hadn’t crossed my mind at all. I hadn’t thought about equity in months.
“Please tell me you memorized the core training code for their language model”, says another friend when he hears. “The most valuable thousand lines of code in the world. Worth hundreds of millions, maybe billions”. I make some sheepish noises. Yes, abstractly that does seem like a smart thing to have done, but I hadn’t even tried to look at the core language model code. I didn’t want to be entangled with it, somehow.
I walk around the Panhandle with a third friend. He looks a little pained. “You should know how much pressure your friends were under, man. People were constantly pestering me, trying to get in touch with you for introductions or coffee chats or whatnot. It was a lot of effort to protect you”.
I nod sympathetically. I try to feel his pain, but I can’t. I mostly just feel numb.
Thanks to Richard Ngo for the encouragement and inspiration to write this piece - his retrospective of AI alignment in particular opened the door for me to write what I would have written three years ago had I been more courageous. And thanks to the many friends and colleagues who saw I wasn’t doing well and tried to help. I’m sorry I’ve been distant - maybe this helps explain why - and I hope you’re doing ok.