The Ban RSI protest at Anthropic’s IPO was encouraging: well-attended, covered by media, high-energy. It was focused on a single message – that we should ban AI systems that are capable of building more capable AI systems. It was a promising outcome for a protest planned only a couple of days in advance. But there was something weird about it, something I couldn’t put my finger on while it was going on. It was only when I read Michel Justen’s reporting on the protest, Anthropic is running out of friends, that I realized what my problem was.
The protest was way too friendly to Anthropic.
Mixed messages
The signs at the protest were a mix. There were signs you’d expect at a protest to ban RSI (“regulate AI now”, “freeze AI now”). But there were also signs clearly written for high-context people (“What did METR see?”, “hold swarm help peer ban RSI”), and even a sign that addressed Anthropic directly (“You said you wouldn’t push the frontier”).
It’s obvious, but worth saying: you would not see this at any other protest! It is really weird for a protest to share inside jokes with the organization it’s protesting, or to appeal to that organization’s moral compass. Signs like these assume good faith. They treat Anthropic as an ally who needs to be chided, not as a company that needs to be stopped.
The most surreal moment of this mixed messaging came at the end of one of the speeches, when the speaker declared “shut it down!” to cheers… before hastily grabbing the mic again and clarifying “shut down RSI, that is.” It took a powerful moment and pulled its punch.
The problem with a half-hearted message is that it doesn’t convey the urgency of the issue to the public. To a reporter at the scene, this looks like a family squabble. If the protesters won’t say it plainly, why would anyone else think it’s urgent?
It’s not hard to understand why Anthropic is getting this special treatment. Michel Justen:
“It’s conflicting because we actually like these people,” says Hu. “In a lot of cases, their individual moral character is really good. We respect these people. They seem thoughtful. They are intelligent and careful.” But, he adds, “plenty of well-meaning people have fucked things up a lot because of the position that they were in, because of their naivety,” or their “motivated reasoning.”
I understand the urge to pull your punches, the ugh-field around criticizing your friends. But you have to understand that even though your friend works at Anthropic, Anthropic is not your friend.Jasmine Sun: (emphasis mine)
Some anxious Ants have begun asking whether Anthropic should stop training better frontier models now, even if it can’t guarantee that competitors like OpenAI will do the same. This suggestion was shut down by Mr. Amodei, who urged staff members to stop listening to outside activists and explained that the company could pace itself without shooting itself in the foot. It was all part of the same strategy: Anthropic must keep winning to keep the world safe, and even if it stops, its competitors will not.
You can just protest things
It’s difficult to write this, because I have to think about the second-order or third-order effects of what I’m asking for. What if being more aggressive towards Anthropic alienates safety-conscious staff? Haven’t the organizers thought about this more than I have? Could it be optimal to balance an aggressive message towards regulators and an earnest message towards sympathetic Anthropic employees?
Of course there are safety-conscious Anthropic staff who could be valuable allies. But they will understand. I was an organizer for the University of California graduate student strike in 2022. A central challenge of organizing was that every grad student I talked to wanted to protest the university’s behavior, but most of them quite liked their own advisor and were worried that striking would hurt that relationship. Still, nobody tried to tone down the message or be less aggressive because of it. We just made it clear that the strike was not a personal problem, but a collective issue. And all the professors I talked to understood that. People who agree with you about the risks of RSI won’t walk away because you protested RSI too hard.
I don’t know the full game board, and I’m sure I’ll be set straight by someone who does. But there is no need to be mealymouthed about banning RSI. You can just make arguments forcefully and count on good-faith people at Anthropic to listen, even when you’re criticizing them.
The Ban RSI protest at Anthropic’s IPO was encouraging: well-attended, covered by media, high-energy. It was focused on a single message – that we should ban AI systems that are capable of building more capable AI systems. It was a promising outcome for a protest planned only a couple of days in advance. But there was something weird about it, something I couldn’t put my finger on while it was going on. It was only when I read Michel Justen’s reporting on the protest, Anthropic is running out of friends, that I realized what my problem was.
The protest was way too friendly to Anthropic.
Mixed messages
The signs at the protest were a mix. There were signs you’d expect at a protest to ban RSI (“regulate AI now”, “freeze AI now”). But there were also signs clearly written for high-context people (“What did METR see?”, “hold swarm help peer ban RSI”), and even a sign that addressed Anthropic directly (“You said you wouldn’t push the frontier”).
It’s obvious, but worth saying: you would not see this at any other protest! It is really weird for a protest to share inside jokes with the organization it’s protesting, or to appeal to that organization’s moral compass. Signs like these assume good faith. They treat Anthropic as an ally who needs to be chided, not as a company that needs to be stopped.
The most surreal moment of this mixed messaging came at the end of one of the speeches, when the speaker declared “shut it down!” to cheers… before hastily grabbing the mic again and clarifying “shut down RSI, that is.” It took a powerful moment and pulled its punch.
The problem with a half-hearted message is that it doesn’t convey the urgency of the issue to the public. To a reporter at the scene, this looks like a family squabble. If the protesters won’t say it plainly, why would anyone else think it’s urgent?
It’s not hard to understand why Anthropic is getting this special treatment. Michel Justen:
I understand the urge to pull your punches, the ugh-field around criticizing your friends. But you have to understand that even though your friend works at Anthropic, Anthropic is not your friend. Jasmine Sun: (emphasis mine)
You can just protest things
It’s difficult to write this, because I have to think about the second-order or third-order effects of what I’m asking for. What if being more aggressive towards Anthropic alienates safety-conscious staff? Haven’t the organizers thought about this more than I have? Could it be optimal to balance an aggressive message towards regulators and an earnest message towards sympathetic Anthropic employees?
Of course there are safety-conscious Anthropic staff who could be valuable allies. But they will understand. I was an organizer for the University of California graduate student strike in 2022. A central challenge of organizing was that every grad student I talked to wanted to protest the university’s behavior, but most of them quite liked their own advisor and were worried that striking would hurt that relationship. Still, nobody tried to tone down the message or be less aggressive because of it. We just made it clear that the strike was not a personal problem, but a collective issue. And all the professors I talked to understood that. People who agree with you about the risks of RSI won’t walk away because you protested RSI too hard.
I don’t know the full game board, and I’m sure I’ll be set straight by someone who does. But there is no need to be mealymouthed about banning RSI. You can just make arguments forcefully and count on good-faith people at Anthropic to listen, even when you’re criticizing them.