We have battled with Big Pharma and health insurance companies in pursuit of Medicare for All; with Big Oil in pursuit of a Green New Deal; and with the military industrial complex and the Israel lobby in pursuit of an end to endless wars.
But these people and groups are largely positive forces for good, in many cases directly responsible for a great deal of human flourishing, prosperity, justice and peace.
I also believe that non-superintelligent AI is primarily a force for good, a view which I expect to become increasingly widely-held as the benefits from AI compound and disperse. The downside is that it will eventually lead to the development of ASI that kills everyone.
So I think folding "AI safety" into the left-wing omnicause is both morally wrong, and also not likely to work very well as a strategic matter.
(It would be a sad choice, but I would ally with the left and trade away self-driving cars and abundant robots in order to stave off unaligned ASI. In practice though, I think that trade is not actually on offer, and left-wing AI safety advocacy is more likely to bring about something closer to the opposite, by strangling outlets for short-term innovation and pushing labs to focus on internally scaling directly to ASI through internal RSI.)
The problem, though I do not want to bring up ideological knifefights, is that how well have appealing to the corporate right (or the corporate world in general) to develop safe ASI worked out for us? Eliezer and the original LessWrong leadership approached the Valley in hopes of developing safe AGI, and the result is the modern AI race - at the very least they're fully or partially responsible for the emergence of GDM, OpenAI and Anthropic. (Which I consider to be both an unforced and catastrophic error.) The bias towards gaining power in the Valley has already been well-established among alignment thought leaders, and in general (not just in the Valley), corporate self-regulation has a famous track record for catastrophic failure and/or malicious corruption, in some cases leading to genocide-level mortality rates.
Also, I disagree with the notion that partisan movements can fail to achieve significant change. Sufficient dominance is the only thing necessary. The original New Deal has shown that it's possible to ally with one party and push through massive changes without appealing to the other party, if sufficient dominance is achieved on one side to shatter the deadlock. I don't believe that aggressive measures will be met with no progress: 30-hour work week bills were proposed as sticks that were used to force businesses to concede to 40-hour work weeks, and sufficient aggressive pressure, backed with legislative and military threats of nationalization, could force AI companies to submit meaningfully to concessions on safety monitoring. I also don't believe that labs can focus on internal scaling under military-backed threats of jail or nationalization without compensation, given the extreme national security threats that ASI, and by extension ASI-aiming labs, posed to America and all other countries.
So far, appealing directly to the labs have produced ineffectual, even outright angering, results: Anthropic walked back on safety commitments and more recently released Fable 5.2, by far the strongest public model to date, showing no signs of extending the external-internal gap. OpenAI is developing new architecture (looped transformers) that are severely more illegible than classical architectures, and ran a 10,000-agent swarm, despite recent severe alignment failures like Hugging Face with Sol and Astra, for prestige wins.
If bipartisanship is desired in a solution for AI safety, then the other side must show active signs of cooperativeness. So far, the Valley and the Trump administration has shown little to no activity that'd justify confidence in their commitments to safety. If that is the case, political subjugation via dominance, electorally first and then legislatively against companies, will be necessary; of course I'd be happy to walk back my stance if Anthropic, OpenAI and GDM significantly escalate their commitments to safety (or if Trump passed or called for extensive safety legislation).
But until then, more aggressive measures (and much more aggressive if you believe in supercritical RSI) seem to be required; and it seems quicker and easier to get the Democrats a supermajority of the Senate and the House by framing the Republicans as an anti-safety party (or use it as a threat to force the Republicans to treat AI safety seriouly, and publicly) than to get the two parties to work together on such a large scale, at least while one party has made little commitments to play ball. It'd be unfortunate, but also necessary.
ptd -- I appreciate your perspective and think you're right! We are seeing the results of leaving things to the labs, and we are also seeing the results of having appealed to Republicans when they were/are in power.
My fear is that establishment Democrats might feel equally reluctant to really lean in to safety, but I'm hoping Guardrails can do enough accountability to hold them to it, because that's the message they won on.
I would be interested in a version of this post that contains more actionable insights for things the AI Safety community can do in ways that learn from the techniques of the populist left.
I would be especially interested in versions of this pitch that are "the Left did XYZ, the AI Safety community should too, here's why it will achieve the AI Safety Community's goals", as opposed to "the AI Safety community should use Populist Left techniques because {Populist Left = Good}"
I think currently the Five Words this post gives me are "Fight Against AI Corporate Power", but I don't see any arguments for why we expect this approach to be effective, or indeed at all different from merely folding AI Safety into the progressive omnicause.
If we can accurately understand this as a fight against corporate power and for populist ends, we have effective messages and tactics to form the basis for our campaign.
Name three such messages and tactics?
I'm not the most read in to the preferred style of writing for LW so apologies! Here's what I think:
Politics is ALWAYS a gamble. It's very hard to measure impact, winners yesterday become losers tomorrow, and almost no one tactic is always effective. Wins can be undone, you can always find the opposite example if you look ahead a few years. With that in mind, I am giving an analysis that makes the case for a populist, anti-corporate campaign being the most effective in meeting the AI safety community's goals in this moment.
Don't worry, you're doing great! LW definitely has an unusual style if you're new to the community!
To respond to your points directly:
In fact, the only group in American politics over the last decade that has consistently and substantially stood up to billionaires and big corporations is the populist left (sometimes with partnership from the populist right).
Sure though there are some pretty big criticisms available there in terms of them getting their own aims. Eg the environmental movement currently standing in the way of BAAJA, which would reduce emissions.
I am not saying you're wrong, but I do think that populism doesn't always achieve its stated goals and can sometimes end up leading to the opposite of them.
Totally agree with this! The left, like every other group, is definitely not a monolith and has had successes and failures (which can be used to make both sides of the effectiveness argument).
One thing the mayor did well, which I try to articulate in my piece, is having a really sharp assessment of the moment and the shape and direction of public opinion, and then using that to build momentum for his campaign. My analysis is that the current mood of the country is not "let's pass bipartisan legislation" or "let's work with these smart AI safety researchers in Silicon Valley who are all making millions of dollars" but "let's take down billionaires and big corporations." Looking ahead at the coming years, it's not clear to me that public sentiment will change tremendously if people continue to experience health insurance denials, job loss, kids killing themselves because of chatbots, data center expansion etc. And I think that folks who are looking to make progress on AI safety goals could benefit a lot from adopting this messaging and agenda.
I am curious what tactics / messaging you think have been more consistently effective in American politics!
Yeah I mean I think the Mamdani is a generational politician. Your team was very successful in its aims and I wish him the best with New York.
hi! this is maya (op) - i gave a talk last week in the bay with a similar thesis, and sparked some robust debate and thoughtful follow-up questions, so i thought i'd memorialize it here for discussion! the thing i'm most interested in is the idea that people are underestimating how hard it's going to be to get ANY establishment politicians to take meaningfully curtailing corporate power seriously. welcome your feedback :)
Hello, I noticed that your policy priority list was more towards the risks of essentially contemporary or near-contemporary AIs, like data centers, corporate spending, chatbots in school, etc.
I was curious as to your perspective and views for mobilizing these political resources specifically with regards to existential risks against humanity, ie. risks of AI takeover, permanent power concentration, or gradual disempowerment.
With the current pace of progress as quick as it is, I wonder if a progressive populist political movement that centered around the recklessness or such of the frontier labs, but aimed it toward x-risk in advance of the public conversation fully awakening to the risks, might have more foresight, especially in the case where we see even more severe warning shots this coming year, and with recursive self-improvement, rapidly accelerating AI capabilities and even a potential intelligence explosion also forecasted for 2027 or '28.
This is a good question. From experience, it's really hard to organize people around priorities that feel distant from their immediate self-interest, and it's rarely effective to try to explain to people what their immediate self-interest should be. People are experiencing near-term harms that meaningfully destabilize, demoralize, and scare them right now, and it's important to leverage that momentum and organize around it. In the talk I gave, I described this strategy as "run the version of the campaign that you can win," and Zohran did it when he ran on affordability instead of public safety. At the end of the day, he still won and can affect policy on both affordability AND public safety-- but he chose to run the version of the campaign for which he had a winning message.
At Guardrails, here's what we're doing in Michigan: We have built relationships with a bunch of organizations-- data center activists, environmental justice groups, parent advocates, college students-- that are all doing different versions of anti-AI fights. We are funding them to form a statewide coalition and running political education and narrative trainings with them where we explain why all these fights are connected (AI oligarchs/companies are their common enemy) and also part of a fight to prevent catastrophic incidents. We're then working with them to plan actions together-- reaching out to their Members of Congress, responding to moments of crisis, etc.
I understand the urgency around the timeline... it's hard because if I knew a way to instantly create the exact political change I wanted to see, I would have done it already. What I do know is that building organizing infrastructure and relationships, and doing political education are all no-regrets work that allow for the opportunity to capitalize on a moment, if/when it does happen, to move as much policy/public opinion/etc. as possible.
The goal of my post was to start a conversation
okay. some points ...
suggestions:
peak risk occurs next year 2027. the bulk of irreversible risk likely occurs before the end of 2028.
What's the argument for this? What makes the world after 2028 safer? Back when alignment theory rather than political activism was viewed as the path to safety, it was said that the risk simply increases every year, as AI theory and practice advances.
What's the argument for this?
The nature of AI recursive self improvement (RSI) means that AI research accelerates. Compared to a baseline (e.g., last year) the number of collective research-days per wall clock day is greater than one. For example, on October 6, OpenAI published 700+ math papers resolving over 50 field-defining problems and ~200 named conjectures, representing something like 8 years of unassisted math research by the entire world (say min 3, max 25 years). It took their internal model ~4 weeks to obtain these results (the amount of human assistance is undisclosed), for a global research acceleration ratio of ~100x (say min 20x, max 600x). Math and AI research are different, and METR says AI research uplift is only 2x so far, but ...
The thing about AI RSI is that some fraction of the research will further accelerate research speed, resulting in what has been called a technological singularity. The term singularity is burdened by all sorts of connotations, so let's abandon it. Assume the acceleration plateaus, and just consider a 100 day interval of intensive research progress before that: "the jump."
At a flat 100x multiplier, the jump is equivalent to 25 years of baseline research. Starting at 100x and growing 5% per day (7 doublings), the jump is equivalent to over 700 years of baseline research.
The particular numbers don't really matter. What matters is that the jump may actually occur between now and the end of 2028. The exact start and end dates don't matter much either. Pre-jump research progress is small by comparison, although it is still important because it sets the starting conditions.
So risk does rise as wall clock time elapses ... right up to the jump. During the jump most of the irreversible risk gets realized, and afterward the outcome is largely set. The post-2028 world isn't so much safer as decided. The question is whether safety research gets uplift comparable to capabilities during the jump. If not, the risk could be irreversibly resolved unfavorably (loss of control, irreversible concentration of power, etc).
OK, I understand now. @themayadecline talks about "pausing frontier model development" and "banning super intelligence" and I thought you were speaking from a similar paradigm. But you are assuming that nothing is going to stop or even significantly slow AI capabilities progress, and pin your hopes on AI safety making enough progress in the same period.
@themayadecline talks about "pausing frontier model development" and "banning super intelligence" unilaterally, in the US. I do think this is unlikely before 2029 (encouraging democratic resurgence, but still not veto-proof), but as I commented, a unilateral single nation ban likely buys you less than 6 months, even when that nation is the US.
you are assuming that nothing is going to stop or even significantly slow AI capabilities progress
for context, the recent METR report has resulted in several posts like Orienting to 1 year AGI timelines which has passages like
The METR report on AI R&D automation within Anthropic is substantial evidence that timelines to superintelligence are now less than 2 years, even accounting for uncertainty about takeoff modeling, the reliability of the report’s conclusions, and other types of uncertainty.
and the Anthropic IPO filing has a risk section which they just a few days ago amended to state that "we made a determination that we have met" their own threshold for dramatically accelerated AI progress, which could be IPO hype, or could be an official statement that they have the ability to initiate at least weak RSI. This statement has actual legal consequences, which leans towards the latter being true.
say a miracle happens: the US frontier labs confirm that they could initiate RSI, but voluntarily choose not to pending further safety research. unless this results in a global pause that is at least decades long, it is still relatively small amount of research time compared to progress during the jump itself. defection during the pause is a constant threat. the defectors may even be safety researchers because notice that most safety research progress also occurs during the jump.
pin your hopes on AI safety making enough progress
yes. I don't think you can uninvent AI. and I think even a durable global ban on commercial AI research will just result in AI research becoming classified.
If everyone dies in 2027 because of a misaligned ASI, then no galaxy-brained plan which was supposed to give fruits in 2029 makes sense. Therefore, one shouldn't consider such politics-related plans in an attempt to reach a pause.
Thank you for your comment!
What is the best framework to lessen risk? Happy to do reading if you wanna link me.
"if you just polarize the issue, you are doing more harm than good. climate change shouldn't be a partisan issue, but people are not rational. please don't squander the current bipartisan skepticism just to "raise awareness"" --> my argument here is that anti-corporate sentiment is actually bipartisan and would be a useful messaging path for the AI safety community to pursue.
We are very much in support of whistleblowers and work closely with Rep. Casar (did an event with him on Friday)!
In my new role, I believe the analysis and tactics we’ve long used and studied on the populist left can help us win the fight for AI safety and shared benefits.
This seems to me like a poor strategy given the populist left has not been very successful at achieving its policy agenda.
Thank you for your comment! You're right -- it's very very hard to go up against big corporations and win, though it has happened in the past. The goal of my post was to start a conversation so I'm curious, what tactics have you found to be most effective?
Concentrated power. Policy winners includes corporations and unions. They have strong incentives around what policies they want and money and power to get what they want.
The other thing that works is overwhelming consensus. If 90+% of people want something, it usually happens.
I agree with you, and I consider those to be strategies -- vs tactics (short term/specific actions). As I discussed above, the left uses tactics like strikes, boycotts, door-knocking, and paid and earned media narrative campaigns to both build power and change public opinion.
I would love to discuss what other tactics we should consider / that y'all have found effective.
limit their ability to consolidate influence and build monopolies
I think it's good that frontier model development is currently an oligopoly with only a few companies on the frontier. If there were hundreds of companies on the frontier competing it would accelerate race dynamics a lot. While I don't think frontier AI labs are advocating for the best policy, they're much better than what you would get if you had hundreds of purely-profit motivated labs at the frontier.
This is an interesting point! I guess I'm interested in having no companies on the frontier -- and I think there's a lot of momentum around that anti-corporate message to jump on right now.
Would you support a "Personal Superintelligence for All" policy, democratizing the capabilities hitherto restricted to corporate power, where you allow individuals to run whatever open weight models to choose privately without surveillance, or would you instead advocate for tracking GPU use, imposing prior restraint on model publication, and putting the government into people's bedrooms (or wherever they happen to keep their computers)?
Honestly, I'm not a policy expert! There are a lot of different frameworks that I suspect could accomplish some key goals / be in line with our values. I think a LOT depends on how we communicate these frameworks to the American public. If you have good recommended reading on different options I'd love to check it out.
This is so wrong on so many levels. AI safety is not about fighting corporate power at all. If the threat is that the AI technology itself kills us all, then fundamentally humanity is all on the same side, including the labs. The labs are also where the most important expertise is. We have to work with them. If we start viewing them as the enemy, then we loose.
We also need to be safe from AI regardless of which political party is in power, because they will each be in power some of the time. That means that if we let AI safety become a partisan issue, if we couch AI safety arguments in terms that signal a partisan allegiance, then we loose.
I agree that I also don't want to see people treating the labs generally as enemies to be fought, and I think it's the wrong strategy. However, it is a coherent strategy; all it requires is a belief that the best course of action involves action against the labs by an outside force, such as the government, and that collaboration with the labs is of less worth than the political gain from vilifying them.
I think it's wrong because it will lead to bad thinking about the reasons the labs act like they do, and because regulation may fail or be ineffective. I guess maybe I feel most strongly that epistemic quality goes out the window when this kind of dynamic takes root, as it seems it might increasingly be on Lesswrong.
Thank you for your comment! Why do you think the labs are acting the way they do? I think they are trying to become as rich and powerful as possible, just like most big corporations. I have seen very little evidence that they are trying to save humanity, but I am open to other understandings of their behavior.
And, I am curious: if regulation might fail or be ineffective, what do you think is a better alternate strategy or set of tactics we should pursue (feel free to link me, happy to do my reading!)?
Firstly, I didn't say that seeing the labs as enemies will make people have incorrect beliefs about their motivations, but that it will lead to bad thinking on that question. It's an important distinction, because using flawed or motivated reasoning doesn't guarantee that you won't reach the correct conclusion. But it's a bad method for trying to reach that correct conclusion.
My personal sense is that this community is most useful to the world right now if it maintains whatever edge it has in rationally modeling the situation of AI development. I believe that political instrumentalization of the thinking and communication that goes on here will be actively hostile to that edge, for reasons that are well explained in the Sequences (ideas as soldiers etc).
My sense of your posting here so far, is that your project is exactly what I don't want to see happen: you are clearly trying to recruit here for your political tribe.
"If the threat is that the AI technology itself kills us all, then fundamentally humanity is all on the same side, including the labs."
It feels hard for me (and most Americans) to wrap my head around the idea that the labs are on our side, based on how they are behaving and the way we feel their impact. Right now, most Americans, according to polling, believe that AI is going to cause more harm than good and should be regulated. AI companies have terrible favorability. Having spoken to and organized with many folks on the ground across the country, I'm almost exclusively hearing complaints, fears, and anger about the impact of AI companies' products on people's lives. People don't trust these companies to regulate themselves or deliver benefits to humanity, and see them racing to develop harmful products and profit (especially after Anthropic floated that weak bill). If we need to get to a place where the American public sees the labs as their allies, then we are very behind. I'm curious what tactics you think might be able to change public opinion and momentum here.
What do you think about liability as a framework for reducing harms from AI, requiring AI companies get insurances against it.
Liability is a hugely important piece of the puzzle. It will both correct negative externalities and hopefully prevent them.
Thanks for sharing your perspective and working to help convince the public of the need for public institutions to get involved in making AI safer. While it will be difficult to mobilize the public on this issue before the die has been cast, it is a noble goal to work towards with little downside. While some are concerned about polarizing AI Safety, it is better to have the support of the Left against the Right than to have neither side engaged with the issue while corporations quietly dictate how society reacts to AI.
Thanks for your response! I agree -- there's little downside. And on the off chance it isn't too late, we will have built important infrastructure and hopefully moved public opinion to continue pushing for regulation.
After a couple of years working in data and tech at GiveDirectly, I left to work in politics, campaigning for candidates at the state, local, and federal levels. I most recently managed Mayor Zohran Mamdani’s general election campaign, but earlier this year, recognizing the urgency of the AI moment, I began as a senior advisor at Guardrails Alliance. In my new role, I believe the analysis and tactics we’ve long used and studied on the populist left can help us win the fight for AI safety and shared benefits.
The fight for AI regulation is fundamentally the same type of fight against corporate power that the populist left and labor have led throughout history. The message and tactics that win that fight are both effective and incredibly popular, and embracing them will help us put the American people in control of AI today and permanently in control of their futures. Here– in what I am told is likely the shortest LessWrong post of all time– I'll explain what the AI safety movement can learn from the populist left as it attempts to raise the salience of its message, build political power, and ultimately pass transformational legislation.
First, we need to be clear about the nature of the fight: who our opponents are and what they are trying to win. At its core, the fight for AI safety is a campaign to put significant checks on corporate power, which will ultimately curtail these companies’ profits, and limit their ability to consolidate influence and build monopolies. While some draw the distinction between “bad AI” (OpenAI) and “good AI” (Anthropic), both companies are rapidly developing products that put humanity at risk in pursuit of making more money than anyone in the history of the world, and there is reasonable evidence that Anthropic’s pitch to be seen as “good AI” is simply a hollow branding exercise designed to taint its competition (which itself originally claimed to be “good AI”!). The milquetoast bill Anthropic circulated earlier this year makes it clear that its business motives take precedence over concern for the future of humanity. Now that it has IPO’d, the company’s primary incentive, as with any major corporation, is to increase profits through any means necessary.
The reality is that any type of AI safety regulation that meaningfully reduces the chance of existential risk, whether by pausing frontier model development, banning super intelligence, or any other powerful lever, will also meaningfully reduce these companies’ profits. We should not expect either frontier lab to be our ally when substantial checks on their power are on the table. And the fight for AI safety isn’t just a fight to pass one bill– after we pass baseline regulation, it’s going to be a decades-long struggle as technology continues to advance. This means that any type of AI legislation addressing other harms that erode society’s resilience to catastrophic incidents– like denying people health benefits, replacing jobs without recompense, or placing chatbots in schools– will also reduce these companies’ bottom lines and be another front in the battle.
Second, we need to understand that we are waging an ideological and factional fight, not a partisan one. Some in the AI safety community feel that it’s most strategic to pursue a bipartisan approach to pass legislation. But that ignores a basic fact: establishment Democrats and Republicans are often equally hesitant to check corporate power, and we can’t rely on them to oppose the industry’s influence. In fact, the only group in American politics over the last decade that has consistently and substantially stood up to billionaires and big corporations is the populist left (sometimes with partnership from the populist right).
We have battled with Big Pharma and health insurance companies in pursuit of Medicare for All; with Big Oil in pursuit of a Green New Deal; and with the military industrial complex and the Israel lobby in pursuit of an end to endless wars. It’s clear to us that in the fight for AI safety legislation, and any other legislation that genuinely addresses the impact of AI on our society, our enemies will be Big Tech and the frontier labs. And we know that this fight is likely to be bipartisan in a different way than expected: it may be equally difficult to convince Republicans and establishment Democrats to take real action when they’re dining with Big Tech oligarchs, accepting their campaign donations, and hiring their family members.
If we can accurately understand this as a fight against corporate power and for populist ends, we have effective messages and tactics to form the basis for our campaign. Democrats across the ideological spectrum triggered an unprecedented blue wave election by using the economic populist message that Bernie Sanders and other leftists have been beating the drum on for decades (and that Republicans used to win in 2024). This election didn’t just demonstrate the effectiveness of an anti-corporate message; it reminded us of the blueprint for tactics. The wave of activism against AI data centers caused Republicans and Democrats alike to reverse their pro-data center messaging in the face of voter anger, with many of those fights carrying new elected officials to victory. When the populist left has won major fights against powerful enemies in the past, we have relied on confrontational and aggressive tactics like boycotts, strikes, sit-ins, civil disobedience, and arrests– the same kinds of tactics that are now winning the anti-data center war.
Using this playbook, we will be able to hold Democratic majorities accountable to the populist ideas they ran on, passing legislation that addresses voters’ anger and economic anxiety and meaningfully curtails the risks and harms caused by AI. In particular, I encourage anyone reading this who feels deep urgency around the need for AI safety regulation to explore using the political tactics and messaging discussed above. This is a moment in which we can take our fight directly to the people, rather than relying on inside games, op-eds, or campaign donations, none of which match the scale or power of what is needed to meaningfully rein in these companies on the timeline required.
I have spent most of my career working for progressive politicians, but I joined the AI fight earlier this year because I am clear on the risks of letting these companies accelerate unchecked. I am just as clear on the role of this fight in a broader realignment around an anti-corporate agenda. Our work at Guardrails began as a successful attempt to toxify AI money because we understood that Leading the Future’s undue influence would curtail our ability to pass meaningful regulation. Now, we’re looking to build on the opportunities we have preserved. As 2028 approaches, we plan to continue organizing, holding power to account, and remaining clear-eyed about our oligarch enemies. My hope is that the AI safety community can see themselves in our movement against corporate power and work with us to create a safer, more resilient, just, and equitable society.