Transparency is less neglected than some other topics -- check out HR 5539 (Transparent by Design Act), S 3312 (AIRIA), and HR 6881 (AI Foundation Model Transparency Act).

There's room for a little bit more useful drafting work here, but I wouldn't call it orphaned, exactly.

I could carry on debating the pros and cons of the EO with you, but I think my real point is that bipartisan advocacy is harmless. You shouldn't worry that bipartisan advocacy will backfire, so we can't justify engaging in no advocacy at all out of fear that advocacy might backfire.

If you believe strongly enough in the merits of working with one party to be confident that it won't backfire, fine, I won't stop you -- but we should all be able to agree that more bipartisan advocacy would be good, even if we disagree about how valuable one-party advocacy is.

Yes, CAIP's strategy was primarily to get the legislation ready, talk to people about it, and prepare for a crisis. We also encouraged people to pass our legislation immediately, but I was not especially optimistic about the odds that they would agree to do so.

I don't object to people pushing legislators from both parties to act more quickly...but you have to honor their decision if they say "no," no matter how frustrating that is or how worried you are about the near-term future, because trying to do an end-run around their authority will quickly and predictably backfire. 

In my opinion, going behind legislators' backs to the Biden administration was particularly unhelpful for the Biden AI EO, because the contents of that EO would have led to only a small reduction in catastrophic risk -- it would be nice to require reports on the results of red-teaming, but the EO by itself wouldn't have stopped companies from reporting that their models seemed risky and then releasing them anyway. We would have needed to follow up on the EO and enact additional policies in order to have a reasonable chance of survival, but proceeding via unilateral executive action had some tendency to undermine our ability to get those additional policies passed, so it's not clear to me what the overall theory of change was for rushing forward with an EO.

I think it's worth distinguishing between "this AI policy could be slightly inconvenient for America's overall geopolitical strategy" and "this AI policy is so bad for America's AI arms race that we're going to lose a shooting war with China."

The former is a political problem that advocates need to find a strategy to cope with, but it's not a reason not to do advocacy -- we should be willing to trade away a little bit of American influence in order to avoid a large risk of civilizational collapse from misaligned AI.

If you're earnestly worried about maximizing American influence, there are much better strategies for making that happen than trying to make sure that we have zero AI regulations. You could repeal the Jones Act, you could have a stable tariff regime, you could fix the visa system, you could fund either the CHIPS Act or a replacement for the CHIPS Act, you could give BIS more funding to go after chip smugglers, and so on. 

I think the "concern" about the harmful geopolitical effects of moderate AI regulation is mostly opportunistic political theater by companies who would prefer to remain unregulated -- there's a notable absence of serious international relations scholars or national security experts who are coming out in favor of zero AI safety regulation as a geopolitical tool. At most, some experts might be pushing for easier land use and environmental approvals, which are not in conflict with the regulations that organizations like CAIP are pushing for.

I tend to agree with John -- I think the Republican opposition is mostly a mixture of general hostility toward regulation as a tool and a desire to publicly reject the Biden Administration and all its works. My experience in talking with Republican staffers, even in 2025, is that they weren't hostile to the idea of reducing catastrophic risks from AI; they're just waiting to see what the Trump Administration's official position is on the issue. We'll know more about that when we see the OSTP AI Action Plan in July.

Part of the problem with AI safety advocacy in 2022 - 2024 was that it ignored my advice to "insist that the bills you endorse have co-sponsors from both parties." By definition, an executive order is not bipartisan. You can frame the Biden EO as general technocracy if you like, but it's still careless to push for an EO that's only supported by one party. If you want to avoid alienating Republicans (and you should want to avoid doing that), then you need to make sure you have at least some Republicans willing to go on record as publicly endorsing your policy proposals before you enact them.

Hmm. There are lots of valuable advocacy targets besides literal regulation; advocates might try to pass laws, win court cases, sustain a boycott, amend a corporate charter, amend a voluntary industry standard at NIST, and so on. I'll discuss several examples in post #5. 

I suppose advocacy might also have some side benefits like teaching us more about politics, helping us bond as a community, or giving us a sense of efficacy, but it's not obvious to me that any of those are of comparable importance to reducing the likelihood that AI developers release misaligned superintelligence.

Does that answer your question? If not, please let me know more about what you mean and I'll try to elaborate.

I have no complaints at all about technical AI research! Indeed, I agree with you that industry will spontaneously adopt most of the good technical AI safety ideas shortly after they're invented.

I'm arguing about the relative merits of AI governance research vs. AI governance advocacy. This probably wasn't clear if you're just jumping in at this point of the sequence, for which I apologize.

Thanks! If you are directly and literally comparing the utility of two or three or four concrete policy options, that's more valuable than general academic research -- it's at least level 3, and possibly better if you write up your conclusions in a format that can be easily digested by policy wonks (level 2) or by politicians (level 1).

Unfortunately, most academic research weighs the pros and cons of general categories of policies and then offers no concrete recommendations, so a lot of what looks like "comparing different options" at first glance is not immediately valuable as a tool for picking the best currently available option.

Brainstorming new policy options to further explore the available space is at least a level 3 activity, and possibly level 2 (if you write up some of those options in enough detail that someone could advocate for them). However, as I'll argue in the fifth post in this sequence, we already have a dozen "orphaned" policy proposals that nobody is drafting up or advocating for, so unless and until this changes, we'll get limited value from having an even larger stack of orphaned policies. Even if you find a policy that's an order of magnitude better than previous ideas, that policy still doesn't generate any utility until someone goes and does the political work needed to turn it into law.

I agree that people like Bengio can be very valuable assets for AI safety advocacy, although there are diminishing marginal returns -- the first computer supergenius who likes your policies is transformative; the third is helpful; the tenth is mostly just a statistic in a survey and will not meaningfully change, e.g., journalists or staffers' opinions about an issue.

If you think that technology or history will move far enough ahead that people like Bengio and Hinton will lose their relevance, then it might be a good idea to try to convince the next Bengio to support some AI safety policies. If that's your strategy, then you should develop a short list of people who might be the next Bengio, and then go find them and talk to them in person. Once you've identified some leading young computer scientists and some questions that they're uncertain and curious about, then you can go do research to try to help convince them to take a particular stance on those questions. 

Just publishing AI governance research of general academic interest is very far removed from the goal of recruiting computer science superstars for AI x-risk advocacy.

I would call the main AI 2027 report an advocacy document -- its function is to tell a compelling story about how AI could cause large harms in the very near future. Although the authors claim that a "slowdown" is not their real policy recommendation, the entire structure of the piece (a good ending with a green button and a bad ending with a red button) strongly pushes readers in the direction of thinking that it would be good if the White House "centralizes compute and brings in external oversight." The story contains enough concrete details that particular White House officials (like the VP) could imagine doing this, so whether the authors intended it or not, the document acts like an advocacy pitch that asks those officials to take these actions.

The research supporting the AI 2027 report is pushed back to a secondary page that readers can check in on if they're still interested after they finish reading the advocacy document. This is good practice, and I would like to see more researchers adopt this type of structure. Unfortunately, most of our research today doesn't include an advocacy document at all, let alone as the primary focus of media engagement.