I have written a paper on Chinese domestic AI regulation with coauthors James Zhang, Zongze Wu, Michael Chen, Yue Zhu, and Geng Hong. It was presented recently at NeurIPS 2025's Workshop on Regulatable ML, and it may be found on ArXiv and SSRN. Here I'll explain what I take to...
Only the US can make us ready for AGI, but Europe just made us readier. Cross-posted from the AI Futures blog. We’ve previously written about what an individual can do to make the development of transformative AI less likely to end in disaster. How about an AGI company?[1] What steps...
Today I'm launching SB53.info, a resource to help people understand California's Senate Bill 53. The site shows the text of SB 53 enriched with annotations to explain its political and scientific context. What does SB 53 do? In a nutshell, SB 53 says seven things. These are the four I...
I have written a paper on Chinese domestic AI regulation with coauthors James Zhang, Zongze Wu, Michael Chen, Yue Zhu, and Geng Hong. It was presented recently at NeurIPS 2025's Workshop on Regulatable ML, and it may be found on ArXiv and SSRN.
Here I'll explain what I take to be the key ideas of the paper in a more casual style. I am speaking only for myself in this post, and not for any of my coauthors.
The top US AI companies have better capabilities than the top Chinese companies have for now, but the US lead isn't more than a year at most, and I expect... (read 708 more words →)
My understanding is that the US Marshals are not only accountable to the Court either. They take their day-to-day commands from the Director of the US Marshals Service, a presidential appointee, who in turn reports to the US Attorney General, another presidential appointee.
This makes me even more doubtful that the US Marshals would side with SCOTUS and, eg, arrest the President in a worst-case constitutional crisis. Both the Marshals' boss and their boss's boss would likely side with the President, having been chosen by him for their loyalty.
It's sometimes suggested that as a defense against superpersuasion, one could get a trusted AI to paraphrase all of one's incoming messages. That way an attacker can't subliminally steer the target by choosing exactly the right words and phrases to push their buttons.
Maybe one could also defend against superhuman persuasion and manipulation by getting a trusted AI to paraphrase all of one's outgoing messages. That way an attacker can't pick up on subtle clues about the target's psychology embedded in their writing style. If this paraphrasing strategy works, the attacker is denied the training data they need to figure out how to push the target's buttons.
Cross-posted from the AI Futures blog.
We’ve previously written about what an individual can do to make the development of transformative AI less likely to end in disaster. How about an AGI company?[1] What steps should they take right now to prepare for crunch time?
The first thing we’d recommend an AGI company do is to coordinate with other companies and with governments to stop the reckless race toward superintelligence. Failing that, our backup recommendation would be for an AGI company to invest in planning and transparency.
We expect that during takeoff, leading AGI companies will have to make high-stakes decisions based on limited... (read 4396 more words →)
Today I'm launching SB53.info, a resource to help people understand California's Senate Bill 53. The site shows the text of SB 53 enriched with annotations to explain its political and scientific context.
In a nutshell, SB 53 says seven things. These are the four I consider most important:
If your two assumptions hold, takeover by misaligned AGIs that go on to create space-faring civilizations looks much worse than existential disasters that simply wipe humanity out (eg, nuclear extinction). In the latter case, an alien civilization will soon claim the resources humanity would have taken over had we become a space-faring civ, and they'll create just as much value with these resources as we would have created. Assuming that all civs eventually come to an end at some fixed time (see this article by Toby Ord), some amount of potential value will be lost, but not as much as would be lost if a misaligned AI used all the resources in our future light cone for something valueless while excluding aliens from using them. So the main strategic implication is that we should try to build fail-safes into AGI to prevent it from becoming grabby in the event of alignment failure.
Interesting! Thanks for the link.