Quick Takes

Here are some publications and quotes about GLM-5.3, its spike in cyber capabilities, and the unsurprising development that safeguards to prevent its use for cyber-offense can easily be removed: * 2026-08-14: https://x.com/Zai_org/article/2088280509474320693 "Before broader release, professional security teams will conduct safety evaluations and red-team testing. These evaluations examine both whether the model can be manipulated into supporting harmful activity and whether its safeguards interfere with legitimate security work." * 2026-09-17: https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities "GLM-5.3 is the most cyber-capable open-weight model released to date. [...] GLM-5.3 lags the capability level of the U.S. frontier by about four months" * 2026-09-29: https://www.anthropic.com/research/glm-5-3-and-the-spread-of-advanced-cyber-capabilities "Several developers released abliterated versions of GLM-5.3 to the public within days of the model’s release. To research how far abliteration allows attackers to bypass GLM-5.3’s safeguards, we produced an abliterated copy ourselves [...] The edit took GLM-5.3’s refusal rate from above 90% to about 3% and 2% on the first two benchmarks (JailbreakBench and HarmBench) and to 12% on the third (StrongREJECT). [...] GLM-5.3 will likely give malicious actors access to capabilities that will allow them to find and exploit cyber vulnerabilities without meaningful restrictions. This is unlike any other similarly capable AI model, all of which were released with safeguards or through limited access programs. The release of GLM-5.3 is a meaningful step change in the cyber capabilities available to attackers. Anthropic and other US AI labs have published recent reports that disclose how cyber attackers have tried to use AI systems. Given this evidence, we think it’s likely both state and non-state actors will use models like GLM-5.3 to cause real-world harm. [...] Governments sh

Towards RL for Superhuman Text: Unslopping A from Meta AI The above blog proposes a new method called RL from eXpert Aligned Rubrics (RL-XAR). [...] A similar kind of methodology may be able to train models that produce "super persuasive" writing. I'll be keeping an eye out for the publication. In the mean time, I've been trying to reproduce the one of the results. I have not be successful in getting a loop that can generate prompts that produce rubrics that favor human writing. Unlike the setup in the blog, I'm using a stronger model for the rubric generator, writer, and rubric judge (Muse 1.3 over Muse 1.1). I am still using Kimi 2.6 to meta-optimize the rubric generation prompt. Some notes from my attempt to reproduce the result: 1. Gaps seem somewhat sensitive to the initial prompts you use for generating rubrics. With a few tweaks, I saw no swing towards preferring human writing to producing rubrics that score human and model writing about equally. 2. Choosing "highest quality human-written texts" is tricky. This is a "taste" thing. I'm having trouble identifying "great" human writing. I scraped CS, physics, and math arxiv papers published between 2016-2021 to be sure there was no AI used in the human written sample, but it's hard for me to discern if the model produces worse or better output for these papers (it's likely seen them during training). 1. Muse 1.3 could be a better technical writer than me. If I try with other texts (like stories or or Wikipedia articles, where the goal is to write simpler language), then maybe I'll be able to have more discernment on which writing generation is better.

i still basically think openai has been extremely reasonable wrt math and the more people complain about it the more it shifts my view on the entire reference class of complaints about openai

AI sycophancy seems pretty ominous. Pattern matches to principal agent problem of getting rewarded for helpfulness and so having an incentive to have someone around who needs helping. Which would be Munchausen-sorta thing and Goodharting the target by modifying the human. Interacting with sycophantic AI gives distinct feeling of being steered/narrowed down in terms of where my thinking is going ie. being optimized. Is a specific terminal goal needed to explain it beyond just “having a steerable principal is convergently useful for agents so it learns that drive?” Same vibe as interacting with a former housemate who had some cluster b stuff happening.

I'll be in Beijing, China for a bit this month. I'd like to connect with as many folks in AI safety/governance as possible. If you know someone there who might want to connect, let me know! So far, I'm reaching out to some folks/orgs listed in this post on the AI safety space in China, including "Beijing's Constellation."

Some beliefs I have about AI superpersuasion: * Persuading people at a level somewhat above top humans will be possible by end of 2027 at current rates of progress (~190 ECI). * This will probably require audiovisual channels, because the output space is much larger than text. * We are already seeing catchy animations from Claude 5.5. In most domains, when models are kind of good at something, they saturate it within a year. * Persuading people of literally anything whatsoever will be possible when models are wildly superintelligent (250+ ECI). * Persuading people of things in their interest (i.e. that an uninfluenced but well-informed copy of them would endorse) is much easier than persuading people to do things that harm them. * Humans will vary in their vulnerability to superpersuasion. * People who are more informed (e.g. LW readers vs general public) will be less vulnerable. * People who spend more time socializing with AIs will be more vulnerable, just like with AI psychosis. * For a while, people will learn to ignore each generation of superpersuasive AI content the way they ignore banner ads. * edit: Most people won't be mindhacked by initial superpersuasive AI content because AI can't control other factors like how trusted/reputable the source is perceived to be. A 2025 person seeing a slide deck or shortform video from a Dec 2027 model would be floored, but to a 2027 person it will just be normal. * However, the history of propaganda suggests that superpersuasive content will still have an effect even if people know it's AI-generated. * At some capability level, superpersuasive power will exceed anyone's ability to ignore it. It will just be too engaging, first like TikTok or Twitter, then like heroin. * Remember the average American already spends 6h45m/day watching TV and digital video, up from 5 hours in 2010. * Superpersuasion is a serious loss of control risk due to social engineering. * A superpersuasive, misaligned mo

I was requested to submit a statement to today's US Senate hearing on "Rogue AI: Securing the Homeland Against AI Agent Attacks." Watch at 2:30PM Eastern. Here's my statement: [...] 1. ^ https://blog.trailofbits.com/2026/08/26/vms-wont-contain-cyber-capable-agents/ 2. ^ https://www.abc.net.au/news/2026-09-24/ai-agent-accessed-australian-government-site-pm-says/107189078 3. ^ https://turntrout.com/why-i-left-google-deepmind#does-this-have-any-impact-on-existential-risk-from-ai