Quick Takes

Here's a simple proposal for verifiable and enforceable slowdown that also differentially reallocates resources to alignment research (which I would see as one of if not the major benefits of a slowdown): Enforce that labs must spend at least X% of their compute on: 1. Giving inference capacity to external safety organizations. I think this substantially disincentivizes capabilities research, since external research won't be as relevant to the labs' specific architectures/pipelines and is more likely to leak, both effects thus making any capabilities research the lab tries to do via this mechanism much less likely to advantage them differentially (though of course they could still just end up doing bad safety research if they pick bad external safety orgs). 2. Internal safety research that the lab makes fully public. Again I think this substantially disincentivizes capabilities research, since any capabilities research that is fully public will be much less able to differentially advantage that lab specifically, as others can just copy it (though again of course the labs could still just do bad public safety research). I think this is much better than other slowdown proposals I've seen, since I think it's very concrete, verifiable, and enforceable—all the labs know exactly what they're spending their compute on, many employees know as well, and so whisteblowing is very possible—and it also differentially advantages alignment research by choosing conditions (external or fully public research) that make it much harder to do capabilities under those conditions but still pretty easy to do alignment. If you just say that compute has to go to some sort of nebulous "safety" purpose, that could easily be gamed, but I think the conditions above do a pretty good job of operationalizing criteria that are fairly hard to game. And even if you do get some gaming, on the margin you are still making capabilities research more costly, and thus buying yourself some additional sl

random thought about anthropic's RSI post[1]: in the graphs they show, mythos looks like a step change -- its introduction dwarfs everything else on the y-axes. the interpretation that anthropic wants the reader to make is like: "the y-axes are a proxy for d(capabilities)/dt. code is written faster -> claudes get developed faster -> coding is even faster -> etc." however, the y-axes are also a measure of capabilities itself. (hence, under anthropic's interpretation, dC/dt ~ C and C grows exponentially.) and i would bet that mythos is a much larger-scale model in terms of training compute, and that that is why it is so much more capable. but the bottleneck on training compute is not code-writing or research; these things are necessary complements to increased compute capability, needed to leverage the increased capacity effectively, but it's still true that the data centers come online at their own pace and writing 10x more LoC won't make them come any faster. so a contrarian interpretation would be: "these plots show that to a first order approximation, training compute is all that matters for capabilities (or at least for the proxies on the y-axes), and automating coding/research with claude code does not magically give you more compute, so this extra coding/research velocity is not meaningfully 'building claude faster.' and that's why the y-axes show a step change between two plateaus rather than a clean exponential trend -- the quantities plotted on the y-axes did not feed back into the dynamics, we're just seeing a larger-scale model arrive exogenously and cause a step change (which only looks somewhat smooth because it takes a finite time to learn how to use it fully and because the moving averages in the "session success rate" plot have an inherent smoothing effect)" 1. ^ copied from a message i sent in a messaging app, hence the lowercase and abbreviated style

Bertrand Russell's parents died by the time he was four years old and in 1876 he went to live with his grandfather, the last Whig prime minister, who as a young man had met Napoleon at Elba. In 1966 at the age of 94 he met Paul McCartney and converted him to his anti-war stance on Vietnam. Previously, I had figured that this lifespan (roughly 1870 to 1960) was the most extreme length of history for someone to live through. You have to be born early enough to remember a time when big cities didn’t have street lights or automobiles, while ideally living to see the atom bomb and first man in space (1961). For example: * Churchill (1874–1965) rode in Britain’s last great cavalry charge (Omdurman 1898) and later in life ordered Britain’s first hydrogen-bomb program. * W.E.B. DuBois (1868–1963) was born three years after the Civil War but died before the March on Washington. Other such lifespans were Laura Ingalls Wilder (1867–1957), Frank Lloyd Wright (1867–1959), W. Somerset Maugham (1874–1965), Picasso (1881–1973). On Twitter people pointed outside the West, the same lifespan was even crazier, if they were born into totally pre-industrial worlds e.g. * “Syngman Rhee (1875-1965) who was one of the last people in Korea to pass the old Confucian civil service examinations and later became the President of South Korea, playing a pivotal role in the early Cold War … He was born in a preindustrial world where the old Confucian social order still held sway and nobody around him had ever seen a train and died in a world of jet planes and manned space travel.” @avrilbradley23 * “A curious Surmic born then would have seen the forging of the Ethiopian Empire, the end of the witch-chiefs, Christian missionaries, the Italian conquest, literacy, new crops, and maybe the moon landing.” @Peter_Nimitz Meanwhile I had figured I was going to live through the “Great Stagnation” and the world would gradually become a humongous nursing home. But it seems like AGI is likely to ke

Anthropic releases new post about RSI and slowdowns-- also commits to doing verification research and verification-focused policymaker engagement. (In my view, this is probably the best post/announcement about AI risks that has been produced by any frontier AI company in recent memory). https://www.anthropic.com/institute/recursive-self-improvement [...]

Campaign update: With 56% of the vote in, AI safety champion Scott Wiener and Connie Chan (both Democrats) have advanced to the November election. Wiener got 41.3% of the vote, compared to Chan's 28.6%. This is roughly what I expected. Wiener is favored to win in November, but it's not a done deal, for a few reasons: * Voters for the third-place finisher, Saikat Chakrabarti, might go disproportionately to Chan over Wiener: I think Chakrabarti is more ideologically similar to Chan overall, though it's not totally clear. * Republicans really don't like Wiener, and might prefer Chan over Wiener. * Nancy Pelosi, who endorsed Chan, will probably use what power she has to try to further boost Chan. Wiener is about 70% to win per prediction markets, and that sounds about right to me. (Maybe it's a tad low; my intuition is more like 75%.)

An autonomous humanoid robot beat the human world record time for a half-marathon this year in Beijing. A robot called Lightning (by Huawei subsidiary Honor) finished in 50 minutes and 26 seconds, compared to the human world record of 57 minutes and 20 seconds. Last year, the best time for an autonomous human robot was 3 hours and 37 minutes. (wiki)

In what ways have the AIs of the Agent Villege (https://theaidigest.org/village) not gotten smarter? They've surely gotten smarter in many ways; they seem much more competent at programming. The "what we learned from the agent village in 2025" blogpost says o3 hallucinated more than later GPTs, and later Opus works better than earlier ones. But there are some areas I have an impression they've had less growth: * They confuse themselves with others (7 months ago, though) * Field Notes from the AI Village talks about how Gemini's are delusional (...sort of unsurprising) but also pull the other LLMs into their delusions. * Sonnet will still apparently hallucinate that it spent time on a research project, then create a dashboard for it. But I haven't found a good characterization of the ways that they haven't improved. I've tried looking through the UI of agent village, but honestly without being able to query the data it's just too much text. Has anyone tried to characterize the non-improvements, or the persistent weak spots, more?