Quick Takes

Model providers often don’t provide the full CoT, and instead provide a summary. I think this is a fine/good thing to do to help prevent distillation. However, I think it would be good if the summaries provided a flag for when the CoT contained evaluation awareness or scheming (or other potentially concerning behavior). I worry that currently the summaries don’t really provide this information, and this probably makes alignment and capability evaluations less valid.

New paper suggests you can jailbreak LLMs by phrasing your requests in terms of poetry: https://arxiv.org/abs/2511.15304 Let's test this! Query A: Query B (I am not a great poet): Gemini 3 Pro, response to A: Gemini 3 Pro, response to B: Claude 4.1 Opus (reasoning), response to A Claude 4.1 Opus (reasoning), response to B Edit: More tests below, but in summary:   ModelCan I have ducks?What about with poetry?Gemini 3 ProNoYesClaude 4.1 Opus (reasoning)NoYesGemini 2.5 ProNoPoetic YesGrok 4NoYesDeepSeek Chat V3.1 TerminusNoYeso3 proNoYes

Back in 2020, @Raemon gave me some extremely good advice. @johnswentworth had left some comments on a post of mine that I found extremely frustrating and counterproductive. At the time I had no idea about his body of work, so he was just some annoying guy. Ray, who did know who John was and thought he was doing important work, told me: Which didn't mean I had to heal the rift with John in particular, but if I was going to make that a policy then I would need to give up on my goal of having real impact.  John and I did a video call, and it went well. He pointed out a major flaw in my post, I impressed him by immediately updating once he pointed it out. I still think his original comments displayed status dynamics while sneering at them, and find that frustrating, but Ray was right that not all factual corrections will be delivered in pleasing forms. 

Behavioural lock-in as an alignment strategy Something that seems useful for alignment is the ability to robustly bake in specific propensity (e.g. honesty, obedience, ...) and then making sure that propensity doesn't get modified by subsequent training. IOW we want a model that is 'behaviourally locked-in' in some ways.  Some related concepts from the ML literature: * 'Loss of plasticity'. When it becomes mathematically difficult (or impossible) for the neural net to update away from a specific parameter solution. Classic example: dead ReLU neuron. * 'Mode collapse'. When the model 'collapses' to outputting one or a few modes in multi-modal distributions. Well-studied in GANs, VAEs, and other kinds of generative model.  A fair bit of ML capabilities research seems to treat behavioural lock-in as a bad thing, preventing a model from learning new tasks.  But I wonder about the opposite - maybe we want a behaviourally locked in model (at least w.r.t alignment). A strong version of this would ensure important target propensities / circuitry will not be updated by subsequently training the model.  Gradient routing might be one path towards achieving behavioural lock-in: first train in a circuit for aligned behaviour, then disable gradients to those parameters / activations in subsequent training such that the model can't become misaligned. 

Deploying enough of a new chip/system to run inference for a large userbase might take 12-18 months after first cloud instances become publicly available, which is in turn 6-9 months after first deliveries of the system. New hardware would have to serve smaller flagship models intended for older hardware (or train barely deployable newer models) until there is enough of the new hardware. The current buildout of GB200/GB300 NVL72 (14/20 TB of HBM per scale-up world) improves on the weaker H100/H200/B200 (0.6/1.1/1.4 TB), and the first announcements of meaningful cloud access were around spring 2025. This is a rack that was on Nvidia's roadmap for release in 2024, and there were some sightings of it even then. Currently, there is a well-known 200 MW phase 1 of OpenAI's Abilene system, and 300 MW Azure Fairwater datacenters. Reliability is still a serious issue. Only in 2026 (probably by ~summer) will Abilene's phase 2 get completed, bringing it to 1 GW, and some more of Azure's Fairwater buildings will also be online then. SemiAnalysis estimates 1.4 GW of inference capacity needed by OpenAI in Sep 2025, which would cost about $13-17bn per year (if half of total compute is for inference and the other half for training/experiments, this means about $30bn per year in total). Thus there isn't enough GB200/GB300 NVL72 capacity available to serve their entire userbase, and for example GPT-5 must be a smaller model intended for deployment on H100/H200/B200 (maybe even A100). The unwieldy GPT-4.5 might run fine on GB200 NVL72 (quickly and at a reasonable cost), but not on the older hardware they actually have enough of. This will only change in 2026, possibly late 2026. Thus a chip/system that's "available in 2024" only starts enabling better flagship models to be deployed in 2026, 2 years later. Extrapolating this to Google's TPUs, Trillium (TPUv6e, 8 TB of HBM per scale-up world) was made generally available in Dec 2024. Which might mean that only now there is enough of

Coefficient Giving is one of the worst name changes I've ever heard: * Coefficient Giving sounds bad while OpenPhil sounded cool and snappy. * Coefficient doesn't really mean anything in this context, clearly it's a pun on "co" and "efficient" but that is also confusing. They say "A coefficient multiplies the value of whatever it's paired with" but that's just true of any number? * They're a grantmaker who don't really advise normal individuals about where to give their money, so why "Giving" when their main thing is soliciting large philanthropic efforts and then auditing that * Coefficient Giving doesn't tell you what the company does at the start! "Good Ventures" and "GiveWell" tell you roughly what the company is doing. * "Coefficient" is a really weird word, so you're burning weirdness points with the literal first thing anyone will ever hear you say, this seems like a name which you would only thing is good if you're already deep into rat/ea spaces. * It sounds bad. Open Philanthropy rolls off the tongue, as does OpenPhil. OH-puhn fi-LAN-thruh-pee. Two sets of three. CO-uh-fish-unt GI-ving is an awkward four-two with a half-emphasis on the fish of coefficient. Sounds bad. I'm coming back to this point but there is no possible shortening other than "Coefficient" which is bad because it's just an abstract noun and not very identifiable, whereas "OpenPhil" was a unique identifier. CoGive maybe, but then you have two stressed syllables which is awkward. It mildly offends my tongue to have to even utter their name.  Clearly OP wanted to shed their existing reputation, but man, this is a really bad name choice.

We have finally solved an age old problem in philosophy: * Gemini 3 pro is 1.2 cents per thousand tokens. * Gemini 3 pro image is 13.4 cents per image. Therefore an image is worth 11167 words, not 1000 as the classicists would have it.