When thinking about the impacts of AI, I’ve found it useful to distinguish between different reasons for why automation in some area might be slow. In brief:  1. raw performance issues 2. trust bottlenecks 3. intrinsic premiums for “the human factor” 4. adoption lag 5. motivated/active protectionism towards humans I’m posting this mainly because I’ve wanted to link to this a few times now when discussing questions like "how should we update on the shape of AI diffusion based on...?". Not sure how helpful it will be on its own! (Crossposted from the EA Forum.) ---------------------------------------- In a bit more detail: (1) Raw performance issues There’s a task that I want an AI system to do. An AI system might be able to do it in the future, but the ones we have today just can’t do it.  For instance: * AI systems still struggle to stay coherent over long contexts, so it’s often hard to use an AI system to build out a massive codebase without human help, or write a very consistent detective novel. * Or I want an employee who’s more independent; we can get aligned on some goals and they will push them forward, coming up with novel pathways, etc.  * (Other capability gaps: creativity/novelty, need for complicated physical labor, …) A subclass here might be performance issues that are downstream of “interface mismatch”.[1] Cases where AI might be good enough at some fundamental task that we’re thinking of (e.g. summarizing content, or similar), but where the systems that surround that task or the interface through which we’re running the thing — which are trivial for humans — is a very poor fit for existing AI, and AI systems struggle to get around that.[2] (E.g. if the core concept is presented via a diagram, or requires computer use stuff.) In some other cases, we might separately consider whether the AI systems has the right affordances at all.  This is what we often think about when we think about the AI tech tree / AI capabilities. Others are of

Flagship models need inference compute at gigawatt scale with a lot of HBM per scale-up world. Nvidia's systems are currently a year behind for serving models with trillions of total params, and will remain behind until 2028-2029 for serving models with tens of trillions of total params. Thus if OpenAI fails to access TPUs or some other alternative to Nvidia (at gigawatt scale), it will continue being unable to serve a model with a competitive amount of total params as a flagship model until late 2028 to 2029. There will be a window in 2026 when OpenAI catches up, but then it's behind again. The current largest flagship models are Gemini 3 Pro and Opus 4.5, probably at multiple trillions of total params, requiring systems with multiple TB of HBM per scale-up world to serve efficiently. They are likely using Trillium (TPUv6e, 8 TB per scale-up world) and Trainium 2 Ultra (6 TB per scale-up world), and need north of high hundreds of megawatts of such systems to serve their user bases. Nvidia's system in this class is GB200/GB300 NVL72 (14/20 TB per scale-up world), but so far there isn't enough of it built, and so models served with Nvidia's older hardware (H100/H200/B200, 0.6-1.4 TB per 8-chip scale-up world) either have to remain smaller or become more expensive. The smaller amount of NVL72s that are currently in operation can only serve large models to a smaller user base. As a result, OpenAI will probably have to keep the smaller GPT-5 as the flagship model until they and Azure build enough NVL72s, which will happen somewhere in mid to late 2026 (the bigger model will very likely get released much earlier than that, perhaps even imminently, but will have to remain heavily restricted by either price or rate limits). Paradoxically, xAI might be in a better position as a result of having fewer users, and so they might be able to serve their 6T total param Grok 5 starting early 2026 at a reasonable price. But then in 2026, there is a gigawatt scale buildout of Iron

I'm considering writing a follow-up FAQ to my pragmatic interpretability post, with clarifications and responses to common objections. What would you like to see addressed?

I write little poems on the tram to work. This one is kind of old, but (for perhaps obvious reasons) LW-relevant. --- Light-shaper, sky-weaver, dream-waker, bringer of grief and heart-ache maker, idol-smasher, truth-teller, hand-shaker, gray-hearted one, the hollow caretaker. Gold-dangler, lie-spinner, truth-slayer, weaver of fate, the iron betrayer. Fire-feeder, song-ender, world-changer, depth-seeker, the blind star-arranger.

I find the METR task length measurements very helpful for reasoning about timelines. However, they also seem like approximately the last benchmark I want the labs to be optimizing: high agency over long timescales, and also specifically focused on tasks relevant to recursive self-improvement. I’m sure that the frontier labs can (and do) run similar internal evals, but raising the salience of this metric (as a comparison point between models from different companies) seems risky. What has METR said about this?

SpaceX is amazing. As best as I (and Claude) can tell, the situation is as follows: Competent competitors to SpaceX are developing rockets that should provide around $2000/kg cost-to-orbit. This is a big improvement over legacy space competitors like ULA, Ariane, etc. which range from $5000/kg to $50,000/kg. (It's amazing that those competitors are still getting any business... the answer is nepotism/corruption basically afaict) However, these competent upcoming rockets that can do around $2000/kg? They aren't ready yet. Probably it'll be like 5 more years before they really solidly hit that milestone at scale. SpaceX, meanwhile, has already hit that milestone with Falcon 9 and has been there for years. It's why they have Starlink and nobody else does. But that's not all. SpaceX is working on Starship, which is afaict about as close to being finished as the aforementioned competitor rockets, and when it is finished it'll should provide somewhere between $15/kg and $150/kg. So... ONE TO TWO ORDERS OF MAGNITUDE CHEAPER STILL. This is a really big deal. The reason space looks the way it does today (mostly empty, a few satellites and probes, two space stations) is that it's so damn expensive to go there. SpaceX brought costs down by an OOM already and this unlocked Starlink already, and probably in the fullness of time will lead to 10x more of the other things as well (10x bigger space stations for example). Or heck, could be more than 10x; when the costs drop by 10x you often don't merely do 10x more of the same thing, you do even more than that because now a bunch of stuff is profitable that wasn't profitable before. And then once Starship is online we'll get another 1-2 OOMs on top of that. We won't have finished adjusting to the price drop from Falcon 9 (which only happened over the course of the last decade), when we'll be hit by another even bigger price drop! Intuition pump: A typical house weighs something like 50,000 kg. and costs about $300,000 to build

I analyzed the research interests of the 454 Action Editors on the Transactions on Machine Learning Research (TMLR) Editorial Board to determine what proportion of ML academics are interested in AI safety (credit to @scasper for the idea). * 14% of editors listed any of "robustness", "trustworthy", "interpretable", "safety", "alignment", "evaluation", or "security" as a research interest. * 10% of editors listed any of "trustworthy", "interpretable", "safety", "alignment", "evaluation", or "security" as a research interest. I excluded "robustness" as much of this research is considered more "capabilities" than "safety". * 3.7% of editors listed "safety" or "alignment" as a research interest.