Quick Takes

On January 1st, 2026, I started focusing full time on technical alignment research. Well, to be more accurate, I started laying the groundwork for doing technical alignment research. Although I have a solid and deep technical background, mostly in computer science and software engineering, my background is not very wide and was missing a bunch of important pieces. So the year so far has involved a lot more studying than thinking new thoughts at the frontier. I'm working with John Wentworth on natural abstractions. It's my full time job to understand what he's been up to, but it's just a side gig for him to explain it to me. A few hours of explanations from him gives me ~a week of material to chew on. As I've been building all these background models, I've discovered that John has a lot of information and thought-structures built up in his head that he's never properly written down or explained. One of my tasks this year will be to write those down properly and post them here. I mostly think of those write-ups as prelude to the real work, but you never know. Sometimes when you take the time to write down your background models, you realize that there was a missing or slightly off-kilter piece, and fixing that up leads to new insights. We'll see.

The AI safety ecosystem is so well resourced that it has been correctly identified by many as one of the best paths into high prestige AI research jobs. This person on twitter has written a popular article about getting into frontier ai labs and a Field Guide to AI Fellowships. The "AI Fellowships" are mostly AI safety programs funded by CG/OpenPhil. I have also noticed that people in ML research are quite likely to have heard of MATS and be interested in participating, even when they have very little interest in AI safety. Idk how good or bad this is. It definitely causes a lot of ML researchers to engage with the AI safety literature, where they otherwise would not have. But it's worth noting that while the primary driver of applications to programs like MATS used be concern about AI safety, now it is increasingly a desire to work at a frontier lab. I used to select participants for LASR labs, one of programs listed, and we actively tried to choose people who cared about AI safety, but I think we often did not succeed and indeed some people on the program now work at frontier labs in roles that I think have little to do with safety. Safety-washing in practice mostly looks like people rationalizing research or jobs that they would have done anyway as necessary for safety. It's easy to do because it is in fact genuinely unclear in most cases what is helpful and harmful for safety. Only by looking at the larger pattern can we notice the suspicious abundance of conveniently overlapping opportunities that further both safety and a person's short-term interests. However, I think it will be less common in people whose primary motivation is to prevent AI catastrophe.

Playfulness, like many social stances, sometimes undergoes a critical transition. People hang out, someone does or says something a little playful, someone else responds playfully. Sometimes the response is a little *less* playful, and if that goes back-and-forth, then the playfulness dies out (left to its own devices). Other times the response is a little *more* playful, and if that goes back-and-forth, then silliness escalates until things get big and interesting. When I think about recent times when I've brought some playfulness into my social environment, e.g. at parties, I notice that almost everyone responds in subcritical ways. So if I want to have a fun playful time, I can drive that by pumping in lots of playfulness myself (e.g. sitting on the roof at LessOnline tossing a big ball to passersby), but it feels like I'm pushing against a current. The silliness doesn't take on a life of its own. Me at LessOnline There are exceptions to this. For instance, a few months ago Garrett and I had some supercritical back-and-forth over a fire pit at Lighthaven. We ended up cooking a lasagna noodle and convincing a slightly-drunk Jesse Hoogland to put the noodle in his pocket for five minutes to test whether it turned into a sticky mess. But that sort of thing is the exception to the rule in the Berkeley rationalist/EA community, as far as I can tell. I think this is one of the major things I want in a community, that I don't get from this one. In John's ideal community, if someone is on a roof tossing a ball to passersby, other people naturally show up and are like "ooh fun" and decide to join in for a while, and then a little crowd grows in the courtyard/on the roof/on the opposite balcony, all hitting this ball around. And then eventually someone might e.g. run off and find a net, hang that up, and like ten or twenty people end up playing two-story volleyball with the blue ball in the courtyard. Or maybe someone shows up with squirt guns and those get added to

(Crossposted) About a month ago I left my job. I plan to take the next six months (until December) as a sabbatical. Partly because I don't think I've taken more than two weeks' holiday at a time since 2012, when I got my first real job. It'll be nice to have a longer break. Partly because, man, the world is not standing still. 2026-2031 is going to look very different from 2016-2021, and probably I should spend a bit of time orienting to "what do I even do with my life, going forwards". It's not impossible for me to do that while I have a full time job, but I do think I'll be better at it without one. I don't plan to make any long term commitments until December. But some things I'd like to do in that time: * See if I can help with AI notkilleveryoneism. * Do some kind of study course. * The slightly-more-concrete fields that I think I'm most likely to get into are mech interp (possibly working directly with my girlfriend), agent foundations, and brain-like-AGI safety. * I don't want to pay much attention to "what can I get paid for" right now. That seems like it would distort my thinking, and my sense is that the funding landscape is likely to look very different in six months anyway. If it turns out that I think I can do something worthwhile but I can't get paid to do it, I'll have to figure out what to do about it later. * Blog more. (Vaguely aiming for 10 posts before December, including this one.) * Write some fiction. * Get better at LLM-assisted coding. (I'll probably need to start paying for an LLM subscription. I'm a little sad about that because I don't want the labs to get my money, but I don't think that being a lonely holdout here would help much.) It's been a month and I've done… approximately none of that. Partly that's because I've spent 2-3 weeks out of four either on holiday, preparing for holiday, or recovering from holiday. But I think it's time for me to buckle down a bit. I'm going to try "get something productive done approx

As per the advanced chess obituary, we have a rough idea of the length of the centaur stage for chess. But what do we know of the length of the centaur stage for other games? I sent off Claude 4.6 Sonnet for a deep research query, here's the result (sorted by domains with an identifiable gap on top): (Claude-generated table starts here [1] ) Domain AI ≈ human (year) Centaur stage start Centaur stage end Calendar duration Evidence quality Post-centaur exploitation Notes Go 2016 2016 ~2017 ~0–1 yr Low Yes: Wang et al. 2022 Community consensus, no primary tournament data Chess ~1997 ~1998 ~2013–2016 ~15–18 yr Medium unknown Advantage eroded continuously from ~2009 Protein (single-domain) ~2020 (AF2) ~2018 ~2020–2022 ~2–4 yr High unknown CASP15: no significant human advantage on single-domain Weather forecasting ~2003 (ensembles) unclear ~2003–2005 very short / nil High unknown Humans beat single models but not ensembles, even in 2003 Checkers 1994 (Chinook) ~1994 ≤2007 <13 yr (upper bound) Low unlikely (game solved) Tinsley drew Chinook 1994, winning in 1990; weakly solved 2007 sets hard upper bound Radiology (ICH) not yet ~2019–2020? ongoing 5+ yr and counting High N/A (Type 1 ongoing) AI still substantially below human parity as of 2024 Dermatology ~2018 ~2019–2020 ongoing ~5–6 yr and counting High N/A (Type 1 ongoing) Meta-analysis n=67,700; human+AI > human alone Protein (multi-chain) not yet ~2020? ongoing 4+ yr and counting High N/A (Type 1 ongoing) Significant human advantage on assembly targets (p=0.029) Software function completion ~2021–2023 unclear ongoing? unclear Medium N/A (Type 1 ongoing?) Workplace RCT null; lab studies show 42–56% gains Shogi ~2013 ~2013? ~2015? ~2 yr? Very low unknown Speculative; no controlled data found Backgammon ~1992 (TD-Gammon) unknown unknown unknown None unknown No centaur tournament literature found Poker (HUNL) ~2017 (Libratus) unknown unknown unknown None unknown No controlled human+AI vs. AI-alone data found Machine tran

The United States Government has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States. This includes Anthropic employees who are foreign nationals. Anthropic is currently disabling Fable 5 and Mythos 5 for all customers. https://www.anthropic.com/news/fable-mythos-access

A PE teacher once told me that your muscles start atrophying after only a week of not working out, and it's impossible to gain muscle if you don't work out every week. I'm not sure why it took me so long to question this, but my results from a somewhat-consistent but definitely-not-every-week workout plan made it really obvious that this is not true. Claude thinks that as long as you're not literally in a coma it's more like 3 weeks (with variation for age/protein/etc.). This actually makes me more motivated, since "make sure to exercise every single muscle every single week" is not really an achievable target with the level of effort I want to put in, but "make sure to hit at least one muscle group once a week" is pretty easy. (Obviously the target should be higher than this if I want to see gains in any reasonable amount of time, but it's nice to know that if I don't hit my actual target there's a lot more slack than I realized)