Top postsTop post
Daniel Tan
Message
1 July 2026 - I have not signed any contracts that I can't talk about. I'll update this periodically as long as it's true. The next planned update date is: 1 Jan 2027.
3919
Ω
1
16
437
TL;DR We stress-test alignment midtraining (AMT) across model and token budget scales. Our results suggest that midtraining cannot tackle the hard problems of AI alignment—namely distributional shift and reward underspecification in the presence of imperfect data. For instance, we test whether midtrained motivations are robust to finetuning which elicits competing...
Tl;dr I am currently worried about current alignment techniques + how they are applied to frontier models. This decomposes into two hypotheses: * Alignment techniques are not working to address misalignment from RL. * Alignment techniques are actively obscuring evidence about misalignment. I think we do not currently have enough...
Various people have asked me how I made my career decisions, so I wrote up some quick thoughts. (This is mostly intended as a personal reflection, but might be interesting nonetheless to some people here, especially to more junior people trying to figure out their own career options) I spent...
This work was done by Purvi Chaurasia with Daniel Tan and Chloe Li as part of the SPAR Program for Spring 2026. All code related to the blog can be found in this repo. We investigate self-report fine-tuning (SRFT), a technique proposed by Li et al. to improve models' honesty....
This is a link post for the paper preprint: Inoculation Adapters: Improved Selective Generalization of Capabilities with Fewer Surprising Backdoors from the Center on Long-Term Risk. Selective generalization. Training can teach desired and undesired traits at once. Selective generalization aims to preserve the generalization of desired traits while preventing the...
Tl;dr I've been thinking hard and experimenting with how to best take advantage of AI for my work. I've developed some practices, mostly through trial and error. These have helped me spend less effort doing work while getting more done, and so I'm writing them up to share them. A...
I've been wearing an Oura ring 4 for about a month now, and I love it! It tracks several statistics related to sleep, activity, and overall health. Simply looking at these numbers motivates me to make decisions that will increase them. As a result, I've been sleeping and exercising way...