Reproduction, Extension & Mitigations Authors: Elizaveta Tennant, Jasper Timm, Kevin Wei, David Quarel In this project, we set out to explore the generality of Emergent Misalignment (via a replication and some extensions) and how easy it is to mitigate. This project was conducted during the capstone week at ARENA (Alignment...
Tl;dr: The AI Safety Student Team (a group of students at Harvard) will be running two 8-week introductory reading groups this summer (in Boston and online), as well as summer socials (in Boston). Apply to our technical fellowship here or our policy fellowship here; express interest in our socials here....
Authors: Elizaveta Tennant, Jasper Timm, Kevin Wei, David Quarel
In this project, we set out to explore the generality of Emergent Misalignment (via a replication and some extensions) and how easy it is to mitigate. This project was conducted during the capstone week at ARENA (Alignment Research Engineering Accelerator) 5.0.[1]
Note: this blog contains examples of harmful and offensive content
We replicate and extend the Emergent Misalignment (EM) paper. We show that severe misalignment via narrow-domain fine-tuning can emerge in smaller (open-source) models and with data from a different domain (dangerous medical advice). We also find that conditional fine-tuning can create misalignment triggers with less data than previously known. We propose one idea for... (read 5078 more words →)
Tl;dr: The AI Safety Student Team (a group of students at Harvard) will be running two 8-week introductory reading groups this summer (in Boston and online), as well as summer socials (in Boston). Apply to our technical fellowship here or our policy fellowship here; express interest in our socials here.
We will host two intro fellowships (reading groups):