Lucius Bushnaq

Announcing Apollo Research

TL;DR 1. We are a new AI evals research organization called Apollo Research based in London. 2. We think that strategic AI deception – where a model outwardly seems aligned but is in fact misaligned – is a crucial step in many major catastrophic AI risk scenarios and that detecting deception in real-world models is the most important and tractable step to addressing this problem. 3. Our agenda is split into interpretability and behavioral evals: 1. On the interpretability side, we are currently working on two main research bets toward characterizing neural network cognition. We are also interested in benchmarking interpretability, e.g. testing whether given interpretability tools can meet specific requirements or solve specific challenges. 2. On the behavioral evals side, we are conceptually breaking down ‘deception’ into measurable components in order to build a detailed evaluation suite using prompt- and finetuning-based tests. 4. As an evals research org, we intend to use our research insights and tools directly on frontier models by serving as an external auditor of AGI labs, thus reducing the chance that deceptively misaligned AIs are developed and deployed. 5. We also intend to engage with AI governance efforts, e.g. by working with policymakers and providing technical expertise to aid the drafting of auditing regulations. 6. We have starter funding but estimate a $1.4M funding gap in our first year. We estimate that the maximal amount we could effectively use is $4-6M $7-10M* in addition to current funding levels (reach out if you are interested in donating). We are currently fiscally sponsored by Rethink Priorities. 7. Our starting team consists of 8 researchers and engineers with strong backgrounds in technical alignment research. 8. We are interested in collaborating with both technical and governance researchers. Feel free to reach out at info@apolloresearch.ai. 9. We intend to hire once our funding gap is closed. If you’d like to s

218May 30, 2023

Activation space interpretability may be doomed

153Jan 8, 2025

Circuits in Superposition: Compressing many small neural networks into one

131Oct 14, 2024

A List of 45+ Mech Interp Project Ideas from Apollo Research’s Interpretability Team

125Jul 18, 2024

Lucius Bushnaq

Message

AI notkilleveryoneism researcher, focused on interpretability.

Personal account, opinions are my own.

I have signed no contracts or agreements whose existence I cannot mention.

4829

196

396

14y

Rotations in Superposition

TL;DR We experimentally test the mathematical framework for circuits in superposition by hand-coding the weights of an MLP to implement many conditional[1] rotations in superposition on two-dimensional input features. The code can be found here. This work was supported by Coefficient Giving and Goodfire AI 1 Introduction Figure 1: The...

Dec 15, 202554

From SLT to AIT: NN generalisation out-of-distribution

TL;DR: This post derives an upper bound on the prediction error of Bayesian learning on neural networks. Unlike the bound from vanilla Singular Learning Theory (SLT), this bound also holds for out-of-distribution generalization, not just for in-distribution generalization. Along the way, it shows some connections between SLT and Algorithmic Information...

Sep 4, 2025114

Circuits in Superposition 2: Now with Less Wrong Math

Summary & Motivation This post is a continuation and clarification of Circuits in Superposition: Compressing many small neural networks into one. That post presented a sketch of a general mathematical framework for compressing different circuits into a network in superposition. On closer inspection, some of it turned out to be...

Jun 30, 202573

[Paper] Stochastic Parameter Decomposition

Abstract A key step in reverse engineering neural networks is to decompose them into simpler parts that can be studied in relative isolation. Linear parameter decomposition— a framework that has been proposed to resolve several issues with current decomposition methods—decomposes neural network parameters into a sum of sparsely used vectors...

Jun 27, 202547

Proof idea: SLT to AIT

EDIT 15.12.2025: This post is now superseded by this one. It was just a sketch I wrote up to clarify my own thinking, and contains many mistakes. The newer post is the finished product. I think we may be able to prove that Bayesian learning on transformers[1] or recurrent neural...

Feb 10, 202542

Can we infer the search space of a local optimiser?

Suppose we have a system that we suspect is some form of local optimiser. Maybe it's gradient descent training a neural network. Maybe it’s a neural network doing in-context learning. Maybe it's a mind, because we guess that the operation of minds in general is to an extent well-described as...

Feb 3, 202525

Attribution-based parameter decomposition

This is a linkpost for Apollo Research's new interpretability paper: "Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition". We introduce a new method for directly decomposing neural network parameters into mechanistic components. Motivation At Apollo, we've spent a lot of time thinking about how the computations...

Jan 25, 2025108

Load More (7/25)

LESSWRONG
LW

LESSWRONG
LW

Lucius Bushnaq

Lucius Bushnaq

Lucius Bushnaq

Announcing Apollo Research

Activation space interpretability may be doomed

Circuits in Superposition: Compressing many small neural networks into one

A List of 45+ Mech Interp Project Ideas from Apollo Research’s Interpretability Team

Lucius Bushnaq

Rotations in Superposition

From SLT to AIT: NN generalisation out-of-distribution

Circuits in Superposition 2: Now with Less Wrong Math

[Paper] Stochastic Parameter Decomposition

Proof idea: SLT to AIT

Can we infer the search space of a local optimiser?

Attribution-based parameter decomposition

Announcing Apollo Research

Activation space interpretability may be doomed

Circuits in Superposition: Compressing many small neural networks into one

A List of 45+ Mech Interp Project Ideas from Apollo Research’s Interpretability Team

Rotations in Superposition

From SLT to AIT: NN generalisation out-of-distribution

Circuits in Superposition 2: Now with Less Wrong Math

[Paper] Stochastic Parameter Decomposition

Proof idea: SLT to AIT

Can we infer the search space of a local optimiser?

Attribution-based parameter decomposition