Yesterday, Goodfire released the companion technical report to Nick Cammarata’s viral interpretability essay "Neural Networks Are Already Clean", but it seems to have gotten buried in all the recent crazy AI news.

The technical report is titled: "Neurons as an Interpretable Basis". Goodfire's Twitter announcement for it can be found here.

Like Nick Cammarata's essay, it's full of inventive, oft-interactive data displays. Don't just feed this to an AI agent and assume they can grok the post by pulling a PDF.

Quoting from the Abstract: (emphasis mine)

We study the neurons of Qwen3.6-35B-A3B using sparsification methods, attribution tools, and research agents, and find three properties that make neurons interpretable units of computation:

First, they are already functionally sparse. Ablating the 92.2% weakest MoE neuron activations for each forward pass has only modest effects on model capabilities – a 2.9% increase in language model loss.

Second, neurons have relatively few semantic groupings (modes) of computational functions, and we can identify these and describe them concisely. The majority of neurons have either one mode only or a single mode that is overwhelmingly causally significant. In cases where multiple causal modes are present, the number is small and legible to our methods.

Finally, we can use agent-driven experimentation to generate neuron descriptions in natural language, and these descriptions allow us to predict, simulate, and activate the neuron. Providing the description to a judge model allows us to (1) predict whether the neuron will fire in a given passage of text with 95% accuracy vs random text; (2) simulate its token-by-token behavior in high-activation regimes sufficiently to recover 73% of the damage (as a proportion of a ceiling of damage restorable) from the neuron’s ablation; and (3) write text that causes the neurons to fire more strongly than anything observed in a 10B-token corpus.

Overall, we argue that neurons are meaningful units of computation that can be effectively used for interpretability in their own right, letting us study the model on its own terms rather than through a learned proxy.

You may also recall the saga of the Colorado neuron that also activated somewhat on the Tornado Python library.

Did that show neurons really are dirty after all? The paper includes an appendix which analyzes that neuron further. They find:

About 98.8% of the neuron's causal effect comes from text about Colorado, other mountainous places, or other locations, decreasing as places become less Colorado-like.

(...)

Tornado is not one of a large set of hidden secondary functions. (...) On a code corpus, 98% of strong firings were on Tornado or code that calls it.

The two computations interact very little. Tornado predictions run mostly through code and web-library neurons. (...) Colorado and Tornado route 2% of their relevance through shared neurons, comparable to what Colorado shares with unrelated web libraries.

(...)

We do not know why the Tornado association exists. We suspect Qwen3-32B was initialized by copying layers of Qwen3-14B, since many neurons align semantically 24 layers apart.

(...)

We consider this a benign form of overlap. The two modes are easy to identify and separate: clustering the neuron's firings by their upstream inputs separates Colorado from Tornado perfectly, and the two computations meet only in a narrow pathway around layer 14 and its copy. Tornado also has its own representation among code neurons, so an analysis starting there would reach the overlap quickly.

There is much more to digest here, but I think this work continues to look like a promising direction for interpretability.