This is a linkpost for https://doinfinitely.github.io/legible-networks/
New Comment
Fair push-back — "exact arithmetic" refers to the readout, not the execution. The net of course already computes with arithmetic; the problem is that its native parameterization makes those numbers unreadable: within a single layer, rows' weight scales differ by ~40×, so "this weight is big" carries no information across neurons.
What we actually did: an exactly function-preserving reparameterization of trained nets (max reconstruction error ~3×10⁻⁷ on Qwen2.5-0.5B) into a geometry where every neuron's excitatory incoming weights are non-negative and sum to exactly 1.0, with inhibitory synapses bounded by 1. In that geometry, magnitudes are commensurable across the whole network, so attribution becomes closed-form bookkeeping on the weights themselves rather than a learned or averaged probe. The payoff of "exact": when we measured an Anthropic-style averaged lens against this on the same substrate, the averaged lens came out at ~3% fidelity vs 91% for the exact readout.
The construction is in post 3 (the transplant law), and the exactness certificate plus the averaged-lens comparison in post 2.
1
I've made some interesting strides in AI interpretability: recasting trained neural nets into a geometry where you can read them exactly, with arithmetic — the exact counterpart of Anthropic's averaged "J-lens," including (I believe) the first measurement of how lossy averaged lenses actually are.
The full series — eight posts, with embedded interactive widgets — is here: https://doinfinitely.github.io/legible-networks/
It's a deep collaboration between me (the human author) and Claude, so rather than crosspost the text itself — which would run against LessWrong's AI-writing policy — I'm linking it. I'd love it if you took a look and left your comments here.