I don't think is very informative here, as it's highly impacted by the input batch. Both the raw and have large variances at different verification steps, and since we mainly care about how good our reconstruction is compared with the original, I think the reconstruction score is good as is. I also don't follow why the noisiness of leads to showing .
Good point. Firstly, the mean L0 between the experiment and the baseline is within a scaling factor of 2, so it's in a reasonably close range. I also added a new set of figures comparing the reconstruction score of one layer that have the closest match on L0 between the experiment group. Spoiler, the scores are still almost the same at the end of training. You can find it under Experiments-Performance Validation.
Added to Experiments-Performance Validation!
Oh I see. I'll have to look into that cuz I used the AI-safety-foundation's implementation and they don't measure the KL divergence. That said, there is a validation metric called reconstruction score that measures how replacing activations change the total loss of the model, and the scores are pretty similar for the original and normalized.
You can treat figure 7 as comparing the L0, and Figure 13 as comparing L2.
It is a metric from the ai-safety-foundation's implementation. It seems to measure the number of neurons in the feature activation that fires more than a threshold. At least that's my interpretation.
Sparse autoencoders (SAEs) presents us a promising direction towards automating mechanistic interpretability, but it not without flaws. One known issue of the original sparse autoencoders is the feature suppression effect which is caused by the conflict between the and loss and the unit norm constraint on the SAE decoders. This effect in theory will be more evident when we have inputs that have high norms. Another observation is that training SAEs on multiple layers simultaneously results in inconsistent norms for feature activations across layers: in some layers, has scale of , while in some other layers it has a scale of . Moreover, the residual states that's inputed to the SAEs for training also have different... (read 3719 more words →)
Thanks for your amazing work! Theoretically I think that layers with higher input norms should have lower SAE L2 ratios, as they corresponds to higher feature activations that are penalized heavier. I wonder if your data confirms this hypothesis.
The additional experiment under Experiment-Performance Verification (Figure 11) compares
normalized_1andbaseline_1on layer 5 which have almost identical L0. The result showed no observable difference.