by Narmeen Oozeer, Dhruv Nathawani, Nirmalendu Prakash, Amirali Abdullah

This work was done by Narmeen Oozeer as a research fellow at Martian, under an AI safety grant supervised by PIs Amirali Abdullah and Dhruv Nathawani. Special thanks to Sasha Hydrie, Chaithanya Bandi and Shriyash Upadhyay at Martian for suggesting researching generalized backdoor mitigations as well as extensive logistical support and helpful discussions.

TLDR:

We show that representations across models of different sizes are weakly isomorphic when trained on similar data, and that we can "transfer" activations between them using autoencoders.
We propose a technique to transfer safe behavior from one model to another through the use of steering vectors.
Our representation transfer technique paves the way for transferring insights across LLMs, saving efforts and potentially compute.

Context

Large Language Models (LLMs) have demonstrated the capacity for strategically deceptive behaviors, adapting their actions to appear helpful in some contexts while acting deceitfully in others. Hubinger et. al. [1] demonstrate in their sleeper agents work how backdoor behaviors resist fine tuning based attempts at mitigation. Price et. al [21] further explore temporal vulnerabilities in LLMs and demonstrating that current LLMs can distinguish between past and future events - a capability that can be exploited to introduce backdoors. Advancements in LLMs may exacerbate rather than resolve these issues. Bowen et. al. [2] reveal an inverse scaling law whereby larger and more capable models are better at generalizing and reproducing backdoor behavior they have learnt!

Mechanistic interpretability, aimed at decoding the internal algorithms of models, can help us understand how backdoor behaviors are evoked and potentially suppressed. However, one key challenge is understanding and transferring insights from smaller models to larger, more complex ones. The difficulties arise due to several intertwined factors in larger models:

Increased Redundancy:
Larger models tend to exhibit significant redundancy in the roles performed by their components, making it challenging to isolate specific mechanisms, as shown by the Hydra effects paper [3].
More Complex Algorithms:
The algorithms that govern larger models are typically more intricate, adding another layer of difficulty to their anal

...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments