x

Hoang Khiem

Message

31

2y

Can SAE steering reveal sandbagging?

Summary * We conducted a small investigation into using SAE features to recover sandbagged capabilities. * We used the Goodfire API to pick out 15 SAE features, 5 related to sandbagging, 5 related to general expertise, and 5 unrelated features. * We used these features to steer a Llama 3.3...

Apr 15, 202535

Hoang Khiem

Subscribe

Message

31

2y

Hoang Khiem

Can SAE steering reveal sandbagging?

Summary * We conducted a small investigation into using SAE features to recover sandbagged capabilities. * We used the Goodfire API to pick out 15 SAE features, 5 related to sandbagging, 5 related to general expertise, and 5 unrelated features. * We used these features to steer a Llama 3.3...

Apr 15, 202535

Can SAE steering reveal sandbagging?

jordine

jordine, Hoang Khiem, Felix Hofstätter, Cleo Nardo

10mo

Summary

We conducted a small investigation into using SAE features to recover sandbagged capabilities.
We used the Goodfire API to pick out 15 SAE features, 5 related to sandbagging, 5 related to general expertise, and 5 unrelated features.
We used these features to steer a Llama 3.3 70b Instruct that has been prompted to sandbag
We observed that steering can recover up to ~25% of true capabilities
We observed that using sandbagging/deception/evaluation-awareness relevant features were slightly better than unrelated features, though results are mixed.
While steering with SAE features partially recovered capabilities, this is not as effective as other existing interventions

Introduction

During our work on linear probing for evaluation awareness, we thought about using evaluation-awareness relevant features to uncover... (read 1031 more words →)

3

35

2

LESSWRONG
LW

LESSWRONG
LW

Hoang Khiem

Hoang Khiem

Hoang Khiem

Can SAE steering reveal sandbagging?

Hoang Khiem

Hoang Khiem

Hoang Khiem

Can SAE steering reveal sandbagging?

Summary

Introduction