Dmitrii Kharlapenko

Wiki Contributions

Comments

Sorted by

Do you mean SAE encoder weights by input features? We did not look into them.

Thanks! We did try to use it in the repeat setting to make the model produce more than a single token, but it did not work well.

And as far as I remember it also did not improve the meaning prompt much.