I often read interpretability papers and I come away thinking “ok, but what’s the point? What problem does this help us solve?” So last winter, I organized a MATS/Pivotal stream to build examples of deceptive models (aka “model organisms”). The goal was to build a diverse ‘zoo’ of these model...
This post is a project report from the AI Safety Fundamentals course, spring 2024. TL;DR 1. Transferring debate to an abstract algebra MMLU dataset is not trivial. 2. When GPT-3.5 is used as a judge, the outcomes may be sensitive to exact prompt phrasing. 3. GPT-3.5 may perform worse in...
I often read interpretability papers and I come away thinking “ok, but what’s the point? What problem does this help us solve?” So last winter, I organized a MATS/Pivotal stream to build examples of deceptive models (aka “model organisms”). The goal was to build a diverse ‘zoo’ of these model organisms and empirically test whether white-box methods could help us detect their deceptive reasoning.
Unfortunately, I don’t think our empirical results were very informative. Our model organisms were too toy for me to expect that our results will transfer to the powerful AI (or even current state-of-the-art AI models). But I think we still developed methodological details that might help people build better... (read 4187 more words →)
This post is a project report from the AI Safety Fundamentals course, spring 2024.
A recent paper by Khan et al. shows that arguing for correct information in the debate game is easier. However, since the current language models are trained not to be deceptive, this favorable debate property may go away if the models were actively deceptive. This project works towards measuring the... (read 793 more words →)
As far as I understand, it is possible that the golem foresees this strategy. In this case its future copies will cooperate (sacrifice themselves) until past the point you will be sure they are safe.