Curious if you could elaborate more on why MACHIAVELLI isn't a good test for outer alignment!

Yep, it's a language model agent benchmark. It just feeds a scenario and some actions to an autoregressive LM, and asks the model to select an action.

GPT-4 seems to pass the banana test.