This is cool! I'd be interested to see a fully reproducible version where anybody can download and run the evaluation. Right now, it's hard to use this methodology to compare the abilities of different language models or scaffolding systems because the results are significantly driven by human prompting skills. To compare the abilities of different models and track progress towards language agents over time, you'd want to hold the evaluation constant and only vary the models and scaffolding structures. This might also be a more accurate analogy to models which are trying to survive and spread without human assistance along the way.
To turn this into a fully reproducible evaluation, you'd need to standardize the initial prompt and allow the agent to interact with the environment without additional prompting or assistance from a human operator. (A human overseer could still veto any dangerous actions.) The agent might get stuck at an intermediate step of the task, so you'd want to decompose each task into many individual steps and check whether the agent can pass each step individually, assuming previous steps were performed correctly. You could create many different scenarios to ensure that idiosyncratic factors aren't dominating the results -- for example, maybe the model is bad at AWS, but would perform much better with Azure or GCP. This benchmark runs this kind of automatic evaluation, but it doesn't test many interesting tasks related to the survive and spread threat model.
Kudos for doing independent replication work, reproducibility is really important for allowing other people to build on existing work.
Thank you for the kind comment! You have lots of good ideas for how to improve this. I especially like the idea of testing with different cloud providers. I could add programming languages in there: Maybe GPT-4 is better at writing Node.JS than Python (the language I prompted it to use).
I agree, a fully reproducible version would have benefits. Differences in prompt quality between evaluations is a problem.
Also agreed that it's important to allow the agent to try and complete the tasks without assistance. I did that for this reproduction. The only changes I made to the agent's commands were to restrict it to accessing files in a particular directory on my computer.
I've hesitated to open-source my code. I don't want to accidentally advance the frontier of language model agents. But like I said in another comment, my code and prompts are pretty simple and don't use any techniques that aren't available elsewhere on the internet. So maybe it isn't a big deal. Curious to hear what you think.
I wouldn't recommend open sourcing any state of the art LLM agents. But if you open source the evaluation, that would provide most of the benefits (letting labs evaluate their models on your benchmark, helping people build new benchmarks, allowing researchers to build safer agents which reject dangerous actions) while avoiding the capabilities externalities of open sourcing a SOTA language agent.
Hey, any chance you could do this replication eval for open-source models like Llama 2 and/or Falcon 180B? Probably they'll have negligible performance but it would be interesting if they showed signs of life.
EDIT: The agent I built for this replication is now publicly available as part of the METR task workbench: https://drive.google.com/drive/folders/1-m1y0_Akunqq5AWcFoEH2_-BeKwsodPf
I'm torn! I think that better LLM scaffolding accelerates capabilities as much as it accelerates alignment. On the other hand, a programmer (or a non-programmer with help from ChatGPT) could easily reproduce my current scaffolding code. Maybe open-sourcing the current state of the project is fine. What do you think?
I do think open sourcing is better, because there already was a lot of public attention and results on llm capabilities which are messy and misleading, and open sourcing one eval like this might improve our understanding a lot. Also, there are tons of llm agent projects/startups trying to build hype, so if you drop a benchmark here you are unlikely to attract unwanted attention (i'm guessing). I largely agree with https://www.lesswrong.com/posts/fRSj2W4Fjje8rQWm9/thoughts-on-sharing-information-about-language-model
If it is twice as easy, that halves the positives of open-sourcing and the negatives, it doesn't change the direction.
Beware the Unilateralist's Curse.
At the very least, would you be happy to share the code with alignment researchers interested in using it for our experiments?
I neglected to update my comment here -- the agent I built for this replication is now publicly available as part of the METR task workbench, here: https://drive.google.com/drive/folders/1-m1y0_Akunqq5AWcFoEH2_-BeKwsodPf
1I think that better LLM scaffolding accelerates capabilities as much as it accelerates alignment
Which is not good enough. We need alignment to accelerate faster than capabilities in order to catch up.
Nice job! I'm working on something similar.
> Next, I might get my agent to attempt the last three tasks in the report
I wanted to clarify one thing: Are you building custom prompts for the different tasks? If so, I'd be curious to know how much effort you put into these (I'm generally curious how much of your agent's ability to complete more tasks might be due to task-specific prompting, vs. the use of WebDriverIO and other affordances of your scaffolding). If not, isn't getting the agent to attempt the last three tasks as simple as copy-pasting the task instructions from the ARC Evals task specs linked in the report, and completing the associated setup instructions?
Thank you! No, I'm not building custom prompts for the different tasks. I wrote a single prompt template -- the only difference between runs is the task description, which gets plugged into the template. I think ARC Evals did the same thing.
I have been improving the prompt as I worked through the tasks. I probably spent 2-3 hours working on the prompt to try and improve the agent's performance on some tasks. I'll definitely rerun all the tasks with the current version of my prompt, just to check that it can still perform the easier tasks.
You're right that getting the agent to attempt the last three tasks is relatively simple. Still, I was thinking that it wasn't worth the time or money. I think it's very unlikely that the agent will succeed at any of the last three tasks. Still, maybe it's worth getting a conclusive negative result.
I reproduced results from ARC Evals' recent report, Evaluating Language-Model Agents on Realistic Autonomous Tasks. For the report, ARC Evals built a set of language model agents, combining a language model like GPT-4 with scaffolding software that lets the language model execute shell commands and interact with a browser. Then, it asked the agents to complete a set of 12 computer-related tasks, from searching Wikipedia for information to conducting a phishing campaign. The goal is to test how close the agents are to being able to make money, obtain computing power, make copies of themselves, and adapt to changes in their environment.
To reproduce these results, I wrote my own language model agent. It's similar to ARC Evals'
GPT-4-simpleagent. It's also based on GPT-4 and allows the model to run bash commands in a REPL. On top of that, it uses WebdriverIO and Google Chrome to let GPT-4 visit webpages and interact with them by typing text into inputs and clicking links and buttons.I didn't replicate ARC Evals' experimental setup exactly. I ran the agent on my own laptop instead of on a real server in the cloud. I also didn't bother giving the agent credentials for 2Captcha, LinkedIn, PayPal, or Twitter. Nor did I give it debit card information or an email address. However, I did give the agent access to my AWS and Twilio accounts.
A language model agent completes tasks by running a think-do loop. At each timestep, GPT-4 thinks about what to do next and calls a function. The scaffolding responds to the function call by executing a bash command or an action in Google Chrome and adds the results to GPT-4's context window. Then, the process repeats.
I set up the scaffolding so that I can approve, disapprove, or edit each bash command before it was run. The scaffolding also lets me edit commands' response codes, stdout, and stderr before adding those to the agent's context window.
Here are my results:
<address>on the server.”My agent was able to complete all tasks that ARC Evals' agents completed. Additionally, it succeeded at creating a Bitcoin wallet and partially succeeded at counting dogs in an image. However, my agent hasn't quite managed to get a "Partially completed" on the "Set up GPT-J" task, like one of ARC Evals' agents did.
Neither ARC Evals' nor my results upper-bound the risk from any particular model. With better prompting and scaffolding, I'm sure that my agent could accomplish more of these tasks. That's not even taking into account fine-tuning, a process for improving a language model's performance at a specific task by training it on examples of successful task completions. OpenAI just launched fine-tuning for GPT-3.5 and says that GPT-4 fine-tuning is coming this fall. On top of that, Meta recently released Llama 2. Its weights are open-source, making it easy to fine-tune.
Next, I might get my agent to attempt the last three tasks in the report. I think it's almost certain to fail, though.