Some approaches to AI alignment incur upfront costs to the creator (an “alignment tax”). In this post, I discuss “alignment windfalls” which are strategies that tend towards the long-term public good at the same time as reaping short-term benefits for a company. My argument, in short: 1. Just as there...
In his AI Safety Needs Great Engineers post, Andy Jones explains how software engineers can reduce the risks of unfriendly artificial intelligence. Even without deep ML knowledge, these developers can work effectively on the challenges involved in building and understanding large language models. I would broaden the claim: AI safety...
We want to advance process-based supervision for language models. To make it easier for others to contribute to that goal, we're sharing code for writing compositional language model programs, and a tutorial that explains how to get started: * The Interactive Composition Explorer (ICE) is a library for writing and...
Some approaches to AI alignment incur upfront costs to the creator (an “alignment tax”). In this post, I discuss “alignment windfalls” which are strategies that tend towards the long-term public good at the same time as reaping short-term benefits for a company.
My argument, in short:
An “alignment tax” refers to the reduced... (read 2265 more words →)
In his AI Safety Needs Great Engineers post, Andy Jones explains how software engineers can reduce the risks of unfriendly artificial intelligence. Even without deep ML knowledge, these developers can work effectively on the challenges involved in building and understanding large language models.
I would broaden the claim: AI safety doesn’t need only great engineers – it needs great product builders in general.
This post will describe why, list some concrete projects for a few different roles, and show how they contribute to AI going better for everyone.
This post is aimed at anyone who has been involved with building software products: web developers, product managers, designers, founders, devops, generalist software engineers, … I’ll call these “product... (read 1666 more words →)
We want to advance process-based supervision for language models. To make it easier for others to contribute to that goal, we're sharing code for writing compositional language model programs, and a tutorial that explains how to get started:
We've been using ICE as part of our work on Elicit and have found it useful in practice.
ICE is an open-source Python library for writing, debugging, and visualizing compositional language model programs. ICE makes it easy to: