This paragraph was also quoted approvingly by Altman.
I think there's some huge issues with using perplexity here:
And regardless it's crucial to limit AI R&D rather than just training runs or model capabilities. Targeting capabilities of models you train like this is a bad/incomplete way to do pacing since you’ll keep accumulating algorithmic progress (and e.g. when China steals it all then you’ll probably suddenly end your pacing).
And we don't know a way to get strong evidence that it's safe to resume (conditional on that being true), and we're unlikely to find a way soon.
There are way better ways to do pacing; my favorite is compute caps and second-favorite is deployment lags. Restricting creation of too-capable or too-risky models is a fundamentally bad way to do pacing and perplexity is a bad way to do capabilities-based pacing.
I think it's harder to Goodhart perplexity of a pretrain than it is to Goodhart most other plausible metrics, though I agree you could do so pretty trivially if you're sabotaging perplexity on a given dataset intentionally. But I strongly agree that any limitation that is only about pretraining is fundamentally insufficient. (And the correct thing to start with, given that we're uncertain and the risk is high, is a full stop of all training until we have agreed-upon rules.)
But yes, it seems that OpenAI-oriented twitter accounts rallied around this very quickly, and strongly, and are defending it at length.
Sure but come on, labs have some of the most competent ML practitioners. It'd take them two seconds to figure out what metric to convert to so it's comparable.
I imagine they'd just convert to bits per byte and agree on context length. I don't see the need to specify it in a statement also meant for non-technical people (since all solutions are basically equivalent, it's like using cm or inches)
So did they do the same kind of adjustment within the chart which they didn't clarify for tokenizer changes as well?
David Schneider-Joseph noted on twitter the the OpenAI tokenizer changed between 4 and 4o, so they presumably made an adjustment. He suggested that this might mean hey are actually reporting something other than per-token perplexity, despite the fact that the post says "Perplexity measures how well a model predicts the next token on average; lower perplexity means better predictions. The plot opening this blog post shows how this metric evolved for our models starting with GPT‑3.5."
But if they did properly adjust perplexity for the new tokenizer, I'd assume they should have mentioned that fact - and if they didn't, they should clarify.
Jakub Pachocki just posted a new proposal for how to set limits on training future AI systems:
Unfortunately, there's a huge gap in per-token perplexity between OpenAI and Anthropic, because Anthropic uses many more tokens per word. That leads to lower perplexity for Anthropic models, and any naive comparison of per-token perplexity would make Anthropic's models look significantly closer to any fixed bar for perplexity than OpenAI. That said, they say "common limits, based on perplexity and other measurements" - which doesn't preclude doing this adjustment. And Yo Shavit, Boaz Barak, and Joshua Achiam were all quick to respond that they weren't making a specific proposal and that it was not hard to adjust.
But good faith efforts to reach a common rule don't force the other people to try to explain why the rule needs to be adjusted in order to be fair.