nostalgebraist

chinchilla's wild implications

(Colab notebook here.) This post is about language model scaling laws, specifically the laws derived in the DeepMind paper that introduced Chinchilla.[1] The paper came out a few months ago, and has been discussed a lot, but some of its implications deserve more explicit notice in my opinion. In particular: * Data, not size, is the currently active constraint on language modeling performance. Current returns to additional data are immense, and current returns to additional model size are miniscule; indeed, most recent landmark models are wastefully big. * If we can leverage enough data, there is no reason to train ~500B param models, much less 1T or larger models. * If we have to train models at these large sizes, it will mean we have encountered a barrier to exploitation of data scaling, which would be a great loss relative to what would otherwise be possible. * The literature is extremely unclear on how much text data is actually available for training. We may be "running out" of general-domain data, but the literature is too vague to know one way or the other. * The entire available quantity of data in highly specialized domains like code is woefully tiny, compared to the gains that would be possible if much more such data were available. Some things to note at the outset: * This post assumes you have some familiarity with LM scaling laws. * As in the paper[2], I'll assume here that models never see repeated data in training. * This simplifies things: we don't need to draw a distinction between data size and step count, or between train loss and test loss. * I focus on the parametric scaling law from the paper's "Approach 3," because it's provides useful intuition. * Keep in mind, though, that Approach 3 yielded somewhat different results from Approaches 1 and 2 (which agreed with one another, and were used to determine Chinchilla's model and data size). * So you should take the exact numbers below with a grain of salt. They may be o

425Jul 31, 2022

nostalgebraist

Message

I write original fiction.

Also I have opinions about AI and stuff, sometimes.

Elsewhere:

Same person as nostalgebraist2point0, but now I have my account back.

I have signed no contracts or agreements whose existence I cannot mention.

8312

1355

274

13y

the void

A long essay about LLMs, the nature and history of the the HHH assistant persona, and the implications for alignment. Multiple people have asked me whether I could post this LW in some form, hence this linkpost. ~17,000 words. Originally written on June 7, 2025. (Note: although I expect this...

Jun 11, 2025405

a confusion about preference orderings

Here's a confusion I have about preference orderings in decision theory. Caveat: the observations I make below feel weirdly trivial to me, to the point that I feel wary of making a post about them at all; the specter of readers rolling their eyes and thinking "oh he's just talking...

May 11, 202593

when will LLMs become human-level bloggers?

"Short AI timelines" have recently become mainstream. One now routinely hears the claim that somewhere in the 2026-2028 interval, we'll have AI systems that outperform humans in basically every respect. For example, the official line from Anthropic holds that "powerful AI" will likely arrive in late 2026 or in 2027....

Mar 9, 2025125

the case for CoT unfaithfulness is overstated

[Quickly written, unpolished. Also, it's possible that there's some more convincing work on this topic that I'm unaware of – if so, let me know. Also also, it's possible I'm arguing with an imaginary position here and everyone already agrees with everything below.] In research discussions about LLMs, I often...

Sep 29, 2024269

instruction tuning and autoregressive distribution shift

[Note: this began life as a "Quick Takes" comment, but it got pretty long, so I figured I might as well convert it to a regular post.] In LM training, every token provides new information about "the world beyond the LM" that can be used/"learned" in-context to better predict future...

Sep 5, 202440

Am I confused about the "malign universal prior" argument?

In a 2016 blog post, Paul Christiano argued that the universal prior (hereafter "UP") may be "malign." His argument has received a lot of follow-up discussion, e.g. in * Mark Xu's The Solomonoff Prior is Malign * Charlie Steiner's The Solomonoff prior is malign. It's not a big deal. among...

Aug 27, 202497

nostalgebraist's Shortform

May 27, 20247

Load More (7/27)

LESSWRONG
LW

LESSWRONG
LW

nostalgebraist

nostalgebraist

nostalgebraist

chinchilla's wild implications

the void

the case for CoT unfaithfulness is overstated

larger language models may disappoint you [or, an eternally unfinished draft]

nostalgebraist

the void

a confusion about preference orderings

when will LLMs become human-level bloggers?

the case for CoT unfaithfulness is overstated

instruction tuning and autoregressive distribution shift

Am I confused about the "malign universal prior" argument?

nostalgebraist's Shortform

chinchilla's wild implications

the void

the case for CoT unfaithfulness is overstated

larger language models may disappoint you [or, an eternally unfinished draft]

the void

a confusion about preference orderings

when will LLMs become human-level bloggers?

the case for CoT unfaithfulness is overstated

instruction tuning and autoregressive distribution shift

Am I confused about the "malign universal prior" argument?

nostalgebraist's Shortform