Jesse Hoogland

Neural networks generalize because of this one weird trick

> Produced under the mentorship of Evan Hubinger as part of the SERI ML Alignment Theory Scholars Program - Winter 2022 Cohort > A big thank you to all of the people who gave me feedback on this post: Edmund Lau, Dan Murfet, Alexander Gietelink Oldenziel, Lucius Bushnaq, Rob Krzyzanowski, Alexandre Variengen, Jiri Hoogland, and Russell Goyder. Statistical learning theory is lying to you: "overparametrized" models actually aren't overparametrized, and generalization is not just a question of broad basins. The standard explanation thrown around here for why neural networks generalize well is that gradient descent settles in flat basins of the loss function. On the left, in a sharp minimum, the updates bounce the model around. Performance varies considerably with new examples. On the right, in a flat minimum, the updates settle to zero. Performance is stabler under perturbations. To first order, that's because loss basins actually aren't basins but valleys, and at the base of these valleys lie "rivers" of constant, minimum loss. The higher the dimension of these minimum sets, the lower the effective dimensionality of your model.[1] Generalization is a balance between expressivity (more effective parameters) and simplicity (fewer effective parameters). Symmetries lower the effective dimensionality of your model. In this example, a line of degenerate points effectively restricts the two-dimensional loss surface to one dimension. In particular, it is the singularities of these minimum-loss sets — points at which the tangent vanishes — that determine generalization performance. The remarkable claim of singular learning theory (the subject of this post), is that "knowledge … to be discovered corresponds to singularities in general" [1]. Complex singularities make for simpler functions that generalize further. The central claim of singular learning theory is that the singularities of the set of minima of the loss function determine learning behavior and generalization.

204Jan 18, 2023

Jesse Hoogland

Message

Executive director at Timaeus. Working on singular learning theory and developmental interpretability.

Website: jessehoogland.com

Twitter: @jesse_hoogland

3160

875

SLT for AI Safety

> This sequence draws from a position paper co-written with Simon Pepin Lehalleur, Jesse Hoogland, Matthew Farrugia-Roberts, Susan Wei, Alexander Gietelink Oldenziel, Stan van Wingerden, George Wang, Zach Furman, Liam Carroll, Daniel Murfet. Thank you to Stan, Dan, and Simon for providing feedback on this post. Alignment ⊆ Capabilities. As...

Jul 1, 202576

The Sweet Lesson: AI Safety Should Scale With Compute

A corollary of Sutton's Bitter Lesson is that solutions to AI safety should scale with compute.[1] Let's consider a few examples of research directions that are aiming at this property: * Deliberative Alignment: Combine chain-of-thought with Constitutional AI to improve safety with inference-time compute (see Guan et al. 2025, Figure...

May 5, 202597

Timaeus in 2024

> TLDR: We made substantial progress in 2024: > > * We published a series of papers that verify key predictions of Singular Learning Theory (SLT) [1, 2, 3, 4, 5, 6]. > * We scaled key SLT-derived techniques to models with billions of parameters, eliminating our main concerns around...

Feb 20, 2025100

The Simplest Good

Common Law AI worked better than anyone expected. Dr. Sarah Chen was skeptical from the start. "You're essentially training them to be moral judges," she warned during the initial architecture review. "What if they overfit on ethics?" The room laughed. "Better than the alternative," someone quipped. The idea was simple...

Feb 2, 202576

Kessler's Second Syndrome

It started as so many dooms do, with a flash in the night sky over the South China Sea. Testing a new ASAT weapon, the Chinese military shattered a derelict spy satellite into 40,000 shards of shrapnel. The debris pattern suggested a fragmentation warhead optimized for lethal scatter. Within 48...

Jan 26, 202570

Brainrot

January: In early 2026, Meta launches a fleet of new AI influencers, targeting the massive audience displaced by the Xiaohongshu-TikTok wars. They are beautiful, funny, smart—whatever you want them to be. Equipped with the latest in online learning, the agents immediately begin adapting to social media trends as they occur....

Jan 26, 202543

The Rising Sea

And then we hit a wall. Nobody expected it. Well... almost nobody. Yann LeCun posted his "I told you so's" all over X. Gary Marcus insisted he'd predicted this all along. Sam Altman pivoted, declaring o3 was actually already ASI. The first rumors of scaling laws breaking down were already...

Jan 25, 202597

Load More (7/30)

LESSWRONG
LW

LESSWRONG
LW

Jesse Hoogland

Jesse Hoogland

Jesse Hoogland

Neural networks generalize because of this one weird trick

Towards Developmental Interpretability

Announcing Timaeus

Timaeus's First Four Months

Jesse Hoogland

SLT for AI Safety

The Sweet Lesson: AI Safety Should Scale With Compute

Timaeus in 2024

The Simplest Good

Kessler's Second Syndrome

Brainrot

The Rising Sea

Neural networks generalize because of this one weird trick

Towards Developmental Interpretability

Announcing Timaeus

Timaeus's First Four Months

SLT for AI Safety

The Sweet Lesson: AI Safety Should Scale With Compute

Timaeus in 2024

The Simplest Good

Kessler's Second Syndrome

Brainrot

The Rising Sea