gjm

Hyperbolic model fits METR capabilities estimate worse than exponential model

This is a response to https://www.lesswrong.com/posts/mXa66dPR8hmHgndP5/hyperbolic-trend-with-upcoming-singularity-fits-metr which claims that a hyperbolic model, complete with an actual singularity in the near future, is a better fit for the METR time-horizon data than a simple exponential model. I think that post has a serious error in it and its conclusions are the reverse of correct. Hence this one. (An important remark: although I think Valentin2026 made an important mistake that invalidates his conclusions, I think he did an excellent thing in (1) considering an alternative model, (2) testing it, (3) showing all his working, and (4) writing it up clearly enough that others could check his work. Please do not take any part of this post as saying that Valentin2026 is bad or stupid or any nonsense like that. Anyone can make a mistake; I have made plenty of equally bad ones myself.) [EDITED to add:] Valentin2026 has now (5) agreed that the thing I think was an error was an error, and (6) edited his post to say so. Thanks, Valentin! The models Valentin2026's post compares the results of fitting two different models to METR's time-horizon benchmark data (METR's original blogpost is at https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ but the data have been updated since then, most recently as of current writing to account for the GPT-5 launch). One model, or more accurately one family of models, is hyperbolic: if we let H be the time horizon estimate produced by METR, the model is H=A/(t1−t)q where t1 is the time at which the estimate shoots off to infinity ("the singularity", though note that divergence here does not necessarily imply e.g. superintelligence in any very strong sense), A is an overall scale factor, and q is larger when the divergence happens more abruptly. (If you start with a Moore's-law sort of model of technological progress dHdt=kH and then suppose that actually progress gets faster in proportion to H, dHdt=kH2,

203Aug 19, 2025

"AI achieves silver-medal standard solving International Mathematical Olympiad problems"

133Jul 25, 2024

Large language models learn to represent the world

101Jan 22, 2023

On "aiming for convergence on truth"

68Apr 11, 2023

gjm

Message

Hi. I'm Gareth McCaughan. I've been a consistent reader and occasional commenter since the Overcoming Bias days. My LW username is "gjm" (not "Gjm" despite the wiki software's preference for that capitalization). Elsewehere I generally go by one of "g", "gjm", or "gjm11". The URL...

31819

7389

17y

Hyperbolic model fits METR capabilities estimate worse than exponential model

Aug 19, 2025203

"AI achieves silver-medal standard solving International Mathematical Olympiad problems"

Google DeepMind reports on a system for solving mathematical problems that allegedly is able to give complete solutions to four of the six problems on the 2024 IMO, putting it near the top of the silver-medal category. Well, actually, two systems for solving mathematical problems: AlphaProof, which is more general-purpose,...

Jul 25, 2024133

Humans, chimpanzees and other animals

[Epistemic status: underinformed musings; I am posting this not because I am sure of anything in it but because the point seems important and I don't recall seeing anyone else make it. Maybe that's because it's wrong.] A common analogy for the relationship between postulated superhumanly-smart AIs and humans is...

May 30, 202321

On "aiming for convergence on truth"

Background Duncan Sabien wrote a list of proposed "basics of rational discourse" guidelines. Zack M Davis disagrees with (his interpretation of) one of the guidelines. I think the question is interesting and don't feel that those two posts and their comments resolve it. (Spoiler: I largely agree with Duncan on...

Apr 11, 202368

Large language models learn to represent the world

There's a nice recent paper whose authors did the following: 1. train a small GPT model on lists of moves from Othello games; 2. verify that it seems to have learned (in some sense) to play Othello, at least to the extent of almost always making legal moves; 3. use...

Jan 22, 2023101

Suspiciously balanced evidence

What probability do you assign to the following propositions? * "Human activity has caused substantial increases in global mean surface temperature over the last 50 years and barring major policy changes will continue to do so over at least the next 50 years -- say, at least half a kelvin...

Feb 12, 202051

"Future of Go" summit with AlphaGo

Apr 10, 20178

Load More (7/24)

LESSWRONG
LW

LESSWRONG
LW

gjm

gjm

gjm

Hyperbolic model fits METR capabilities estimate worse than exponential model

"AI achieves silver-medal standard solving International Mathematical Olympiad problems"

Large language models learn to represent the world

On "aiming for convergence on truth"

gjm

Hyperbolic model fits METR capabilities estimate worse than exponential model

"AI achieves silver-medal standard solving International Mathematical Olympiad problems"

Humans, chimpanzees and other animals

On "aiming for convergence on truth"

Large language models learn to represent the world

Suspiciously balanced evidence

"Future of Go" summit with AlphaGo

Hyperbolic model fits METR capabilities estimate worse than exponential model

"AI achieves silver-medal standard solving International Mathematical Olympiad problems"

Large language models learn to represent the world

On "aiming for convergence on truth"

Hyperbolic model fits METR capabilities estimate worse than exponential model

"AI achieves silver-medal standard solving International Mathematical Olympiad problems"

Humans, chimpanzees and other animals

On "aiming for convergence on truth"

Large language models learn to represent the world

Suspiciously balanced evidence

"Future of Go" summit with AlphaGo