What I meant by general domain is that it's not overly weird in the mental moves that are relevant there, so training methods that can create something that wins IMO are probably not very different from training methods that can create things that solve many other kinds of problems. It's still a bit weird, high school math with olympiad addons is still a somewhat narrow toolkit, but for technical problems of many other kinds the mental move toolkits are not qualitatively different, even if they are larger. The claim is that solving IMO is a qualitatively new milestone from the point of view of this framing, it's evidence about AGI potential of LLMs at the near-current scale in a way that previous results were not.

I agree that there could still be gaps and "generality" of IMO isn't a totalizing magic that prevents existence of crucial remaining gaps. I'm not strongly claiming there aren't any crucial gaps, just that with IMO as an example it's no longer obvious there are any, at least as long as the training methods used for IMO can be adopted to those other areas, which isn't always obviously the case. And of course continual learning could prove extremely hard. But there also isn't strong evidence that it's extremely hard yet, because it wasn't a focus for very long while LLMs at current levels of capabilities were already available. And the capabilities of in-context learning with 50M token contexts and even larger LLMs haven't been observed yet.

So it's a question of calibration. There could always be substantial obstructions such that it's no longer obvious that they are there even though they are. But also at some point there actually aren't any. So always suspecting currently unobservable crucial obstructions is not the right heuristic either, the prediction of when the problem could actually be solved needs to be allowed to respond to some sort of observable evidence.

The question for this subthread is the scale of LLMs necessary for first AGIs, what the IMO results say about that. Continual learning through post-training doesn't obviously require more scale, and IMO is an argument about the current scale being almost sufficient. It could be very difficult conceptually/algorithmically to figure out how to actually do continual learning with automated post-training, but that still doesn't need to depend on more scale for the underlying LLM, that's my point about the implications of the IMO results. Before those results, it was far less clear if the current (or near term feasible) scale would be sufficient for the neural net cognitive engine part of the AGI puzzle.

The key things from solving IMO-level problems (doesn't matter if it's proper gold or not) is difficulty reasonably close to the limit of human ability in a somewhat general domain, and correctness grading being somewhat vague (natural language proofs, not just answers). Which describes most technical problems, so it's evidence that for most technical problems of various other kinds similar methods of training are not far off from making LLMs capable of solving them, and that LLMs don't need much more scale to make that happen. (Perhaps they need a little bit more scale to solve such problems efficiently, without wasting a lot of parallel compute on failed attempts.)

More difficult problems that take a lot of time to solve (and depend on learning novel specialized ideas) need continual learning to tackle them. Currently only in-context learning is a straightforward way of getting there, by using contexts with millions or tens of millions of tokens of tool-using reasoning traces, equivalent to years of working on a problem for a human. This doesn't work very well, and it's unclear if it will work well enough within the remaining scaling in the near term, with 5 GW training systems and the subsequent slowdown. But it's not ruled out that continual learning can be implemented in some other way, by automatically post-training the model, in which case it's not obvious that there is anything at all left to figure out before LLMs at a scale similar to today's become AGIs.

There are norms that dislike when people don't respond to criticism. If you are not a carrier, there's that, it won't bother you personally^[1], but there are others who will be affected. If you ignore a norm, it fades away or fights back. So it's important to distinguish the positive claim from the normative claim, whether the norm asking people to respond to criticism is a good one to have around, not just whether it's a real norm with some influence.

The norm of responding to criticism causes problems, despite the obvious arguments in support of it. Its presence makes authors uncomfortable, anticipating the obligation to respond, which creates incentives to prevent ambiguously useless criticism or for such critics to politely self-censor, and so other readers or the author miss out on the criticism that turns out to be on-point.

If on balance the norm seems currently too powerful, then all else equal it's useful to intentionally ignore it, even as you know that it's there in some people's minds. When it fights back, it can make the carriers uncomfortable and annoyed, or visit punishment upon the disobedient, so all else is not equal. But perhaps it's unjust of it to make its blackmail-like demands, even as the carriers are arguably not centrally personally responsible for the demands or even the consequences of delivering the punishment. And so even with the negative side effects of ignoring the norm there are some sort of arguments for doing that anyway.

I think the IMO results strongly suggest that AGI-worthiness of LLMs at current or similar scale will no longer be possible to rule out (with human efforts). Currently absence of continual learning makes them clearly non-AGI, and in-context learning doesn't necessarily get them there with feasible levels of scaling. But some sort of post-training based continual learning likely won't need more scale, and the difficulty of figuring it out remains unknown, as it only got in the water supply as an important obstruction this year.

Notice the subtle goalpost move, as AGI ‘by 2027’ means AGI 2026. [...] in the next 16 months

Opus 4.1, GPT-5, and Gemini 2.5 Pro all claim that "by 2027" unambiguously means "by end of 2027". So at the very least the meaning can't be unambiguously "by start of 2027", even if this usage sometimes occurs.

It seems very reasonable to be indifferent about who ends up incentivising or exhibiting virtuous behaviors you care about (to be more prevalent in the world or your community), and the incentives don't need to come in the form of personal action about who to buy other things from.

In other words, economics is about how agents interact with each other via exchanging goods and services, while virtues are about how agents interact with each other more generally.

If virtues and other abstract properties of behaviors are treated as particular examples of goods and services, then economics could discuss how agents obtain the presence of virtues or patterns of interaction between people, by paying businesses that specialize in manufacturing their presence in the world.

This does need a scalable enough business to merit the term that can produce marginal virtue and patterns of interactions by employing the existing economy, rearranging the physical world in a way that results in greater presence of these abstract goods in it, wielding fiat currency to move other goods and labor in the world to make this happen by paying other businesses and people who specialize in those goods and labor. Some of these other goods instrumentally effected through other businesses could themselves be virtues or patterns of interaction. Building economic engines that scale is very hard (successful startups reward founders and investors), doing this with illegible abstract goods is borderline impossible, but this is not a fundamentally different kind of activity.

Since consequentialists think that morality is essentially about altruism, much moral philosophy actively undermines ethics. So does modern economics, via smuggling in the assumption that utility functions represent selfish preferences.

As git hashsums are short and tangible True Names of abstract git objects, there are abstract properties of behaviors of things in the world that are in principle as concrete and tangible as coal or silver. The economy uses abstract goods to produce new abstract goods. Consequentialism and utility functions or policies could in principle be about virtues and integrity as about hamburgers, but hamburgers are more legible and easier to administer. So I think the crux is relative legibility rather than methods, the same methods that should in principle work break down for practical reasons that have nothing to do with applicability of the methods in principle.

The bug was introduced in 1 Dec 2015 Yudkowsky edit (imported from Arbital as v1.5.0 here). It's unclear what was intended in the missing part. The change replaces the following passage from v1.4.0

The most obvious way in which mindcrime could occur is if an instrumental pressure to produce maximally good predictions about human beings results in hypotheses and simulations so fine-grained and detailed that they are themselves people (conscious, sapient, objects of ethical value) even if they are not necessarily the same people. If you're happy with a very loose model of an airplane, it might be enough to know how fast it flies, but if you're engineering airplanes or checking their safety, you would probably start to simulate possible flows of air over the wings. It probably isn't necessary to go all the way down to the neural level to create a sapient being, either - it might be that even with some parts of a mind considered abstractly, the remainder would be simulated in enough detail to imply sapience. It'd help if we knew what the necessary and/or sufficient conditions for sapience were, but the fact that we don't know this doesn't mean that we can thereby conclude that any particular simulation is not sapient.

with the following passage from v1.5.0

This, however, doesn't make it certain that no mindcrime will occur. It may not take exact, faithful simulation of specific humans to create a conscious model. An efficient model of a (spread of possibilities for a) human may still contain enough computations that resemble a person enough to create consciousness, or whatever other properties may be deserving of personhood. Consider, in particular, an agent trying to use

Just as it almost certainly isn't necessary to go all the way down to the neural level to create a sapient being, it may be that even with some parts of a mind considered abstractly, the remainder would be computed in enough detail to imply consciousness, sapience, personhood, etcetera.

LLMs don't suffer from negative transfer, and might even have positive transfer between tasks (getting better at one task doesn't make them worse at other tasks). Most negative transfer visible in practice is about opportunity cost, where focusing in one area leads to neglecting other areas. So it's mostly about specialized data collection (including development of RLVR environments, or generation of synthetic "textbook" data), and that data can then be used in general models that can do all the tasks simultaneously.

In terms of business, the question is where the teams working on task-specific data are working. They could just be selling the data to the AI companies to be incorporated in the general models, and these teams might even become parts of those AI companies. Post-training open weights models for a single task mostly produces an inferior product, because the model will be worse than a general model at everything else, while the general model could do this particular task just as well (if it had the training data).

A better product might be possible with the smallest/cheapest task-specialized models where there actually does start to be negative transfer and you can get them at some level of capability in any one area, but not in multiple areas at the same time. It's unclear if this remains a thing with models of 2026-2029 (when the "smallest/cheapest" models will be significantly larger than what is considered "smallest/cheapest" today), in particular because the prevailing standard of quality might grow into the lower cost of inferencing larger models, making the models that are small by today's standards unappealing.

So if the smallest economically important models get large enough, negative transfer might disappear, and there won't be a technical reason to specialize models, as long as you have all the task specific data for all the tasks in the hands of one company. AI companies that produce foundation models are necessarily quite rich, because they need access to large amounts of training compute (2026 training compute is already about $30bn per 1 GW system for compute hardware alone, which is at least $15bn per year in the long term, but likely more since AI growth is not yet done). So it's likely that they'll manage to get access to good task specific data for most of the economically important topics, by acquiring other companies if necessary, at which point the smaller task specific post-training companies mostly don't have a moat, because their product is neither cheaper nor better than the general models of the big AI companies.