"CS-ReFT finetunes LLMs at the subspace level, enabling the much smaller Llama-2-7B to surpass GPT-3.5's performance using only 0.0098% of model parameters."
https://x.com/IntologyAI/status/1901697581488738322
The Moravec paradox is an observation that high-level reasoning is relatively simple for computers, while sensorimotor skills that humans find effortless are computationally challenging. This is why AI is superhuman at chess but we have no self driving cars. Evolutionarily recent developments such as critical thinking is easier for computers than older ones, because these recent developments are less efficient in humans, and they are computationally simpler anyway (much more straightforward to pick one good decision among many rather than to move a limb through space).
This is what fast takeoff looks like. The paper is very math heavy, and the solution is very intelligent. It is still very short. Why would it not be? If you can make something simple that works, that's all it will take.
It reminds me of two papers, one written by Anthropic about interpretability in neural networks, where they used self supervised learning to discover the abstract representations in latent space, a technique that they later used to launch "Golden Gate Claude" wherein they made the language model was forced to include the Golden Gate Bridge in all of its inputs, and another paper called "The Platonic Representation Hypothesis" where the main hypothesis is that neural networks trained with different objectives on different data and modalities (like vision and language) are converging toward a shared statistical model of reality in their representation spaces, analogous to Plato's concept of an ideal reality that underlies our sensations.
Anthropic interpretability research: https://claude.ai/chat/4400cbcc-66d6-4794-8c10-ad09efafa74c
The platonic representation hypothesis: https://arxiv.org/abs/2405.07987
The paper itself, as written by the AI researcher over the course of several days:
https://arxiv.org/abs/2503.10617
Thanks!
So, the claim here is that this is a better "artificial AI scientist" compared to what we've seen so far.
There is a tech report https://github.com/IntologyAI/Zochi/blob/main/Zochi_Technical_Report.pdf, but the "AI scientist" itself is not open source, and the tech report does not disclose much (besides confirming that this is a multi-agent thing).
This might end up being a new milestone (but it's too early to conclude that; the comparison is not quite "apple-to-apple", there is human feedback in the process of its work, and humans make edits to the final paper, unlike Sakana, so it's too early to conclude that this one is substantially better).