NickyP

LLM Modularity: The Separability of Capabilities in Large Language Models

Separating out different capabilities. Post format: First, a 30-second TL;DR, next a 5-minute summary, and finally the full ~40-minute full length technical report. Special thanks to Lucius Bushnaq for inspiring this work with his work on modularity. TL;DR One important aspect of Modularity, is that there are different components of the neural network that are preforming distinct, separate tasks. I call this the “separability” of capabilities in a neural network, and attempt to gain empirical insight into current models. The main task I chose, was to attempt to prune a Large Language Model (LLM) such that it retains all abilities, except the ability to code (and vice versa). I have had some success in separating out the different capabilities of the LLMs (up to approx 65-75% separability), and have some evidence to suggest that larger LLMs might be somewhat separable in capabilities with only basic pruning methods. My current understanding from this work, is that attention heads are more task-general, and feed-forward layers are more task-specific. There is, however, still room for better separability techniques and/or to train LLMs to be more separable in the first place. My future focus, is to try to understand how anything along the lines of "goal" formation occurs in language models, and I think this research has been a step towards this understanding. 5 Minute Non-Techincal Summary A set diagram of how separable LLM Capabilities might roughly be separated I am currently interested in understanding the "modularity" of Large Language Models (LLMs). Modularity is an important concept for designing systems with interchangeable parts, which could lead to us being better able to do goal alignment for these models. In this research, I studied the idea of "Separability" in LLMs, which looks at how different parts of a system handle specific tasks. To do this, I created a method that involved finding model parts responsible for certain tasks, removing thes

101Mar 26, 2023

LLM Basics: Embedding Spaces - Transformer Token Vectors Are Not Points in Space

84Feb 13, 2023

AISC 2024 - Project Summaries

48Nov 27, 2023

Searching for Modularity in Large Language Models

44Sep 8, 2022

NickyP

Message

Nicky Pochinkov

https://nicky.pro

686

101

Modelling Trajectories - Interim results

Introduction Note: These are results which have been in drafts for a year, see discussion about how we have moved on to thinking about these things. Our team at AI Safety Camp has been working on a project to model the trajectories of language model outputs. We're interested in predicting...

Dec 4, 202511

Energy Markets Temporal Arbitrage with Batteries

Epistemic Status: I am not an energy expert, and this was done rather briefly. All analysis uses pricing data specific to Ireland, but some general ideas are likely applicable more broadly. Data is true as of March 2025. Where there are uncertainties I try to state them, but there are...

Mar 4, 202538

Distillation of Meta's Large Concept Models Paper

Note: I had this as a draft for a while. I think it is accurate, but there may be errors. I am not in any way affiliated with the authors of the paper. Below I briefly discuss the "Large Concept Models" paper released by Meta, which tries to change some...

Mar 4, 202519

ParaScopes: Do Language Models Plan the Upcoming Paragraph?

This work is a continuation of work in a workshop paper: Extracting Paragraphs from LLM Token Activations, and based on continuous research into my main research agenda: Modelling Trajectories of Language Models. See the GitHub repository for code additional details. Looking at the path directly in front of the LLM...

Feb 21, 202541

Literature Review of Text AutoEncoders

This is a brief literature review of Text AutoEncoders, as I used them in a recent project and did not find a good resource covering them. TL;DR: There exist models that take some text -> encode it into a single vector -> decode back into approximately the same text. Meta's...

Feb 19, 202521

Confusing the metric for the meaning: Perhaps correlated attributes are "natural"

Epistemic status: possibly trivial, but I hadn't heard it before. TL;DR: What I thought of as a "flaw" in PCA—its inability to isolate pure metrics—might actually be a feature that aligns with our cognitive processes. We often think in terms of composite concepts (e.g., "Age + correlated attributes") rather than...

Jul 23, 202433

Comparing Quantized Performance in Llama Models

Epistemic Status: Quick tests, most of this was done in less than 48 hours TL;DR: Can you skimp on GPU VRAM? 8bit quantized seems fine, for 4bit it depends. I was asked by @Teun van der Weij, to what degree one can run evaluations on quantized models, and I was...

Jul 15, 202435

Load More (7/19)

LESSWRONG
LW

LESSWRONG
LW

NickyP

NickyP

NickyP

LLM Modularity: The Separability of Capabilities in Large Language Models

LLM Basics: Embedding Spaces - Transformer Token Vectors Are Not Points in Space

AISC 2024 - Project Summaries

Searching for Modularity in Large Language Models

NickyP

Modelling Trajectories - Interim results

Energy Markets Temporal Arbitrage with Batteries

Distillation of Meta's Large Concept Models Paper

ParaScopes: Do Language Models Plan the Upcoming Paragraph?

Literature Review of Text AutoEncoders

Confusing the metric for the meaning: Perhaps correlated attributes are "natural"

Comparing Quantized Performance in Llama Models

LLM Modularity: The Separability of Capabilities in Large Language Models

LLM Basics: Embedding Spaces - Transformer Token Vectors Are Not Points in Space

AISC 2024 - Project Summaries

Searching for Modularity in Large Language Models

Modelling Trajectories - Interim results

Energy Markets Temporal Arbitrage with Batteries

Distillation of Meta's Large Concept Models Paper

ParaScopes: Do Language Models Plan the Upcoming Paragraph?

Literature Review of Text AutoEncoders

Confusing the metric for the meaning: Perhaps correlated attributes are "natural"

Comparing Quantized Performance in Llama Models