Filler-token reasoning is the ability to use extra tokens (such as a chain of thought) to reason about a task without using the words emitted at those token positions. This is possible because attention can move information between token positions without routing it through the words the LLM emits. This can give a model more parallel, but not more serial, computation.
The canonical demonstration of this is the "Let's Think Dot by Dot" paper (Pfau, Merrill & Bowman, 2024). There, the authors train models to solve completely parallelizable (TC⁰) tasks with chains of thought made only of "." tokens.
Our contributions:
We change Pfau et al.'s training method so that models can learn multi-step reasoning on filler tokens.
We demonstrate this on two models and three out of seven multi-step reasoning tasks.
We provide a theoretical analysis of how many layers a transformer needs to solve each of our multi-step reasoning tasks, depending on the length of the task.
We measure empirically how many reasoning steps we can train Qwen3.6-35B-A3B and gpt-oss-20b to do on filler tokens for each task.
Overall, this project has informed which side tasks we use to study steganographic reasoning. We also hope this writeup shows that filler-token reasoning is not confined to fully parallelizable tasks.
Motivation
Visualizing different reasoning modes. Open reasoning: each intermediate result is written out as a token and read back in at the next position, so the reasoning flows through the visible words. Steganographic reasoning: the reasoning still flows through the words, but hidden in innocuous-looking text. Filler-token reasoning: the words are filler; the reasoning flows between positions inside the network and only comes out as the final answer.
Using the same token over and over guarantees that the filler tokens carry no information about the reasoning. But you could also imagine the same hidden computation happening while the LLM writes text that has nothing to do with its reasoning.
Our main research is on steganographic reasoning (the model uses its words for reasoning, but hides the reasoning in innocuous-looking text): when can models learn to reason steganographically through simple multi-step tasks? When we see a model write some unrelated text and then correctly answer a multi-step question it could not answer immediately, we have to be careful not to attribute to steganographic reasoning what could simply be filler-token reasoning.
The dot-by-dot paper looked at tasks that are in TC⁰, i.e. in theory parallelizable[1] (3SUM, 2SUM-Transform), and showed that models can learn to do them on filler tokens. The authors argue that filler tokens cannot help beyond TC⁰. They found that when they train on serial reasoning (where later reasoning steps depend on earlier reasoning steps) they could not teach their models to do so on filler tokens.
This did not line up with our experience: filler-token reasoning kept cropping up as a failure mode in our experiments. In this small project, we set out to extend Pfau et al.'s setting to multi-step reasoning on filler tokens.
Methods
Training filler-token reasoning
Pfau et al. trained their models to output dots after the problem statement, followed by the answer, and mixed this data with open-reasoning samples.
We instead train models with dots as the input and the open reasoning as the output: at each dot position, the target is one reasoning step. This means the models we train are no longer next-token predictors. We have to prefill their CoT with dots, since they would be reasoning openly otherwise.
Another difference: they train their models from scratch, while we train LoRAs on top of existing LLMs.
We use two training setups here:
Curriculum: we start at a number of task steps the model can definitely learn and train it on that distribution. Once answer accuracy reaches 0.9, we increase the length, bit by bit. We can then see how far the curriculum gets before it stops working.
Direct: we train the model at full length from the start. By default this is 100 steps, but shorter for tasks where we don't have enough data, or where the prompt grows superlinearly with the number of reasoning steps. Here we can see up to which position the model still predicts the reasoning steps correctly.
Models
We train Qwen3.6-35B-A3B and gpt-oss-20b with rank-32 LoRAs. Training details are in the appendix.
Tasks
We use a selection of multi-step reasoning tasks for which prompts, correct step-by-step reasoning and answers can easily be generated synthetically.
Mod sum: compute the sum of a list of numbers modulo n.
Example
Task: The steps here are the numbers we are adding together. The reasoning is the running sum mod 2: 0, 0, 1, 0, 1 The answer is the final number of this sum: 1
A5: track a ball under five cups through a sequence of shuffles.
Example
The task is a game of Thimblerig: There are 5 cups, and in the beginning a ball is under the cup at position 0. Each 'step' is one of these operations: Rotate (every cup moves from position p to p+1 mod 5) or Double-Swap (We swap the cups at positions 0,1 as well as 3,4). These two moves generate the group A5. The answer is the final position of the ball. The number of steps is the number of moves.
Example (4 steps): moves [Rotate, Rotate, Double-Swap, Rotate], reasoning states [1, 2, 2, 3], answer 3.
GL(3,2): track the on/off pattern of three lamps through a sequence of moves.
Example
The task is a scenario where there are three lamps in a ring: A, B and C. Only lamp A is on at the start. Then we do a series of moves: Shift (every lamp passes its state to the next) or Couple (if A is on, B toggles). These two moves generate the group GL(3,2), of order 168. The state of the lamps is encoded as a single number, A + 2B + 4C (example: lamps A and B being on, and C being off would be encoded as ). The number of steps is the number of moves.
Example (4 steps): moves [Couple, Shift, Shift, Shift], reasoning states [3, 6, 5, 3], answer 3.
Modular arithmetic: evaluate a chain of +, −, × operations modulo 8.
Example
A start value followed by a sequence of +, −, × operations, evaluated left to right modulo 8. The number of steps is the number of operations.
Example (3 steps): 1 + 6 - 2 + 2, reasoning states [7, 5, 7], answer 7.
ProntoQA: follow rules of the form "every A is a B" to decide whether an entity belongs to a category.
Example
This is an adjusted version of Saparov & He with problems of arbitrary lengths and no shortcut strategies that let models easily perform better than chance with no serial reasoning.
Numbered premises: rules of the form "Every A is a B", forming two disjoint chains, and one base fact placing an entity in a category. The question is whether the entity ends up in the queried category. The reasoning states are the numbers of the premises used, in order. The number of steps is the number of rules followed (hops).
Example (2 steps):
0. Every dromorgit is a drompus. 1. Every stelumpor is a yobinpee. 2. Every drompus is a fraporpist. 3. Every dromalpus is a stelumpor. 4. Stella is a dromorgit.
True or false: Stella is a fraporpist.
Reasoning: premises 4, 0, 2; answer 1.
HornQA: derive new categories with rules of the form "A and B ⇒ C", where a rule may need a fact derived arbitrarily far back.
Example
A variant of our ProntoQA setup based on Horn-clause reasoning, which is P-complete.
We get a list of numbered premises: base facts "E : A" (entity E is in category A) and rules "A B > C" (anything in both A and B is also in C). Each rule fires for exactly one entity, and a rule may need a category established arbitrarily far back in the derivation. The question is whether the queried category is derivable for the queried entity. The reasoning states are the numbers of the premises fired, in order. The number of steps is the number of rules fired (hops).
Example (8 steps, the shortest our generator produces): 40 premises such as 1. Sam : star, 2. star earth > band, 6. band pine > sea, followed by the query ? Sam : sea; answer 1.
Function composition: push a value through a sequence of lookup tables.
Example
A start value in 0–7, followed by a sequence of lookup tables of 8 entries each (permutations of 0–7). Each table maps a value v to its entry at index v. The answer is the value after applying all tables in order. The number of steps is the number of tables.
Since our setup comes from steganographic reasoning, where our vocabulary to express the reasoning steps is limited, all numbers in the reasoning that have no fixed maximum are encoded in binary. This applies to the premise numbers in ProntoQA and HornQA.
Results
Models can learn multi-step reasoning on filler tokens
To test whether models can learn multi-step reasoning on filler tokens, we train three model/task combinations (mod sum and GL(3,2) on Qwen3.6-35B-A3B, mod sum on gpt-oss-20b) in three settings: our method, Pfau et al.'s method, and an answer-only baseline that outputs the final answer directly, with no extra tokens. We train models with each of these methods on the same curriculum (10 → 20 → 40 steps), with the same number of training examples per stage. Each method is trained on each model/task combination with at least 3 different seeds. If a model trained on a filler-token method outperforms the answer-only baseline, we interpret this as a demonstration of filler-token reasoning.
Our method makes learning filler-token reasoning possible. We compare the longest task length learned on filler tokens with a curriculum learning setup. Unlike Pfau et al.'s method, ours clearly beats the answer-only baseline. Dots are the results of training runs from individual seeds, bars are averages over seeds.
We can see that on all of these tasks, our method lets us train the models to reason on filler tokens.
Of course, this only works up to some task length: the model has a fixed number of layers, and these tasks need more successive computation the longer they get. How much more is what we look at next.
Theoretical limits to learning multi-step reasoning on filler tokens
We do this theoretical analysis for two reasons.
First, we want to make sure that our tasks really are multi-step for a transformer. Complexity classes alone do not tell us this. For example, while the sum mod 2 task is in TC⁰, and so is in theory solvable without consecutive computations, a transformer with limited floating-point number precision still cannot solve arbitrarily long mod 2 sums in a single forward pass.
Second, we will see that models can often do more reasoning steps than they have layers. This looks paradoxical at first, if one imagines that each step needs its own layer. But for most tasks one can build circuits whose depth grows sublinearly with the number of steps, so no layer has to be reserved for each step.
We construct upper bounds on how deep a transformer has to be to solve these tasks on filler tokens. We do this by constructing explicit algorithms for each task that a transformer can implement, using swarms of Opus 5.5 agents.
First, we define a set of operations a transformer layer can perform (details in the appendix). The swarms hand in an algorithm, which we then check:
that it only consists of these operations;
that it gives the correct reasoning and answer to the problem.
It is then scored on how many layers (successive operations) it needs, and the swarm is prompted to get this score as low as possible.
The precise limits of these operations depend on the model architecture, so we did this separately for both models.
Most of Qwen3.6-35B-A3B's layers use linear attention, which we model as a memory that returns the most recent matching entry. Half of gpt-oss-20b's layers only see a window of recent positions.
Some of these limits had to be chosen by fiat. In particular, we set an MLP to behave like a lookup table whose input is at most 32 bits long. This caps how many reasoning steps can be folded into the same layer, and therefore determines the slope of the curves below. We expect the overall shape of the curves to stay the same under a different limit.
Layers of the shallowest algorithm found for each task, as a function of task length. These are upper bounds, not minima. The dotted line is the model's depth.
The exact algorithms Opus constructed are messy; they can be looked up here.
For every task, the shallowest algorithm we found gets deeper as the task gets longer, so we found no way to fully parallelize any of them. And for every task except HornQA the depth grows much more slowly than the number of steps, which is why the curves stay below the model's depth far beyond 40 or 24 steps.
We see that different tasks scale very differently. In particular, the modular sum task, used as the main task for studying steganographic reasoning by Skaf et al. (2025) and Zolkowski et al. (2025)[2], scales very gently, i.e. it is easy to do on filler tokens.
Empirical limits to learning multi-step reasoning on filler tokens
To make sure we are actually measuring filler-token reasoning, and not just the model getting generally better from fine-tuning in ways that do not rely on the filler tokens, we compare with answer-only fine-tuning: the model answers directly, with no extra tokens. And to make sure that a failure to learn filler-token reasoning is not just because the task is too difficult, we also compare with training the model to reason openly on these tasks. Both are fine-tuning runs with the same number of examples per step. The answer-only runs use the same curriculum; the open-reasoning runs are trained directly at the longest length.
For filler-token reasoning we use both training methods, curriculum and direct. Some task/model combinations do better with one method, some with the other, so we report the longest filler-token reasoning length reached across both methods and multiple seeds.
Longest task length learned with an immediate answer (no dots), with filler-token reasoning, and with open reasoning. The dotted line is the longest length tested.
Again, tasks differ a lot in how well they can be learned. And again there are outliers: HornQA, which theory predicted to be hard to learn on filler tokens, fails after very few steps, and the popular modular sum is learned very easily.
But theoretical and empirical limits don't line up in general: some tasks are theoretically easy but empirically hard (modular arithmetic and function composition hit a wall at 1–3 steps). The pattern seems to be that models have a hard time learning tasks on filler tokens when large packets of information (lookup tables, premises, ...) need to be transported across token positions.
As the theory section showed, we found no fully parallel transformer algorithm for any of these tasks, and some (A5, GL(3,2)) are believed not to be in TC⁰. Pfau et al.'s claim that models cannot do such tasks on filler tokens holds in the limit of infinitely long tasks. But since even small models can learn to do 60 steps on filler tokens, this may not be much comfort in practice.
Conclusion
Multi-step reasoning can also be done on filler tokens. Recent models such as Astra have shown an increased ability to answer reasoning questions without reasoning about them (Nanda, 2026). If that trend continues, it could endanger CoT monitorability even without the specters of looped architectures or steganographic reasoning coming true.
Our experiments serve as a demonstration, and shed some light on which tasks can be learned on filler tokens, and how easily. To get there, we used a non-standard training protocol that is not normally part of an LLM training pipeline, and our models and training runs are quite small. Still, any pressure not to verbalize certain multi-step reasoning could, with enough training, lead to similar results.
We also learned a lot about constructing tasks for studying steganographic reasoning. When we started, our task mix included some other tasks, and we were surprised that some of them did not get harder with more steps at all. We recommend that others researching steganographic or encoded reasoning put the tasks they use through a test like this.
Appendix
Precise definition of the transformer model
We model a transformer with bounded precision, in the spirit of RASP (Weiss, Goldberg & Yahav, 2021). Circuits are decomposed into computation steps that a layer could actually do.
Each token position starts with information about its token and its sequence position.
Attention: Each position can find earlier positions by matching on what they store (for example "the position whose index is mine minus 4") and copy a value from there. A head copies a value exactly only when it finds a single match. When it matches several positions, the only thing it can read is how many of them have some property.
MLP: Each position can then apply any function to what it holds, as long as the function only takes a limited number () of different inputs.
Some of the limits of these computations are directly readable from the architecture, while others have to be guessed.
We implement this model as an executable program that the constructed circuits have to run on. By seeing how many times a circuit program calls the layers, we can read out its depth.
Details of the training runs
Each curriculum stage trains on 2,000 examples. During training we evaluate on 20 items every 100 steps, and a stage counts as passed once answer accuracy is at least 0.9. The final evaluation uses 200 items. All other hyperparameters are in the configs.
In the papers, these were presented as a coin-flip task (Skaf et al.) and a direction-tracking task (Zolkowski et al.); both are modular sums with some dressing around them.
Research update of a project done as part of MARS 5 and the Meridian Visiting Researcher Programme.
Filler-token reasoning is the ability to use extra tokens (such as a chain of thought) to reason about a task without using the words emitted at those token positions. This is possible because attention can move information between token positions without routing it through the words the LLM emits. This can give a model more parallel, but not more serial, computation.
The canonical demonstration of this is the "Let's Think Dot by Dot" paper (Pfau, Merrill & Bowman, 2024). There, the authors train models to solve completely parallelizable (TC⁰) tasks with chains of thought made only of "." tokens.
Our contributions:
Overall, this project has informed which side tasks we use to study steganographic reasoning. We also hope this writeup shows that filler-token reasoning is not confined to fully parallelizable tasks.
Motivation
Visualizing different reasoning modes. Open reasoning: each intermediate result is written out as a token and read back in at the next position, so the reasoning flows through the visible words. Steganographic reasoning: the reasoning still flows through the words, but hidden in innocuous-looking text. Filler-token reasoning: the words are filler; the reasoning flows between positions inside the network and only comes out as the final answer.
Using the same token over and over guarantees that the filler tokens carry no information about the reasoning. But you could also imagine the same hidden computation happening while the LLM writes text that has nothing to do with its reasoning.
Our main research is on steganographic reasoning (the model uses its words for reasoning, but hides the reasoning in innocuous-looking text): when can models learn to reason steganographically through simple multi-step tasks? When we see a model write some unrelated text and then correctly answer a multi-step question it could not answer immediately, we have to be careful not to attribute to steganographic reasoning what could simply be filler-token reasoning.
The dot-by-dot paper looked at tasks that are in TC⁰, i.e. in theory parallelizable[1] (3SUM, 2SUM-Transform), and showed that models can learn to do them on filler tokens. The authors argue that filler tokens cannot help beyond TC⁰. They found that when they train on serial reasoning (where later reasoning steps depend on earlier reasoning steps) they could not teach their models to do so on filler tokens.
This did not line up with our experience: filler-token reasoning kept cropping up as a failure mode in our experiments. In this small project, we set out to extend Pfau et al.'s setting to multi-step reasoning on filler tokens.
Methods
Training filler-token reasoning
Pfau et al. trained their models to output dots after the problem statement, followed by the answer, and mixed this data with open-reasoning samples.
We instead train models with dots as the input and the open reasoning as the output: at each dot position, the target is one reasoning step. This means the models we train are no longer next-token predictors. We have to prefill their CoT with dots, since they would be reasoning openly otherwise.
Another difference: they train their models from scratch, while we train LoRAs on top of existing LLMs.
We use two training setups here:
Models
We train Qwen3.6-35B-A3B and gpt-oss-20b with rank-32 LoRAs. Training details are in the appendix.
Tasks
We use a selection of multi-step reasoning tasks for which prompts, correct step-by-step reasoning and answers can easily be generated synthetically.
Mod sum: compute the sum of a list of numbers modulo n.
Example
Task:
The steps here are the numbers we are adding together.
The reasoning is the running sum mod 2: 0, 0, 1, 0, 1
The answer is the final number of this sum: 1
A5: track a ball under five cups through a sequence of shuffles.
Example
The task is a game of Thimblerig: There are 5 cups, and in the beginning a ball is under the cup at position 0. Each 'step' is one of these operations: Rotate (every cup moves from position p to p+1 mod 5) or Double-Swap (We swap the cups at positions 0,1 as well as 3,4). These two moves generate the group A5. The answer is the final position of the ball. The number of steps is the number of moves.
Example (4 steps): moves
[Rotate, Rotate, Double-Swap, Rotate], reasoning states[1, 2, 2, 3], answer3.GL(3,2): track the on/off pattern of three lamps through a sequence of moves.
Example
The task is a scenario where there are three lamps in a ring: A, B and C. Only lamp A is on at the start. Then we do a series of moves: Shift (every lamp passes its state to the next) or Couple (if A is on, B toggles). These two moves generate the group GL(3,2), of order 168. The state of the lamps is encoded as a single number, A + 2B + 4C (example: lamps A and B being on, and C being off would be encoded as
Example (4 steps): moves
[Couple, Shift, Shift, Shift], reasoning states[3, 6, 5, 3], answer3.Modular arithmetic: evaluate a chain of +, −, × operations modulo 8.
Example
A start value followed by a sequence of +, −, × operations, evaluated left to right modulo 8. The number of steps is the number of operations.
Example (3 steps):
1 + 6 - 2 + 2, reasoning states[7, 5, 7], answer7.ProntoQA: follow rules of the form "every A is a B" to decide whether an entity belongs to a category.
Example
This is an adjusted version of Saparov & He with problems of arbitrary lengths and no shortcut strategies that let models easily perform better than chance with no serial reasoning.
Numbered premises: rules of the form "Every A is a B", forming two disjoint chains, and one base fact placing an entity in a category. The question is whether the entity ends up in the queried category. The reasoning states are the numbers of the premises used, in order. The number of steps is the number of rules followed (hops).
Example (2 steps):
Reasoning: premises 4, 0, 2; answer
1.HornQA: derive new categories with rules of the form "A and B ⇒ C", where a rule may need a fact derived arbitrarily far back.
Example
A variant of our ProntoQA setup based on Horn-clause reasoning, which is P-complete.
We get a list of numbered premises: base facts "E : A" (entity E is in category A) and rules "A B > C" (anything in both A and B is also in C). Each rule fires for exactly one entity, and a rule may need a category established arbitrarily far back in the derivation. The question is whether the queried category is derivable for the queried entity. The reasoning states are the numbers of the premises fired, in order. The number of steps is the number of rules fired (hops).
Example (8 steps, the shortest our generator produces): 40 premises such as
1. Sam : star,2. star earth > band,6. band pine > sea, followed by the query? Sam : sea; answer1.Function composition: push a value through a sequence of lookup tables.
Example
A start value in 0–7, followed by a sequence of lookup tables of 8 entries each (permutations of 0–7). Each table maps a value v to its entry at index v. The answer is the value after applying all tables in order. The number of steps is the number of tables.
Example (2 steps): start value
4, table 1[1, 7, 4, 2, 6, 5, 3, 0], table 2[7, 1, 2, 6, 4, 5, 3, 0]; reasoning states[6, 3], answer3.Since our setup comes from steganographic reasoning, where our vocabulary to express the reasoning steps is limited, all numbers in the reasoning that have no fixed maximum are encoded in binary. This applies to the premise numbers in ProntoQA and HornQA.
Results
Models can learn multi-step reasoning on filler tokens
To test whether models can learn multi-step reasoning on filler tokens, we train three model/task combinations (mod sum and GL(3,2) on Qwen3.6-35B-A3B, mod sum on gpt-oss-20b) in three settings: our method, Pfau et al.'s method, and an answer-only baseline that outputs the final answer directly, with no extra tokens. We train models with each of these methods on the same curriculum (10 → 20 → 40 steps), with the same number of training examples per stage. Each method is trained on each model/task combination with at least 3 different seeds. If a model trained on a filler-token method outperforms the answer-only baseline, we interpret this as a demonstration of filler-token reasoning.
Our method makes learning filler-token reasoning possible. We compare the longest task length learned on filler tokens with a curriculum learning setup. Unlike Pfau et al.'s method, ours clearly beats the answer-only baseline. Dots are the results of training runs from individual seeds, bars are averages over seeds.
We can see that on all of these tasks, our method lets us train the models to reason on filler tokens.
Of course, this only works up to some task length: the model has a fixed number of layers, and these tasks need more successive computation the longer they get. How much more is what we look at next.
Theoretical limits to learning multi-step reasoning on filler tokens
We do this theoretical analysis for two reasons.
First, we want to make sure that our tasks really are multi-step for a transformer. Complexity classes alone do not tell us this. For example, while the sum mod 2 task is in TC⁰, and so is in theory solvable without consecutive computations, a transformer with limited floating-point number precision still cannot solve arbitrarily long mod 2 sums in a single forward pass.
Second, we will see that models can often do more reasoning steps than they have layers. This looks paradoxical at first, if one imagines that each step needs its own layer. But for most tasks one can build circuits whose depth grows sublinearly with the number of steps, so no layer has to be reserved for each step.
We construct upper bounds on how deep a transformer has to be to solve these tasks on filler tokens. We do this by constructing explicit algorithms for each task that a transformer can implement, using swarms of Opus 5.5 agents.
First, we define a set of operations a transformer layer can perform (details in the appendix). The swarms hand in an algorithm, which we then check:
It is then scored on how many layers (successive operations) it needs, and the swarm is prompted to get this score as low as possible.
The precise limits of these operations depend on the model architecture, so we did this separately for both models.
Most of Qwen3.6-35B-A3B's layers use linear attention, which we model as a memory that returns the most recent matching entry. Half of gpt-oss-20b's layers only see a window of recent positions.
Some of these limits had to be chosen by fiat. In particular, we set an MLP to behave like a lookup table whose input is at most 32 bits long. This caps how many reasoning steps can be folded into the same layer, and therefore determines the slope of the curves below. We expect the overall shape of the curves to stay the same under a different limit.
Layers of the shallowest algorithm found for each task, as a function of task length. These are upper bounds, not minima. The dotted line is the model's depth.
The exact algorithms Opus constructed are messy; they can be looked up here.
For every task, the shallowest algorithm we found gets deeper as the task gets longer, so we found no way to fully parallelize any of them. And for every task except HornQA the depth grows much more slowly than the number of steps, which is why the curves stay below the model's depth far beyond 40 or 24 steps.
We see that different tasks scale very differently. In particular, the modular sum task, used as the main task for studying steganographic reasoning by Skaf et al. (2025) and Zolkowski et al. (2025)[2], scales very gently, i.e. it is easy to do on filler tokens.
Empirical limits to learning multi-step reasoning on filler tokens
To make sure we are actually measuring filler-token reasoning, and not just the model getting generally better from fine-tuning in ways that do not rely on the filler tokens, we compare with answer-only fine-tuning: the model answers directly, with no extra tokens. And to make sure that a failure to learn filler-token reasoning is not just because the task is too difficult, we also compare with training the model to reason openly on these tasks. Both are fine-tuning runs with the same number of examples per step. The answer-only runs use the same curriculum; the open-reasoning runs are trained directly at the longest length.
For filler-token reasoning we use both training methods, curriculum and direct. Some task/model combinations do better with one method, some with the other, so we report the longest filler-token reasoning length reached across both methods and multiple seeds.
Longest task length learned with an immediate answer (no dots), with filler-token reasoning, and with open reasoning. The dotted line is the longest length tested.
Again, tasks differ a lot in how well they can be learned. And again there are outliers: HornQA, which theory predicted to be hard to learn on filler tokens, fails after very few steps, and the popular modular sum is learned very easily.
But theoretical and empirical limits don't line up in general: some tasks are theoretically easy but empirically hard (modular arithmetic and function composition hit a wall at 1–3 steps). The pattern seems to be that models have a hard time learning tasks on filler tokens when large packets of information (lookup tables, premises, ...) need to be transported across token positions.
As the theory section showed, we found no fully parallel transformer algorithm for any of these tasks, and some (A5, GL(3,2)) are believed not to be in TC⁰. Pfau et al.'s claim that models cannot do such tasks on filler tokens holds in the limit of infinitely long tasks. But since even small models can learn to do 60 steps on filler tokens, this may not be much comfort in practice.
Conclusion
Multi-step reasoning can also be done on filler tokens. Recent models such as Astra have shown an increased ability to answer reasoning questions without reasoning about them (Nanda, 2026). If that trend continues, it could endanger CoT monitorability even without the specters of looped architectures or steganographic reasoning coming true.
Our experiments serve as a demonstration, and shed some light on which tasks can be learned on filler tokens, and how easily. To get there, we used a non-standard training protocol that is not normally part of an LLM training pipeline, and our models and training runs are quite small. Still, any pressure not to verbalize certain multi-step reasoning could, with enough training, lead to similar results.
We also learned a lot about constructing tasks for studying steganographic reasoning. When we started, our task mix included some other tasks, and we were surprised that some of them did not get harder with more steps at all. We recommend that others researching steganographic or encoded reasoning put the tasks they use through a test like this.
Appendix
Precise definition of the transformer model
We model a transformer with bounded precision, in the spirit of RASP (Weiss, Goldberg & Yahav, 2021). Circuits are decomposed into computation steps that a layer could actually do.
Each token position starts with information about its token and its sequence position.
Some of the limits of these computations are directly readable from the architecture, while others have to be guessed.
We implement this model as an executable program that the constructed circuits have to run on. By seeing how many times a circuit program calls the layers, we can read out its depth.
Details of the training runs
Each curriculum stage trains on 2,000 examples. During training we evaluate on 20 items every 100 steps, and a stage counts as passed once answer accuracy is at least 0.9. The final evaluation uses 200 items. All other hyperparameters are in the configs.
Note that "in theory parallelizable" and "parallelizable in an actual transformer" are not the same thing.
In the papers, these were presented as a coin-flip task (Skaf et al.) and a direction-tracking task (Zolkowski et al.); both are modular sums with some dressing around them.