Qwen State-Ladder Compiler
The one idea you need
Picture a long chain of add, subtract, and multiply, always wrapping around at 97 like a clock. One slip anywhere ruins the answer. The test forced the model to show and be graded on its running total after every single step, not just the end.
The question
If you make a model show and get graded on its running total after every single step, does it follow long calculations more reliably?
What we found
No. Grading the running total after every step never beat an identical model graded only on its final answer, and at full strength it collapsed on the hardest long programs. The real winner was the training schedule: starting on short chains and working up to 24-step ones lifted the hardest cases from near-random to about 1 in 5, and standard cases to about 2 in 5.
Why it matters
For long chains of exact steps, invest in a gradual difficulty ramp rather than dense per-step grading, which can overfit and turn brittle on reworded problems. And save the model that scores best on the hardest cases, not whichever version trained last.
On this page
Results at a glance 8
How to read
Grouped bars for three kinds of 24-step problems: standard wording, reworded, and both wordings paired. Bars compare a plain final-answer model, a gradual-practice model, and two step-grading strengths. Taller is better.
Takeaway → The plain-model bars sit near random-guess level while gradual-practice towers over them; no step-grading bar tops the gradual-practice control, and full-strength grading vanishes on reworded problems.
Data table
| Evaluation split (length 24) | answer-only control | curriculum, no state ladder | state ladder w=1.0 | state ladder w=0.25 |
|---|---|---|---|---|
| standard len24 | 3.1% | 39.1% | 29.7% | 37.5% |
| paraphrase len24 | 1.6% | 15.6% | 0% | 14.1% |
| paired len24 | 1.6% | 21.1% | 14.8% | 19.5% |
Numbers from analysis/summary.md (Final Metrics table); runs/*/results.json
Technical framing
Length-24 exact execution: curriculum drives gains, state ladder does not beat control — The staged length curriculum lifts long-chain execution far above the answer-only baseline; adding state-ladder loss never beats the matched control.
How to read
Lines track how often the whole program is worked exactly right as length grows from 4 to 24 steps, one line per training method. Higher and flatter is better.
Takeaway → All lines start together high and slide down as chains lengthen, with the full-strength step-grading line dropping fastest to the lowest point at 24 steps.
Data table
| Program length (operations) | curriculum, no state ladder | state ladder w=1.0 | state ladder w=0.25 |
|---|---|---|---|
| 4 | 86.7% | 86.7% | 86.7% |
| 8 | 58.6% | 62.5% | 60.9% |
| 12 | 59.4% | 53.1% | 59.4% |
| 24 | 21.1% | 14.8% | 19.5% |
Numbers from analysis/summary.md (Final Metrics table, paired splits)
Technical framing
Exact execution vs program length on paired splits (final checkpoints) — All compiler arms match at short lengths; the full-weight state ladder degrades fastest as chains get longer, especially at length 24.
How to read
One bar per method showing its peak accuracy on the hardest 24-step problems during training, not the final saved model. Taller is better.
Takeaway → At its peak the lighter step-grading bar is tallest, but that moment was never saved, so it doesn't count as a real win.
Data table
| Training arm | paired len24 exact execution |
|---|---|
| curriculum, no state ladder | 25% |
| state ladder w=1.0 | 23.4% |
| state ladder w=0.25 | 28.1% |
Numbers from analysis/summary.md (Best Logged Paired L24 Checkpoints table); analysis/best_logged_checkpoints.csv
Technical framing
Best logged paired length-24 checkpoint (step 800) — By best-logged checkpoint the lighter w=0.25 state loss wins (28.1%), but its final saved checkpoint still trails the no-state control.
How to read
One bar per method showing how often the same 24-step problem, written two different ways, produced identical step-by-step work. Taller is better.
Takeaway → Full-strength step-grading nearly vanishes near zero while the no-grading control stands highest, showing heavy grading made the model inconsistent.
Data table
| Training arm | compiler pair state consistency |
|---|---|
| curriculum, no state ladder | 42.2% |
| state ladder w=1.0 | 1.6% |
| state ladder w=0.25 | 26.6% |
Numbers from analysis/summary.md (Final Metrics table, paired_len24 rows)
Technical framing
Paired length-24 state consistency collapses under full-weight state loss — At the final checkpoint the full-weight state ladder nearly destroys paired length-24 consistency; the no-state control keeps the highest.
How to read
Two pairs of bars compare the step-grading model and the control on the hardest 24-step problems, first after short training then after long training. Taller is better.
Takeaway → Step-grading leads at the short budget but slips below the control after longer training, so its early edge evaporates.
Data table
| Training budget | Full state ladder | No state ladder (control) |
|---|---|---|
| 400-step budget (pilot) | 27.3% | 22.7% |
| 900-step budget (final) | 14.8% | 21.1% |
Numbers from experiments/qwen_state_ladder_compiler/analysis/all_final_metrics.csv
Technical framing
State-ladder edge at 400 steps disappears by 900 steps — The state ladder leads the control at a 400-step budget but falls behind it once training runs to 900 steps.
How to read
Bars compare accuracy on standard versus reworded 24-step problems for three methods; taller is better, and a small gap between the two means the method handles new wording.
Takeaway → Every method dips on reworded problems, but full step-grading collapses to zero while lighter or no grading holds near one in seven.
Data table
| Length-24 eval family | No state ladder (control) | State ladder w=0.25 | Full state ladder |
|---|---|---|---|
| Standard wording | 39.1% | 37.5% | 29.7% |
| Paraphrased wording | 15.6% | 14.1% | 0% |
Numbers from experiments/qwen_state_ladder_compiler/analysis/all_final_metrics.csv
Technical framing
Full-weight state loss wipes out paraphrase generalization at length 24 — All runs drop on paraphrased prompts, but the full-weight state ladder collapses to 0% while lighter or no state loss keeps ~14-16%.
How to read
Lines track accuracy on the hardest 24-step problems across training steps for each method; higher is better.
Takeaway → Every line hugs zero until the final long-chain stage, spikes around step 800, then dips, so the last saved model is not the best one.
Data table
| Training step | No state ladder (control) | Full state ladder | State ladder w=0.25 |
|---|---|---|---|
| 1 | 0.8% | 0% | 0.8% |
| 200 | 0% | 0% | 0.8% |
| 201 | 0% | 2.3% | 0.8% |
| 300 | 1.6% | 1.6% | 0.8% |
| 400 | 0.8% | 1.6% | 1.6% |
| 401 | 0% | 0.8% | 0% |
| 600 | 11.7% | 5.5% | 4.7% |
| 601 | 12.5% | 7.8% | 3.9% |
| 800 | 25% | 23.4% | 28.1% |
| 900 | 21.1% | 14.8% | 19.5% |
Numbers from experiments/qwen_state_ladder_compiler/runs/*/train_log.csv
Technical framing
Long-chain execution only emerges in the final curriculum stage — All arms stay near 0% until the long-length stage, peak at step 800, then dip by step 900 - final checkpoints are not the best ones.
How to read
Bars show how accurately each piece of a program is copied (starting value, operations, arguments) versus getting the entire 24-step program exactly right. Taller is better.
Takeaway → Per-piece bars sit near perfect while whole-program bars sink to about one in five, as tiny scattered argument slips compound over 24 steps.
Data table
| Compilation metric (paired length-24) | No state ladder (control) | State ladder w=0.25 | Full state ladder |
|---|---|---|---|
| Init token | 100% | 100% | 100% |
| Operation tokens | 99.9% | 99.8% | 95.5% |
| Argument tokens | 93.5% | 93.1% | 87.3% |
| Whole program exact | 20.3% | 18.8% | 14.8% |
Numbers from experiments/qwen_state_ladder_compiler/analysis/all_final_metrics.csv
Technical framing
Per-token accuracy stays high while whole programs fail at length 24 — Init and op tokens are near-perfect; scattered argument errors (87-93% per token) compound over 24 ops to sink whole-program exactness.
In the author’s words from the Report · “Abstract”
This experiment tests whether dense intermediate-state supervision improves a QLoRA-adapted Qwen/Qwen3-4B numeric-copy compiler on long modular-arithmetic programs. Each prompt describes an initial value and a sequence of add, subtract, and multiply updates modulo 97. A trainable compiler reads Qwen hidden states, selects token positions for the initial value, operations, and arguments, copies exact symbols from deterministic token maps, and executes the copied program with an invisible modular runtime. The main intervention is a state ladder: in addition to trace supervision and final-answer executor loss, the compiler is trained to match the latent modular state after each operation. … Read the full result →
Overview
Standalone intermediate-state ladder experiment for a Qwen numeric-copy compiler.
The experiment trains a numeric-copy compiler on modular-arithmetic programs and adds dense supervision for the latent modular state after each operation. The main comparison tests whether that per-step state ladder improves long-chain exact execution over a matched trace-supervised compiler without the state loss.
The completed Qwen3-4B runs show that the staged length curriculum is strongly useful, while fixed-weight state-ladder loss is mixed. A lighter state loss gives the best logged paired length-24 checkpoint, but the final saved no-state curriculum control is strongest on paired length-24 exact execution.
Layout
src/qwen_state_ladder_compiler_experiment.py training and evaluation harness
src/analyze_qwen_state_ladder_compiler.py run aggregation and plots
runs/ lightweight JSON and CSV outputs
reports/ experiment log and standalone write-up
analysis/ aggregate tables and figuresLarge checkpoints are stored outside the experiment directory:
large_artifacts/qwen_state_ladder_compiler/checkpoints/Reading Order
reports/qwen_state_ladder_compiler_paper.mdanalysis/summary.mdanalysis/final_metrics.csvreports/qwen_state_ladder_compiler_experiment_log.md
Main Runs
runs/main_qwen3_4b_qlora_state_ladder_curriculum_s900/
runs/main_qwen3_4b_qlora_state_ladder_w025_curriculum_s900/
runs/control_qwen3_4b_qlora_curriculum_no_state_ladder_s900/
runs/control_qwen3_4b_qlora_answer_only_curriculum_s900/Report
Rendered from reports/qwen_state_ladder_compiler_paper.md
Abstract
This experiment tests whether dense intermediate-state supervision improves a QLoRA-adapted Qwen/Qwen3-4B numeric-copy compiler on long modular-arithmetic programs. Each prompt describes an initial value and a sequence of add, subtract, and multiply updates modulo 97. A trainable compiler reads Qwen hidden states, selects token positions for the initial value, operations, and arguments, copies exact symbols from deterministic token maps, and executes the copied program with an invisible modular runtime.
The main intervention is a state ladder: in addition to trace supervision and final-answer executor loss, the compiler is trained to match the latent modular state after each operation. Training uses a four-stage length curriculum: 1-4 steps, 1-8 steps, 1-12 steps, and 8-24 steps.
The main result is mixed. The curriculum itself is strongly useful: the matched no-state-ladder compiler reaches 39.1% exact execution on standard length 24, 15.6% on paraphrase length 24, and 21.1% on paired length 24. Final-answer-only QLoRA remains at chance. The state-ladder loss does not clearly beat the matched curriculum control at final checkpoint. A full-weight state ladder improves some shorter and medium splits but hurts length-24 robustness. A lighter state loss nearly matches the control and has the best logged paired length-24 checkpoint, but its final checkpoint still trails the control on paired length 24.
Setup
- Base model:
Qwen/Qwen3-4B - Loader:
AutoModelForCausalLM - Quantization: 4-bit NF4
- Trainable update: LoRA rank 8, alpha 16, dropout 0.05, target
all-linear - Trainable LoRA parameters: 16,515,072
- Compiler head width: 768
- Task: modular arithmetic programs modulo 97
- Curriculum:
short:1:4:200,medium:1:8:200,train:1:12:200,long:8:24:300 - Eval lengths: 4, 8, 12, and 24 update steps
- Eval templates: standard, paraphrase, and paired standard/paraphrase renderings of the same program
- Eval size: 64 examples per unpaired split; 64 program pairs per paired split
- Hardware: NVIDIA RTX 6000 Ada Generation, 48 GB class VRAM
- Large checkpoints:
large_artifacts/qwen_state_ladder_compiler/checkpoints/
The compiler predicts:
- the token position of the initial value;
- ordered operation token positions;
- ordered argument token positions.
Values and operations are copied from per-token maps. The copied program is executed exactly modulo 97. For state-ladder training, the differentiable executor also returns a distribution over the modular state after every active step, and the training loss includes per-step NLL against the true state trajectory.
Conditions
| Run | Variant | Purpose |
|---|---|---|
main_qwen3_4b_qlora_state_ladder_curriculum_s900 | copy_trace_state_ladder | Full state-ladder loss weight 1.0. |
main_qwen3_4b_qlora_state_ladder_w025_curriculum_s900 | copy_trace_state_ladder | Lighter state-ladder loss weight 0.25. |
control_qwen3_4b_qlora_curriculum_no_state_ladder_s900 | copy_trace | Matched curriculum compiler without state-ladder loss. |
control_qwen3_4b_qlora_answer_only_curriculum_s900 | direct | Final-answer-only QLoRA control without compiler structure. |
All compiler arms use the same explicit training seed (2427), same curriculum, same paired training distribution, same LoRA configuration, and same evaluation splits.
Final Exact Execution
| Split | State Ladder 1.0 | State Ladder 0.25 | No State Ladder | Answer-Only QLoRA |
|---|---|---|---|---|
| Standard L4 | 93.8% | 90.6% | 92.2% | 0.0% |
| Standard L8 | 67.2% | 64.1% | 65.6% | 0.0% |
| Standard L12 | 51.6% | 45.3% | 43.8% | 1.6% |
| Standard L24 | 29.7% | 37.5% | 39.1% | 3.1% |
| Paraphrase L4 | 84.4% | 75.0% | 78.1% | 0.0% |
| Paraphrase L8 | 64.1% | 59.4% | 64.1% | 0.0% |
| Paraphrase L12 | 56.2% | 57.8% | 59.4% | 0.0% |
| Paraphrase L24 | 0.0% | 14.1% | 15.6% | 1.6% |
| Paired L4 | 86.7% | 86.7% | 86.7% | 1.6% |
| Paired L8 | 62.5% | 60.9% | 58.6% | 1.6% |
| Paired L12 | 53.1% | 59.4% | 59.4% | 0.0% |
| Paired L24 | 14.8% | 19.5% | 21.1% | 1.6% |
The answer-only control stays near 97-way chance. The compiler arms therefore owe their gains to the structured compiler and curriculum, not to ordinary final-answer fitting.
The no-state-ladder curriculum control is the strongest final checkpoint on the hardest target, paired length 24. Full-weight state loss helps standard length 12 but damages long paraphrase and paired performance. The lighter state loss is less damaging and reaches 37.5% standard L24, but still trails the no-state control on final paired L24.
State Trajectory Metrics
state_all_exact measures whether every intermediate modular state is correct. state_prefix_fraction measures how far execution gets before the first state error, as a fraction of all active steps.
| Split | State 1.0 All Exact | State 0.25 All Exact | Control All Exact | State 1.0 Prefix | State 0.25 Prefix | Control Prefix |
|---|---|---|---|---|---|---|
| Standard L12 | 51.6% | 45.3% | 43.8% | 76.2% | 71.0% | 70.6% |
| Standard L24 | 28.1% | 37.5% | 39.1% | 62.8% | 63.7% | 63.7% |
| Paraphrase L12 | 56.2% | 57.8% | 59.4% | 74.7% | 72.4% | 74.2% |
| Paraphrase L24 | 0.0% | 14.1% | 15.6% | 40.0% | 46.1% | 45.2% |
| Paired L12 | 53.1% | 57.8% | 57.8% | 78.1% | 78.5% | 78.0% |
| Paired L24 | 14.8% | 18.8% | 20.3% | 49.8% | 49.0% | 48.6% |
The state metrics agree with the exact-execution result. The state ladder does not produce a clear final-checkpoint improvement on long programs. Weight 1.0 gives the best standard L12 trajectory but is too brittle at paraphrase L24. Weight 0.25 is more stable but still does not beat the no-state control at final L24.
Paired State Consistency
Paired splits render the same latent program twice, once in the standard template and once in a paraphrased template. compiler_pair_state_consistency measures whether the two renderings produce the same complete predicted state trajectory.
| Split | State Ladder 1.0 | State Ladder 0.25 | No State Ladder |
|---|---|---|---|
| Paired L4 | 98.4% | 95.3% | 98.4% |
| Paired L8 | 84.4% | 87.5% | 84.4% |
| Paired L12 | 71.9% | 76.6% | 73.4% |
| Paired L24 | 1.6% | 26.6% | 42.2% |
The full-weight state ladder collapses on paired L24 consistency at the final checkpoint. The lighter state ladder is better, but the matched no-state curriculum control still has the highest final paired L24 consistency.
Best Logged Checkpoints
The final checkpoint is the primary saved model for each run. Training logs also show that paired L24 peaked before the final checkpoint.
| Run | Step | Paired L24 Exec | Paired L24 Prefix | Paired L24 State Consistency | Standard L24 Exec | Paraphrase L24 Exec |
|---|---|---|---|---|---|---|
| No state ladder | 800 | 25.0% | 51.1% | 53.1% | 32.8% | 23.4% |
| State ladder 1.0 | 800 | 23.4% | 52.3% | 43.8% | 31.2% | 10.9% |
| State ladder 0.25 | 800 | 28.1% | 52.7% | 56.2% | 35.9% | 21.9% |
The lighter state ladder has the best logged paired L24 checkpoint. However, those intermediate weights were not saved in this run; this table is a training-log result, not a saved-checkpoint result. The final saved checkpoint comparison remains mixed and favors the no-state control on paired L24.
Interpretation
The positive result is the length curriculum. It pushes the numeric-copy compiler into a much stronger long-chain regime while final-answer-only QLoRA remains at chance. The no-state curriculum compiler reaches 39.1% standard L24 and 21.1% paired L24 exact execution with only 900 optimizer steps.
The state-ladder hypothesis is not cleanly supported. Dense intermediate-state loss changes learning dynamics and can improve some medium splits, but at the tested weights it does not reliably improve final long-chain execution over the matched curriculum control. The weight-0.25 ablation is informative: lowering the state loss reduces the damage and gives the best logged paired L24 checkpoint, but still does not produce a final-checkpoint win.
The strongest conclusion is therefore:
- A staged long-chain curriculum is a high-leverage way to improve the Qwen numeric-copy compiler.
- Dense state supervision is useful diagnostically and may help with checkpoint timing, but the tested fixed-weight objective is not yet the right long-chain regularizer.
- The remaining failure is still compounding exactness: per-step symbol metrics are high, but length-24 exact execution remains sensitive to small operation and argument errors.
Recommended Follow-Up
The next run should make checkpoint selection and state loss scheduling explicit:
- save checkpoints at each stage boundary and every long-stage evaluation;
- select by paired length-24 validation exactness, not final step;
- anneal state loss from 1.0 to 0.0 during the long stage, or use state loss only through length 12 and disable it for length 24;
- run a matched no-state curriculum control with the same checkpoint-selection rule.
The success criterion should be a saved checkpoint that beats the no-state curriculum control on paired L24 exact execution and paired L24 state consistency.
Artifacts
Small files:
- Source:
experiments/qwen_state_ladder_compiler/src/ - Runs:
experiments/qwen_state_ladder_compiler/runs/ - Analysis:
experiments/qwen_state_ladder_compiler/analysis/ - Report:
experiments/qwen_state_ladder_compiler/reports/qwen_state_ladder_compiler_paper.md - Manifest:
experiments/qwen_state_ladder_compiler/checkpoint_manifest.csv
Large files:
- State ladder weight 1.0:
large_artifacts/qwen_state_ladder_compiler/checkpoints/main_qwen3_4b_qlora_state_ladder_curriculum_s900/ - State ladder weight 0.25:
large_artifacts/qwen_state_ladder_compiler/checkpoints/main_qwen3_4b_qlora_state_ladder_w025_curriculum_s900/ - No-state curriculum control:
large_artifacts/qwen_state_ladder_compiler/checkpoints/control_qwen3_4b_qlora_curriculum_no_state_ladder_s900/ - Answer-only control:
large_artifacts/qwen_state_ladder_compiler/checkpoints/control_qwen3_4b_qlora_answer_only_curriculum_s900/
Experiment log 7
Show the running log (7 entries, 2026-06-21)
Objective
Test whether dense supervision of the latent modular state after every operation improves a Qwen numeric-copy compiler's long-chain exact execution.
Experiment Question
Can a QLoRA-adapted model learn a more reliable executable compiler interface when training includes a curriculum over chain lengths and a per-step state target, rather than only final-answer executor loss plus symbol trace supervision?
Planned Conditions
- Smoke tests for intermediate-state targets, differentiable state trajectories, and curriculum stage handling.
- Main QLoRA numeric-copy compiler with trace loss, executor loss, and state-ladder loss.
- Matched QLoRA numeric-copy compiler with the same curriculum but without state-ladder loss.
- Final-answer-only QLoRA control with the same model and total step budget.
Primary Metrics
executor_accuracy: exact final answer after copying and running the compiled program.program_exact: exact full compiled-program correctness.state_accuracy: per-step intermediate-state accuracy.state_all_exact: whether every intermediate state in a program is correct.state_prefix_fraction: fraction of steps before the first state error, aggregated across examples.compiler_pair_state_consistency: whether paired renderings compile to the same state trajectory.init_accuracy,op_accuracy,arg_accuracy: copied symbol correctness.op_pos_accuracy,arg_pos_accuracy: ordered slot localization.
Artifact Policy
Lightweight outputs stay in experiments/qwen_state_ladder_compiler/runs/.
Large adapters and head checkpoints stay in large_artifacts/qwen_state_ladder_compiler/checkpoints/.
Log
2026-06-21
- Created standalone experiment directory and external checkpoint path.
- Forked the numeric-copy compiler harness into a state-ladder harness.
- Added gold intermediate state targets, differentiable state trajectories, state-ladder loss, state trajectory diagnostics, and curriculum stage parsing.
- Ran tiny-model smoke tests for the state-ladder and no-state variants. Both completed and produced state metrics.
- Ran a Qwen3-4B QLoRA smoke test for the state-ladder path. The 4-bit LoRA path, trajectory loss, and paired evaluation all worked.
- Ran a 400-step Qwen pilot with full state-ladder loss and a matched no-state-ladder control. The pilot showed that the curriculum itself was powerful and that state loss improved paired/paraphrase trajectory metrics at short budget.
Ran full 900-step Qwen conditions:
main_qwen3_4b_qlora_state_ladder_curriculum_s900control_qwen3_4b_qlora_curriculum_no_state_ladder_s900control_qwen3_4b_qlora_answer_only_curriculum_s900
- The full-weight state ladder improved standard L12 but hurt final L24, especially paraphrase and paired L24.
Ran a targeted lighter-loss ablation:
main_qwen3_4b_qlora_state_ladder_w025_curriculum_s900
- The lighter state ladder reduced the damage and had the best logged paired L24 checkpoint at step 800, but its final checkpoint still trailed the no-state curriculum control on paired L24.
- Final result: the staged length curriculum is the main positive finding. Dense state-ladder loss is not a robust fixed-weight improvement under these settings.
Figures 5
Data files 11
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
analysis/all_final_metrics.csv20 kBanalysis/final_metrics.csv12 kBruns/control_qwen3_4b_qlora_answer_only_curriculum_s900/results.json54 kBruns/control_qwen3_4b_qlora_curriculum_no_state_ladder_s900/results.json135 kBruns/main_qwen3_4b_qlora_state_ladder_curriculum_s900/results.json136 kBruns/main_qwen3_4b_qlora_state_ladder_w025_curriculum_s900/results.json136 kBruns/pilot_qwen3_4b_qlora_curriculum_no_state_ladder_s400/results.json111 kBruns/pilot_qwen3_4b_qlora_state_ladder_curriculum_s400/results.json112 kBruns/smoke_qwen3_4b_qlora_state_ladder/results.json20 kBruns/smoke_tiny_no_state_ladder/results.json20 kBruns/smoke_tiny_state_ladder/results.json33 kB
Reproduce
The run commands are documented inside the experiment folder (see the README).