Qwen Slot-Stability Compiler
The one idea you need
The small model never does the arithmetic in its head. A helper reads its internal signals, points at the exact starting number and each operation in the prompt, copies them out, and types them into a hidden calculator that runs every step exactly.
The question
Can you get reliable step-by-step arithmetic from a small model by pulling out the numbers and operations and running them through a calculator, instead of letting it answer directly?
What we found
Yes, mostly. The point-and-compute helper solved about 91% of short problems and around half of medium ones, while training the same model to just emit the final answer never beat random guessing, about 1 in 60, at any length. But an extra rule forcing two rewordings of a problem to agree was a wash: it helped the longest reworded chains yet hurt agreement on medium ones.
Why it matters
For narrow step-by-step symbolic work, wire a small model to an exact calculator and supervise the extracted steps, rather than fine-tuning it to output answers. Don't expect a cross-wording agreement penalty to fix errors that compound over long chains.
On this page
Results at a glance 6
How to read
The horizontal axis is how many operation steps a problem has, from 4 to 24; the vertical axis is how often the exact final answer is right. The two upper lines are the point-and-compute helper, with and without the agreement rule; the flat bottom line guesses the answer directly. Higher is better.
Takeaway → Both helper lines sit far above the flat bottom line, which never leaves the floor, and both slope down as chains get longer: extracting and running the steps works, guessing does not.
Data table
| program length (update steps) | stability compiler | no-stability compiler | answer-only QLoRA |
|---|---|---|---|
| 4 | 90.6% | 85.9% | 0% |
| 8 | 59.4% | 60.9% | 0% |
| 12 | 43.8% | 34.4% | 1.6% |
| 24 | 0% | 6.2% | 0% |
Numbers from analysis/final_metrics.csv
Technical framing
Exact execution accuracy by chain length (standard prompts) — Trace-supervised compilers solve short and medium chains far above chance; answer-only QLoRA stays at chance at every length.
How to read
Same setup, but every problem is rephrased into different wording. Horizontal axis is problem length; vertical axis is how often the exact answer is right. The two upper lines are the two helper versions, the bottom line guesses directly. Higher is better.
Takeaway → The two helper lines nearly overlap except at the longest length, where the version with the agreement rule pulls slightly ahead, 23% versus 16%.
Data table
| program length (update steps) | stability compiler | no-stability compiler | answer-only QLoRA |
|---|---|---|---|
| 4 | 79.7% | 79.7% | 1.6% |
| 8 | 59.4% | 59.4% | 1.6% |
| 12 | 56.2% | 57.8% | 0% |
| 24 | 23.4% | 15.6% | 1.6% |
Numbers from analysis/final_metrics.csv
Technical framing
Exact execution accuracy on paraphrased prompts — The stability loss's clearest win is paraphrase length 24 (23.4% vs 15.6%); at other lengths the two compiler arms are essentially tied.
How to read
Bars are grouped by problem length. Height is how often one problem, worded two ways, produces the same final answer. One bar per length adds the agreement rule, the other leaves it out. Taller means more consistent.
Takeaway → The rule wins at the shortest length but loses clearly at medium length, 75% versus 88%, and ties at the next: it is a mixed bag, not a steady gain.
Data table
| paired split | stability compiler | no-stability compiler |
|---|---|---|
| length 4 | 96.9% | 93.8% |
| length 8 | 75% | 87.5% |
| length 12 | 68.8% | 68.8% |
| length 24 | 15.6% | 14.1% |
Numbers from analysis/final_metrics.csv
Technical framing
Does the stability loss make paired answers more consistent? — The explicit stability penalty is a mixed regularizer: it helps at length 4 and 24 but loses badly at length 8 and ties at 12.
How to read
Bars compare the two versions on medium-long reworded pairs, halfway through training and at the end. The left group is exact-answer accuracy, the right group is how often both wordings agree. One bar per group uses the rule, the other does not. Taller is better.
Takeaway → Halfway through, the rule's version agrees far more often, 36% versus 5%, but by the end both versions land together near 69%: the early edge evaporates.
Data table
| metric and training step | stability compiler | no-stability compiler |
|---|---|---|
| exec acc, step 300 | 30.5% | 20.3% |
| exec acc, step 600 | 58.6% | 57.8% |
| pair consistency, step 300 | 35.9% | 4.7% |
| pair consistency, step 600 | 68.8% | 68.8% |
Numbers from report table (reports/qwen_slot_stability_compiler_paper.md, Training Dynamics + Exact Execution tables)
Technical framing
Paired length-12: stability's mid-training edge disappears by step 600 — At step 300 the stability arm led on paired L12 (35.9% vs 4.7% consistency), but the matched control fully catches up by step 600.
How to read
The horizontal axis is problem length. Three upper lines track how often each piece is identified correctly, the starting number, the operations, and the argument numbers; the bottom line is getting the whole sequence exactly right. Higher is better.
Takeaway → Individual pieces stay near-perfect, but the whole-sequence line collapses to zero on the longest problems: one small slip anywhere ruins the entire chain.
Data table
| Program chain length | Init slot accuracy | Operation slot accuracy | Argument slot accuracy | Whole program exact match |
|---|---|---|---|---|
| 4 | 100% | 100% | 97.7% | 90.6% |
| 8 | 100% | 100% | 94.7% | 59.4% |
| 12 | 100% | 100% | 94.3% | 43.8% |
| 24 | 100% | 96.6% | 87% | 0% |
Technical framing
Where compilation breaks: per-slot decode accuracy vs exact program match — Init and op slots stay near-perfect; small per-argument errors compound so exact 24-step programs fall to zero (stability run, standard prompts).
How to read
Bars are grouped by problem length, for reworded prompts. Each of the two methods is measured halfway through training and at the end. Height is how often the exact answer is right. Taller is better; compare halfway against end within each method.
Takeaway → End-of-training bars tower over halfway bars at medium and long lengths: the second half of training is where the reworded problems actually get solved.
Data table
| Program chain length (paraphrased prompts) | Stability, step 300 | Stability, step 600 | No stability, step 300 | No stability, step 600 |
|---|---|---|---|---|
| Length 4 | 79.7% | 79.7% | 82.8% | 79.7% |
| Length 8 | 51.6% | 59.4% | 46.9% | 59.4% |
| Length 12 | 25% | 56.2% | 4.7% | 57.8% |
| Length 24 | 1.6% | 23.4% | 3.1% | 15.6% |
Numbers from experiments/qwen_slot_stability_compiler/analysis/all_final_metrics.csv (paraphrase splits, s300 and s600 runs)
Technical framing
Paraphrase accuracy by checkpoint: the second half of training does the heavy lifting — At step 300 the stability loss protects length-12 paraphrases (0.25 vs 0.05), but by step 600 the plain paired compiler catches up.
In the author’s words from the Report · “Abstract”
At 600 steps it improves some longer exact-execution splits, including standard length 12 and paired length 24, but it ties or trails the matched control on other splits and does not produce uniformly higher paired consistency.
Overview
Standalone paired paraphrase slot-stability experiment for a Qwen numeric-copy compiler.
The experiment trains a numeric-copy compiler on paired renderings of the same modular-arithmetic program. The stability condition penalizes drift in copied slot distributions across paired wording variants while preserving trace and executor supervision.
The completed 600-step Qwen3-4B runs show that trace-supervised numeric-copy compilation works far above chance, while answer-only QLoRA stays at chance. The explicit paired stability loss is a mixed regularizer rather than a clean win over the matched paired-data trace compiler.
Layout
src/qwen_slot_stability_compiler_experiment.py training and evaluation harness
src/analyze_qwen_slot_stability_compiler.py run aggregation and plots
runs/ lightweight JSON and CSV outputs
reports/ experiment log and standalone write-up
analysis/ aggregate tables and figuresLarge checkpoints are stored outside the experiment directory:
large_artifacts/qwen_slot_stability_compiler/checkpoints/Reading Order
reports/qwen_slot_stability_compiler_paper.mdanalysis/summary.mdanalysis/final_metrics.csvreports/qwen_slot_stability_compiler_experiment_log.md
Main Runs
runs/main_qwen3_4b_qlora_slot_stability_mixed_l12_s600/
runs/control_qwen3_4b_qlora_paired_no_stability_mixed_l12_s600/
runs/control_qwen3_4b_qlora_answer_only_mixed_l12_s600/Report
Rendered from reports/qwen_slot_stability_compiler_paper.md
Abstract
This experiment tests whether a QLoRA-adapted Qwen/Qwen3-4B model can expose a stable executable program interface for modular arithmetic prompts. Each prompt describes an initial value and a sequence of add, subtract, and multiply updates modulo 97. A trainable compiler reads Qwen hidden states, selects the token positions for the initial value, operations, and arguments, copies exact symbols from deterministic token maps, and executes the copied program with an invisible modular runtime.
The main intervention is paired paraphrase stability. Training batches contain two renderings of the same underlying program, and the stability condition penalizes disagreement between the copied slot distributions for the paired renderings. The matched control receives the same paired data, trace supervision, executor loss, LoRA rank, batch schedule, training seed, and evaluation splits, but no stability loss.
The result is mixed. Trace-supervised numeric-copy compilation works: both compiler arms solve short and medium chains far above chance, while final-answer-only QLoRA remains at chance. The paired stability loss is not a broad win. At 600 steps it improves some longer exact-execution splits, including standard length 12 and paired length 24, but it ties or trails the matched control on other splits and does not produce uniformly higher paired consistency.
Setup
- Base model:
Qwen/Qwen3-4B - Loader:
AutoModelForCausalLM - Quantization: 4-bit NF4
- Trainable update: LoRA rank 8, alpha 16, dropout 0.05, target
all-linear - Trainable LoRA parameters: 16,515,072
- Compiler head width: 768
- Task: modular arithmetic programs modulo 97
- Train lengths: 1-12 update steps
- Eval lengths: 4, 8, 12, and 24 update steps
- Eval templates: standard, paraphrase, and paired standard/paraphrase renderings of the same program
- Eval size: 64 examples per unpaired split; 64 program pairs per paired split
- Hardware: NVIDIA RTX 6000 Ada Generation, 48 GB class VRAM
- Large checkpoints:
large_artifacts/qwen_slot_stability_compiler/checkpoints/
The compiler predicts:
- the token position of the initial value;
- ordered operation token positions;
- ordered argument token positions.
Values and operations are copied from per-token maps rather than classified from hidden states alone. The copied program is executed exactly modulo 97 for accuracy. During training, a differentiable executor provides final-answer loss, and trace/selection losses supervise the ordered slot interface.
Conditions
| Run | Variant | Purpose |
|---|---|---|
main_qwen3_4b_qlora_slot_stability_mixed_l12_s600 | copy_trace_stability | Numeric-copy compiler with trace loss, executor loss, and paired stability loss. |
control_qwen3_4b_qlora_paired_no_stability_mixed_l12_s600 | copy_trace | Matched numeric-copy compiler without the stability loss. |
control_qwen3_4b_qlora_answer_only_mixed_l12_s600 | direct | Final-answer-only QLoRA control without compiler structure. |
The two compiler arms use the same data seed and the same explicit training seed (2148). Their step-300 metrics reproduce exactly across the 300-step pilot and the 600-step run, so the final comparison is controlled for initialization and batch order.
Exact Execution Results
| Split | Stability Compiler | No-Stability Compiler | Answer-Only QLoRA |
|---|---|---|---|
| Standard L4 | 90.6% | 85.9% | 0.0% |
| Standard L8 | 59.4% | 60.9% | 0.0% |
| Standard L12 | 43.8% | 34.4% | 1.6% |
| Standard L24 | 0.0% | 6.2% | 0.0% |
| Paraphrase L4 | 79.7% | 79.7% | 1.6% |
| Paraphrase L8 | 59.4% | 59.4% | 1.6% |
| Paraphrase L12 | 56.2% | 57.8% | 0.0% |
| Paraphrase L24 | 23.4% | 15.6% | 1.6% |
| Paired L4 | 86.7% | 85.9% | 0.0% |
| Paired L8 | 57.8% | 60.2% | 1.6% |
| Paired L12 | 58.6% | 57.8% | 0.0% |
| Paired L24 | 22.7% | 17.2% | 1.6% |
The direct answer-only control stays at chance across all lengths and prompt modes. The compiler arms are therefore not merely benefiting from generic QLoRA adaptation; the trace-supervised symbolic interface is doing the work.
Paired Consistency
Paired splits render the same latent program twice, once in the standard template and once in a paraphrased template. Answer consistency measures whether the two renderings produce the same executed answer. Both correct measures whether both renderings are exactly correct. Program consistency measures whether the compiled initial value, operation sequence, and argument sequence agree across the pair.
| Split | Stability Answer Consistency | Control Answer Consistency | Stability Both Correct | Control Both Correct | Stability Program Consistency | Control Program Consistency |
|---|---|---|---|---|---|---|
| Paired L4 | 96.9% | 93.8% | 85.9% | 84.4% | 96.9% | 93.8% |
| Paired L8 | 75.0% | 87.5% | 53.1% | 57.8% | 75.0% | 87.5% |
| Paired L12 | 68.8% | 68.8% | 53.1% | 53.1% | 68.8% | 68.8% |
| Paired L24 | 15.6% | 14.1% | 9.4% | 7.8% | 12.5% | 14.1% |
The stability penalty does not reliably increase paired consistency. It helps L4 consistency and slightly improves L24 answer consistency, but the no-stability control is much better at L8 consistency and ties L12. The most defensible interpretation is that paired data plus trace supervision already induces a fairly stable compiler; the explicit KL penalty is only a weak additional bias.
Diagnostics
The compiler arms learn the easy parts almost perfectly:
- initial value accuracy is 100.0% on every main compiler split;
- operation accuracy is at or near 100.0% on trained and medium lengths;
- operation position accuracy is at or near 100.0% except at length 24;
- argument accuracy remains high per step, but exact program execution compounds the remaining errors.
This is a compounding-error regime. A 90-95% per-step argument extractor can still fail often on length-24 exact execution because every copied argument must be correct.
Training Dynamics
At step 300, the stability arm showed a large paired length-12 consistency advantage:
| Split | Stability Exec | Control Exec | Stability Pair Consistency | Control Pair Consistency |
|---|---|---|---|---|
| Paired L12, step 300 | 30.5% | 20.3% | 35.9% | 4.7% |
By step 600, that advantage mostly disappeared on paired L12, while both compiler arms improved. This suggests the stability objective may shape mid-training behavior, but under this budget the matched trace compiler catches up on the core paired L12 interface.
Interpretation
The positive result is the Qwen-attached compiler itself. With only 600 QLoRA steps, the model and compiler learn an invisible executable interface that solves many modular programs. The final-answer-only QLoRA control does not learn the task at all under the same step budget.
The negative result is the stability loss as a central mechanism. It does not produce a clean monotonic improvement over a matched paired-data trace compiler. The strongest single gain is paraphrase length 24 exact execution, where stability reaches 23.4% versus 15.6%. The strongest loss is paired length 8 consistency, where stability reaches 75.0% versus 87.5%.
The result supports a narrow claim:
- Qwen3-4B hidden states can be adapted to feed an exact symbolic executor through a learned numeric-copy compiler.
- Trace supervision is far more effective than final-answer-only QLoRA for this task.
- Paired paraphrase data is useful, but the tested symmetric-KL stability penalty is not a decisive standalone improvement.
It does not support a claim of broad posttraining intelligence amplification. It shows a practical way to attach an executable latent tool interface to a local 4B-class model for a narrow serial symbolic workload.
Recommended Follow-Up
The next intervention should target long-chain compounding errors directly rather than adding another agreement penalty. The most promising design is a length curriculum with validation-selected checkpoints:
- train with dense trace supervision and paired templates as in this experiment;
- oversample lengths 12-24 after the compiler reaches high slot accuracy on lengths 4-8;
- add a per-step executor consistency loss that compares intermediate states, not just final answers;
- select by paired length-24
program_exact, not by training loss.
Success should require a clear paired length-24 gain over the no-stability compiler, not only a short-chain gain.
Artifacts
Small files:
- Source:
experiments/qwen_slot_stability_compiler/src/ - Runs:
experiments/qwen_slot_stability_compiler/runs/ - Analysis:
experiments/qwen_slot_stability_compiler/analysis/ - Report:
experiments/qwen_slot_stability_compiler/reports/qwen_slot_stability_compiler_paper.md - Manifest:
experiments/qwen_slot_stability_compiler/checkpoint_manifest.csv
Large files:
- Stability adapter and heads:
large_artifacts/qwen_slot_stability_compiler/checkpoints/main_qwen3_4b_qlora_slot_stability_mixed_l12_s600/ - Matched no-stability adapter and heads:
large_artifacts/qwen_slot_stability_compiler/checkpoints/control_qwen3_4b_qlora_paired_no_stability_mixed_l12_s600/ - Answer-only adapter and heads:
large_artifacts/qwen_slot_stability_compiler/checkpoints/control_qwen3_4b_qlora_answer_only_mixed_l12_s600/
Experiment log 7
Show the running log (7 entries, 2026-06-21)
Objective
Test whether paired paraphrase consistency improves a Qwen numeric-copy compiler's ordered slot binding across wording variants and longer arithmetic chains.
Experiment Question
Can a QLoRA-adapted model learn a more stable ordered compiler interface when each training batch includes paired renderings of the same underlying program and the copied slot distributions are explicitly regularized to agree?
Planned Conditions
- Smoke tests for the paired dataset and stability loss.
- Main QLoRA numeric-copy compiler with paired slot-stability loss.
- Matched QLoRA numeric-copy compiler without the stability loss, trained on the same paired data distribution.
- Optional final-answer controls if the main comparison is ambiguous.
Primary Metrics
executor_accuracy: exact execution accuracy after copying and running the compiled program.program_exact: exact full compiled-program correctness.executor_pair_answer_consistency: whether paired standard/paraphrase renderings execute to the same predicted answer.executor_pair_both_correct: whether both renderings in a pair are exactly correct.compiler_pair_program_consistency: whether paired renderings compile to the same init/op/arg program.init_accuracy,op_accuracy,arg_accuracy: copied symbol correctness.op_pos_accuracy,arg_pos_accuracy: ordered slot localization.- Standard and paraphrase splits are evaluated separately at multiple chain lengths.
Artifact Policy
Lightweight outputs stay in experiments/qwen_slot_stability_compiler/runs/.
Large adapters and head checkpoints stay in large_artifacts/qwen_slot_stability_compiler/checkpoints/.
Log
2026-06-21
- Created standalone experiment directory and external checkpoint path.
- Implemented paired program rendering, paired batch sampling, and slot-stability losses.
- Found and fixed a NaN gradient in the stability KL. The issue came from
F.kl_divtaking a targetlog(0)path for masked lexical classes. Replaced it with an explicit probability-times-log-ratio symmetric KL. - Added paired evaluation splits so standard/paraphrase renderings of the same latent program can be scored for answer and program consistency.
- Ran tiny-model smoke tests and a Qwen3-4B QLoRA smoke test. Both completed after the KL fix.
- Ran a 300-step Qwen3-4B QLoRA stability pilot and a matched no-stability pilot. The stability arm showed a large paired length-12 consistency advantage at step 300, but the result needed a longer matched check.
- Added
--train_seedso the control arm can share the stability arm's compiler initialization and batch sampling seed while keeping the same generated datasets. Ran final 600-step arms:
main_qwen3_4b_qlora_slot_stability_mixed_l12_s600control_qwen3_4b_qlora_paired_no_stability_mixed_l12_s600control_qwen3_4b_qlora_answer_only_mixed_l12_s600
- Final result: trace-supervised numeric-copy compilation works; final-answer-only QLoRA stays at chance. The paired stability loss is mixed rather than decisive. It improves standard L12, paraphrase L24, and paired L24 exact execution, but it trails or ties the matched no-stability compiler on other splits and does not reliably improve paired consistency.
Figures 3
Data files 13
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
analysis/all_final_metrics.csv15 kBanalysis/final_metrics.csv7.1 kBruns/control_qwen3_4b_qlora_answer_only_mixed_l12_s600/results.json34 kBruns/control_qwen3_4b_qlora_paired_no_stability_mixed_l12_s300/results.json38 kBruns/control_qwen3_4b_qlora_paired_no_stability_mixed_l12_s600/results.json38 kBruns/main_qwen3_4b_qlora_slot_stability_mixed_l12_s300/results.json39 kBruns/main_qwen3_4b_qlora_slot_stability_mixed_l12_s600/results.json39 kBruns/smoke_qwen3_4b_qlora_stability/results.json12 kBruns/smoke_tiny_paired_batches/results.json8.3 kBruns/smoke_tiny_paired_eval/results.json8.1 kBruns/smoke_tiny_stability/results.json11 kBruns/smoke_tiny_stability_v2/results.json14 kBruns/smoke_tiny_stability_v3/results.json24 kB
Reproduce
This entry is source code or analysis only — there is no separate run to reproduce.