Qwen Trace Bootstrap Retention Experiment
The one idea you need
The model turns each word problem into a tiny program — add 17, multiply by 3 — that a fixed calculator then runs. Show it the correct programs first; afterward, grading only the final answer keeps it writing correct programs.
The question
Once a small model learns to turn a problem into step-by-step instructions, can you keep training it on the final answer alone without breaking that skill?
What we found
Yes. Once step-by-step labels install the skill, training on final answers alone preserves and even sharpens it: 97% of the longest 24-step problems solved exactly, versus about 1 in 100 — no better than guessing — when the same setup learns from final answers only from the start. Final-answer feedback can refine an installed skill but never discovers it.
Why it matters
Pay for step-level labels once to install a structured-reasoning skill in a small model. Afterward, cheaper final-answer feedback is enough to keep it and stretch it to problems three times longer than anything it trained on.
On this page
Results at a glance 3
How to read
Each bar is one training recipe; height is the share of the longest 24-step problems solved exactly, so taller is better. The three recipes that first showed worked steps tower over the two flat, answer-only bars.
Takeaway → The three recipes that started with worked steps reach near the top, while every answer-only recipe sits almost flat on the floor, no better than guessing.
Data table
| training recipe | exact execution @ length 24 |
|---|---|
| direct answer head | 0.4% |
| answer-only compiler | 0.8% |
| trace throughout | 96.1% |
| trace then answer-only | 96.9% |
| trace then answer-only (low LR) | 95.7% |
Numbers from experiments/qwen_trace_bootstrap_retention/analysis/final_metrics.csv (main_qwen35_retention, len24)
Technical framing
Held-out length-24 execution accuracy by training recipe — Answer-only training preserves a trace-installed program interface (96.9%) but cannot discover it from scratch (0.8%).
How to read
The horizontal axis is problem length in steps; the vertical axis is share solved exactly, so higher lines are better. Worked-step recipes hold near the ceiling even at the longest problems, while answer-only lines stay flat on the floor.
Takeaway → Worked-step lines stay near the top even at 24 steps, far past their training lengths, while answer-only lines never lift off the floor at any length.
Data table
| program length (steps) | direct answer head | answer-only compiler | trace throughout | trace then answer-only | trace then answer-only (low LR) |
|---|---|---|---|---|---|
| 4 | 0.8% | 0.8% | 100% | 100% | 100% |
| 8 | 0.8% | 1.2% | 100% | 100% | 100% |
| 12 | 1.2% | 1.2% | 99.2% | 99.6% | 99.2% |
| 24 | 0.4% | 0.8% | 96.1% | 96.9% | 95.7% |
Numbers from experiments/qwen_trace_bootstrap_retention/analysis/final_metrics.csv (main_qwen35_retention)
Technical framing
Execution accuracy vs held-out program length — Trace-bootstrapped compilers generalize far past training lengths (1-8 steps); answer-only baselines stay at 97-way chance.
How to read
The horizontal axis is training progress, with worked-step coaching removed after the halfway point; the vertical axis is accuracy on the longest problems, so higher is better. One line dips sharply at the switch, another barely moves, and both climb back up.
Takeaway → The full-speed line dips right after coaching stops but recovers to finish highest, showing the model keeps and even sharpens the method on final answers alone.
Data table
| training step (trace loss removed after step 800) | trace throughout | trace then answer-only | trace then answer-only (low LR) |
|---|---|---|---|
| 800 | 85.2% | 87.1% | 87.1% |
| 801 | — | 75.4% | 89.8% |
| 1600 | 96.1% | 96.9% | 95.7% |
Numbers from report table (reports/qwen_trace_bootstrap_retention_paper.md, Retention Dynamics)
Technical framing
Retention dynamics after trace loss is removed (length-24 accuracy by step) — Removing trace loss causes an immediate long-chain dip at normal LR, but answer-only training then recovers and finishes highest.
In the author’s words from the Report · “Abstract”
This experiment tests whether a structured latent program interface can be installed with symbol-trace supervision and then retained when training continues using only final-answer supervision. A frozen Qwen3.5-4B encoder provides hidden states from modular arithmetic prompts. A small compiler reads those hidden states, emits an initial value and per-step program symbols, and a latent executor computes the final answer. The main result is positive. After trace bootstrap on 1-4 step programs, answer-only continuation on 1-8 step programs reaches 96.9% exact execution at held-out length 24. A compiler trained from final answers only stays at 0.8% at length 24, and a direct answer head stays at 0.4%. … Read the full result →
Overview
This experiment tests whether a frozen Qwen encoder attached to a structured latent executor can keep using a learned program interface after symbol-trace supervision is removed.
Contents
src/qwen_trace_bootstrap_retention_experiment.py: task generator, Qwen feature extraction, staged bridge training, executor evaluation, and checkpointing.src/analyze_qwen_trace_bootstrap_retention.py: analysis tables and figures.reports/qwen_trace_bootstrap_retention_experiment_log.md: chronological experiment log.reports/qwen_trace_bootstrap_retention_paper.md: standalone written report.reports/qwen_trace_bootstrap_retention_paper.html: standalone HTML report.runs/: lightweight JSON and CSV run outputs.analysis/: generated summaries and figures.checkpoint_manifest.csv: saved checkpoint paths and sizes.
Large Files
Trainable bridge checkpoints are stored outside this directory under:
../../large_artifacts/qwen_trace_bootstrap_retention/checkpoints/Download this experiment directory for the research bundle. Download the large artifact directory only when saved model weights are needed.
Report
Rendered from reports/qwen_trace_bootstrap_retention_paper.md
Abstract
This experiment tests whether a structured latent program interface can be installed with symbol-trace supervision and then retained when training continues using only final-answer supervision. A frozen Qwen3.5-4B encoder provides hidden states from modular arithmetic prompts. A small compiler reads those hidden states, emits an initial value and per-step program symbols, and a latent executor computes the final answer.
The main result is positive. After trace bootstrap on 1-4 step programs, answer-only continuation on 1-8 step programs reaches 96.9% exact execution at held-out length 24. A compiler trained from final answers only stays at 0.8% at length 24, and a direct answer head stays at 0.4%. The result shows that final-answer training can preserve and refine an already-installed latent program interface, but does not discover that interface from scratch under this setup.
Task
Each prompt describes a value x modulo 97 and a sequence of operations:
Initial x = 42.
Step: add 17.
Step: multiply by 3.
Step: subtract 5.The target is the final value of x. The executor supports three operations:
ADD a: setx = x + a (mod 97).SUB a: setx = x - a (mod 97).MUL a: setx = x * a (mod 97).
Bootstrap training uses 1-4 step programs. Answer-only continuation uses 1-8 step programs. Evaluation uses held-out lengths 4, 8, 12, and 24.
Model
Qwen3.5-4B is loaded in 4-bit and kept frozen. The trainable bridge reads selected hidden states:
- the numeric token span for the initial value,
- the operation line prefix for each step,
- the numeric token span for each operation argument,
- the answer line for the direct-answer control.
The compiler predicts initial value logits over 97 residues, operation logits over ADD, SUB, and MUL, and argument logits over 97 residues. During training, a differentiable executor composes the predicted distributions. For strict evaluation, the compiler output is argmaxed and executed exactly.
Variants
direct: direct answer classifier from frozen Qwen features.compiler_answer_only: latent compiler trained from final answer loss only.compiler_trace: latent compiler trained with trace supervision throughout.compiler_trace_then_answer: trace bootstrap followed by answer-only continuation.compiler_trace_then_answer_low_lr: same schedule with a lower learning rate during answer-only continuation.
Results
The main run uses 1024 bootstrap examples and 1024 answer-continuation examples. The answer-continuation stage removes all symbol-trace losses.
| Variant | L=4 exec | L=8 exec | L=12 exec | L=24 exec | L=24 mass | L=24 init | L=24 op | L=24 arg | L=24 program exact |
|---|---|---|---|---|---|---|---|---|---|
direct | 0.8% | 0.8% | 1.2% | 0.4% | n/a | n/a | n/a | n/a | n/a |
compiler_trace | 100.0% | 100.0% | 99.2% | 96.1% | 94.2% | 100.0% | 99.8% | 100.0% | 96.1% |
compiler_answer_only | 0.8% | 1.2% | 1.2% | 0.8% | 1.0% | 0.8% | 33.7% | 0.0% | 0.0% |
compiler_trace_then_answer | 100.0% | 100.0% | 99.6% | 96.9% | 95.3% | 100.0% | 99.9% | 100.0% | 96.9% |
compiler_trace_then_answer_low_lr | 100.0% | 100.0% | 99.2% | 95.7% | 92.8% | 100.0% | 99.8% | 100.0% | 95.7% |
Retention Dynamics
The staged log shows what happens immediately after trace loss is removed.
| Variant | Stage | Step | L=24 exec | L=24 mass | L=24 op | L=24 arg |
|---|---|---|---|---|---|---|
compiler_trace | trace bootstrap | 800 | 85.2% | 82.4% | 99.3% | 100.0% |
compiler_trace | trace continuation | 1600 | 96.1% | 94.2% | 99.8% | 100.0% |
compiler_trace_then_answer | trace bootstrap | 800 | 87.1% | 83.5% | 99.4% | 100.0% |
compiler_trace_then_answer | answer retention | 801 | 75.4% | 71.5% | 99.0% | 100.0% |
compiler_trace_then_answer | answer retention | 1600 | 96.9% | 95.3% | 99.9% | 100.0% |
compiler_trace_then_answer_low_lr | trace bootstrap | 800 | 87.1% | 83.5% | 99.4% | 100.0% |
compiler_trace_then_answer_low_lr | answer retention | 801 | 89.8% | 85.5% | 99.6% | 100.0% |
compiler_trace_then_answer_low_lr | answer retention | 1600 | 95.7% | 92.8% | 99.8% | 100.0% |
Normal-rate answer-only continuation causes an immediate long-chain drop, but then recovers and finishes slightly above trace-throughout training at length 24. Low-rate continuation avoids the immediate drop and also retains the interface, but finishes slightly lower.
Interpretation
The central result is that final-answer supervision can maintain and refine a latent executor interface once trace supervision has installed it. This is different from discovering the interface from scratch: the answer-only compiler does not learn initial values, arguments, or executable programs.
The comparison is also not explained by direct answer learning. The direct answer head uses the same frozen Qwen features and remains at 97-way chance. The gain comes from decomposing the task into compiled symbols and executing those symbols with a fixed latent runtime.
At length 24, initial value and argument accuracy are exact in the successful rows. Remaining errors are rare operation mistakes compounded over long programs.
Limitations
The task is synthetic modular arithmetic with standardized text templates. The executor operation set is fixed in advance. The successful recipe uses direct symbol traces during bootstrap. Qwen is frozen, so this experiment tests an attached runtime and bridge rather than full language-model posttraining.
The result does not show broad intelligence improvement. It shows a concrete training recipe: install a latent program interface with traces, then continue training through final-answer supervision while preserving that interface.
Artifact Layout
Lightweight code, metrics, figures, and reports live in:
experiments/qwen_trace_bootstrap_retention/Saved bridge checkpoints live separately in:
large_artifacts/qwen_trace_bootstrap_retention/checkpoints/The checkpoint manifest is:
experiments/qwen_trace_bootstrap_retention/checkpoint_manifest.csvExperiment log 9
Show the running log (9 entries, 2026-06-21)
Objective
Test whether a structured latent program interface can be installed with symbol-trace supervision and then retained when training continues with only final-answer supervision.
Primary Questions
- Does trace-supervised bootstrap train an executable latent compiler?
- After trace loss is removed, does final-answer training preserve the compiled program interface?
- Can answer-only continuation on longer chains improve length generalization without destroying symbol accuracy?
- Does an answer-only compiler trained from scratch discover the same interface?
- Which retained-interface failure dominates: initial value, operation, argument, or accumulated program exactness?
Metrics
executor_accuracy: accuracy after argmax compilation and exact latent execution.executor_target_mass: differentiable executor probability assigned to the target answer.init_accuracy: compiled initial value accuracy.op_accuracy: per-step operation accuracy.arg_accuracy: per-step argument accuracy.program_exact: fraction of examples with all compiled symbols correct.direct_accuracy: direct answer classifier accuracy from the same frozen Qwen features.
Artifact Layout
- Code and lightweight outputs:
experiments/qwen_trace_bootstrap_retention/ - Checkpoints:
large_artifacts/qwen_trace_bootstrap_retention/checkpoints/ - Run outputs:
experiments/qwen_trace_bootstrap_retention/runs/<variant>/ - Analysis outputs:
experiments/qwen_trace_bootstrap_retention/analysis/
2026-06-21 Setup
Created the standalone experiment directory:
experiments/qwen_trace_bootstrap_retention/src/experiments/qwen_trace_bootstrap_retention/reports/experiments/qwen_trace_bootstrap_retention/runs/experiments/qwen_trace_bootstrap_retention/analysis/figures/large_artifacts/qwen_trace_bootstrap_retention/checkpoints/
Next action: adapt the bridge harness for staged trace-bootstrap and answer-only retention training.
2026-06-21 Harness Smoke
Implemented the staged harness:
- Separate bootstrap and answer-continuation training datasets.
Matched variants:
direct: direct answer classifier on frozen Qwen features.compiler_answer_only: latent compiler trained from final answer only.compiler_trace: latent compiler with trace supervision throughout.compiler_trace_then_answer: trace bootstrap followed by answer-only
retention.
- Stage-aware training logs with
trace_loss_active. - External checkpoint writing.
- Analysis summary and figures.
Verification:
- Source compilation passed.
- Stale path scan over the new source and top-level docs passed.
- Tiny random Llama smoke completed.
- Analysis generation completed.
- Checkpoints were written under
large_artifacts/qwen_trace_bootstrap_retention/checkpoints/smoke_tiny/.
Smoke interpretation:
- The smoke validates staged training mechanics, checkpointing, and analysis.
- The tiny random model has no meaningful parsing signal, so its accuracy is only a plumbing check.
Next action: run a small Qwen pilot to estimate whether answer-only continuation preserves a trace-installed program interface.
2026-06-21 Qwen Pilot
Ran pilot_qwen35_retention with frozen Qwen3.5-4B features, bootstrap training lengths 1-3, answer-continuation training lengths 1-6, and evaluation lengths 3, 6, and 12.
| Variant | L=3 exec | L=6 exec | L=12 exec | L=12 init | L=12 op | L=12 arg | L=12 program exact |
|---|---|---|---|---|---|---|---|
direct | 1.6% | 0.0% | 1.6% | n/a | n/a | n/a | n/a |
compiler_trace | 100.0% | 98.4% | 92.2% | 93.8% | 99.7% | 100.0% | 92.2% |
compiler_answer_only | 0.0% | 3.1% | 1.6% | 0.0% | 35.5% | 1.0% | 0.0% |
compiler_trace_then_answer | 81.2% | 71.9% | 56.2% | 89.1% | 99.1% | 97.9% | 56.2% |
Pilot interpretation:
- Trace supervision installs a strong executable interface.
- Answer-only from scratch fails to discover the interface.
- Answer-only continuation retains a usable interface but degrades relative to keeping trace supervision, especially on length 12.
- In the retention variant, operation and argument parsing remain high; the largest degradation is program exactness through accumulated symbol errors.
Added compiler_trace_then_answer_low_lr, where the answer-only retention stage uses a lower learning rate while sharing initialization and bootstrap sampling with the normal retention variant.
Ran pilot_qwen35_retention_lr with the two retention variants:
| Variant | L=3 exec | L=6 exec | L=12 exec | L=12 init | L=12 op | L=12 arg | L=12 program exact |
|---|---|---|---|---|---|---|---|
compiler_trace_then_answer | 76.6% | 78.1% | 56.2% | 89.1% | 99.1% | 97.7% | 56.2% |
compiler_trace_then_answer_low_lr | 79.7% | 73.4% | 59.4% | 89.1% | 97.4% | 98.8% | 57.8% |
Pilot decision:
- Promote
compiler_trace_then_answer_low_lrto the main run, while retaining the normal-LR row as a comparison. - Use a larger bootstrap set and longer bootstrap to ensure the interface is well installed before removing trace loss.
- Evaluate out to length 24.
2026-06-21 Main Qwen Run
Ran main_qwen35_retention with frozen Qwen3.5-4B features, 1024 bootstrap examples, 1024 answer-continuation examples, bootstrap training lengths 1-4, answer-continuation training lengths 1-8, and evaluation lengths 4, 8, 12, and 24.
Final metrics:
| Variant | L=4 exec | L=8 exec | L=12 exec | L=24 exec | L=24 mass | L=24 init | L=24 op | L=24 arg | L=24 program exact |
|---|---|---|---|---|---|---|---|---|---|
direct | 0.8% | 0.8% | 1.2% | 0.4% | n/a | n/a | n/a | n/a | n/a |
compiler_trace | 100.0% | 100.0% | 99.2% | 96.1% | 94.2% | 100.0% | 99.8% | 100.0% | 96.1% |
compiler_answer_only | 0.8% | 1.2% | 1.2% | 0.8% | 1.0% | 0.8% | 33.7% | 0.0% | 0.0% |
compiler_trace_then_answer | 100.0% | 100.0% | 99.6% | 96.9% | 95.3% | 100.0% | 99.9% | 100.0% | 96.9% |
compiler_trace_then_answer_low_lr | 100.0% | 100.0% | 99.2% | 95.7% | 92.8% | 100.0% | 99.8% | 100.0% | 95.7% |
Length-24 stage trajectory:
| Variant | Stage | Step | L=24 exec | L=24 mass | L=24 op | L=24 arg |
|---|---|---|---|---|---|---|
compiler_trace | trace bootstrap | 800 | 85.2% | 82.4% | 99.3% | 100.0% |
compiler_trace | trace continuation | 1600 | 96.1% | 94.2% | 99.8% | 100.0% |
compiler_trace_then_answer | trace bootstrap | 800 | 87.1% | 83.5% | 99.4% | 100.0% |
compiler_trace_then_answer | answer retention | 801 | 75.4% | 71.5% | 99.0% | 100.0% |
compiler_trace_then_answer | answer retention | 1600 | 96.9% | 95.3% | 99.9% | 100.0% |
compiler_trace_then_answer_low_lr | trace bootstrap | 800 | 87.1% | 83.5% | 99.4% | 100.0% |
compiler_trace_then_answer_low_lr | answer retention | 801 | 89.8% | 85.5% | 99.6% | 100.0% |
compiler_trace_then_answer_low_lr | answer retention | 1600 | 95.7% | 92.8% | 99.8% | 100.0% |
Main interpretation:
- Trace bootstrap installs an executable interface that generalizes to long chains before trace loss is removed.
- Answer-only continuation can preserve and improve the interface when it starts from the trace-installed state. The normal-LR retention row reaches 96.9% exact execution at length 24.
- Answer-only training from scratch remains at chance, confirming that the interface is not discovered from sparse final-answer supervision alone under this setup.
- The low-LR retention row avoids the immediate post-removal drop, but the normal-LR row recovers and finishes slightly higher.
- Long-chain errors continue to track rare operation mistakes; initial value and argument parsing are exact at length 24.
Next action: write the standalone report, checkpoint manifest, and final audit.
2026-06-21 Final Audit
Final artifacts created:
reports/qwen_trace_bootstrap_retention_paper.mdreports/qwen_trace_bootstrap_retention_paper.htmlcheckpoint_manifest.csv
Verification:
- Source compilation passed:
python -m py_compile src/qwen_trace_bootstrap_retention_experiment.py src/analyze_qwen_trace_bootstrap_retention.py - Checkpoint manifest validation passed for 15 saved checkpoints.
- Markdown and HTML report image references resolve.
- No
.pt,.pth, or.ckptfiles are stored inside the lightweight experiment directory. - Standalone wording scan passed on the report files.
- Removed the compile cache after verification.
Artifact sizes:
experiments/qwen_trace_bootstrap_retention/: 572Klarge_artifacts/qwen_trace_bootstrap_retention/: 130M
Conclusion:
The answer-only retention recipe succeeds after trace bootstrap. The main compiler_trace_then_answer row reaches 96.9% exact execution at length 24, while compiler_answer_only from scratch remains at 0.8% and the direct answer head remains at 0.4%. This shows that final-answer supervision can preserve and refine an installed latent program interface, but does not discover that interface from scratch under this setup.
Figures 3
Data files 5
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
analysis/final_metrics.csv6.0 kBruns/main_qwen35_retention/results.json60 kBruns/pilot_qwen35_retention/results.json37 kBruns/pilot_qwen35_retention_lr/results.json22 kBruns/smoke_tiny/results.json36 kB
Reproduce
This entry is source code or analysis only — there is no separate run to reproduce.