Qwen Hidden VM Curriculum Repair
The one idea you need
Instead of blurting an answer, the model writes a hidden worksheet of steps that a fixed calculator runs to reach it. To improve it, we hunted for any nearby worksheet that lands on the right number and drilled those in — including lucky ones that got there for the wrong reasons.
The question
If a model solves tasks by writing a hidden worksheet of steps, can you make it more reliable by teaching it nearby worksheets that happen to reach the right answer?
What we found
No. Feeding it nearby worksheets that merely land on the correct answer wrecked it. Plain step-by-step training scored 72% on the main mixed test; the same model after answer-chasing repair fell to 35%, and collapsed on longer problems, dropping to 9% at the hardest length versus 30% for plain training. The gradual length curriculum helped; the repair step actively hurt.
Why it matters
When folding verified solutions back into a model, select for correct reasoning, not just the correct final answer. A correct worksheet almost always sits nearby, so build a checker that picks it at answer time instead of retraining on patches.
On this page
Results at a glance 6
How to read
Bars show accuracy on fresh matched problems for four methods, taller is better: the model's blurted answer, the answer-chasing repaired worksheet, the plain-trained worksheet, and the best possible if a checker hand-picked worksheets.
Takeaway → The plain-trained bar (72%) towers over the repaired bar (35%), so repair roughly halved accuracy even though a correct worksheet nearly always exists nearby.
Data table
| condition | accuracy |
|---|---|
| direct logits | 10.9% |
| repair-distilled hidden VM | 35.2% |
| trace-only hidden VM (control) | 71.9% |
| verified repair ceiling | 99.6% |
Technical framing
Fresh paired accuracy: repair distillation hurts the deployable compiler — The hidden VM beats direct logits, but repair distillation loses 36.7 pp vs the trace-only control; nearby verified programs exist 99.6% of the time.
How to read
Horizontal axis is problem length (both models trained only up to length 6); vertical axis is worksheet accuracy, higher is better. Two lines compare plain training against answer-chasing repair.
Takeaway → The plain line stays high, sloping down to 30% at length 10, while the repaired line drops to near-baseline 9%.
Data table
| program length (trained up to length 6) | trace-only control | repair-distilled |
|---|---|---|
| 6 | 71.9% | 35.2% |
| 8 | 55.2% | 10.4% |
| 10 | 30.2% | 9.4% |
Technical framing
Length robustness: trace-only control degrades gracefully, repair-distilled collapses — Beyond the trained length 6, the trace-only curriculum still holds 55% and 30% at lengths 8 and 10, while the repair-distilled model falls to near-baseline.
How to read
For each task type, three bars compare the blurted answer, the hidden worksheet, and the best possible with a checker; taller is better.
Takeaway → The hidden worksheet beats the blurted answer in every task type except yes/no logic, and the checker reaches near 100% everywhere.
Data table
| domain | direct logits | hidden VM | verified repair |
|---|---|---|---|
| arithmetic | 0% | 22.7% | 97.7% |
| calendar | 18.2% | 40.9% | 100% |
| unit | 0% | 26.2% | 100% |
| list | 4.8% | 69% | 100% |
| boolean | 42.9% | 31% | 100% |
| lookup | 0% | 21.4% | 100% |
Numbers from experiments/qwen_hidden_vm_curriculum_repair/analysis/summary.md (Fresh Paired Domain Breakdown table)
Technical framing
Fresh paired accuracy by domain (repair-distilled run) — The hidden VM beats direct logits in every domain except boolean, and verified repair finds a correct nearby program almost everywhere.
How to read
Horizontal axis is training step, vertical axis is accuracy on fresh matched problems, higher is better. Both runs match until step 700, then only one gets the extra repair phase.
Takeaway → Both runs reach 72% at step 700, then the repair phase alone drags that one down to 35%.
Data table
| Training step | Repair-distilled run | Trace-only control |
|---|---|---|
| 0 | 18.4% | 18.4% |
| 700 | 71.9% | 71.9% |
| 920 | 35.2% | — |
Technical framing
Training phases: the repair-distillation step itself causes the collapse — Both runs are identical at step 700 (71.9%); the 220-step repair-distillation phase alone drops fresh paired accuracy to 35.2%.
How to read
Evaluation splits run from fresh to long-and-hard; each has two bars, the model's own guess versus the best correct worksheet a checker found nearby. Taller is better.
Takeaway → Even on the longest problems where the guess falls under 10%, a correct nearby worksheet exists 88-96% of the time.
Data table
| Evaluation split | Hidden VM (argmax) | Verified local repair |
|---|---|---|
| Fresh standard | 28.1% | 98.4% |
| Fresh paraphrase | 25% | 98.4% |
| Fresh paired | 35.2% | 99.6% |
| Hard standard (len 8) | 10.4% | 94.8% |
| Hard paraphrase (len 8) | 14.6% | 97.9% |
| Harder standard (len 10) | 9.4% | 96.4% |
| Harder paraphrase (len 10) | 8.9% | 88% |
Numbers from analysis/summary.md Final Splits table (main_curriculum_repair_s512)
Technical framing
Verified repair ceiling stays high even where the compiler collapses — Even at length 10 where the argmax compiler scores under 10%, a correct executable program sits in the local neighborhood 88-96% of the time.
How to read
Four training methods, each with two bars: the share of exactly-correct worksheets and the share whose early steps are correct. Taller is better.
Takeaway → Plain training keeps 56% exactly-correct worksheets, while answer-chasing repair cuts that to 18%.
Data table
| Run | Program exact | State prefix |
|---|---|---|
| Repair-distilled (main) | 18% | 72.1% |
| Trace-only control (main) | 55.9% | 77.3% |
| Pilot repair, keep short lengths | 45.8% | 70.8% |
| Pilot repair, length 6 only | 18.8% | 55% |
Numbers from analysis/summary.md Run Summary table
Technical framing
Program fidelity across runs: repair distillation erodes exact programs — Trace-only training keeps 55.9% exact programs; repair distillation cuts that to 18.0%, and the length-6-only pilot ablation is similarly weak.
In the author’s words from the Report · “Abstract”
This experiment tests whether a Qwen 4B model can learn a more length-robust hidden virtual-machine compiler when posttraining uses a staged length curriculum followed by verifier-guided program repair. The model emits invisible typed VM slots, a deterministic runtime executes those slots, and a local repair pass searches nearby hidden programs that verify against the known answer.
Overview
Standalone experiment for training Qwen/Qwen3-4B to compile natural-language tasks into a fixed hidden VM with a length curriculum and verifier-guided program repair.
Small files live here:
experiments/qwen_hidden_vm_curriculum_repair/Large checkpoints live separately here:
large_artifacts/qwen_hidden_vm_curriculum_repair/checkpoints/The main question is whether a Qwen-attached hidden VM compiler can become more length-robust when trained with staged program lengths and then distilled from locally verified program repairs.
Report
Rendered from reports/qwen_hidden_vm_curriculum_repair_paper.md
Abstract
This experiment tests whether a Qwen 4B model can learn a more length-robust hidden virtual-machine compiler when posttraining uses a staged length curriculum followed by verifier-guided program repair. The model emits invisible typed VM slots, a deterministic runtime executes those slots, and a local repair pass searches nearby hidden programs that verify against the known answer.
Setup
- Primary run:
main_curriculum_repair_s512 - Model:
Qwen/Qwen3-4B - Variant:
trace - Train examples:
512 - Train steps:
700 - Repair steps:
220 - VM max steps:
10 - Curriculum schedule:
4:240,6:700 - Train length range:
1to6 - Eval length:
6; hard length:8; harder length:10
The hidden VM uses typed operation slots and copied numeric arguments. Direct logits are the model's next-token numeric answer distribution at the answer marker. Hidden VM accuracy is execution of the compiled invisible program. Repair accuracy is target-aware local search around the compiled program and is reported as a verifier-assisted ceiling, not as a standalone deployable inference path.
Results
Final Splits
| Split | Direct | Hidden VM | Repair | Program exact | Repair exact | State prefix | Repair found |
|---|---|---|---|---|---|---|---|
| val_mixed | 12.5% | 20.1% | 97.9% | 4.9% | 36.8% | 68.1% | 97.9% |
| fresh_standard_mixed | 9.9% | 28.1% | 98.4% | 15.6% | 48.4% | 72.7% | 98.4% |
| fresh_paraphrase_mixed | 12.0% | 25.0% | 98.4% | 8.9% | 30.2% | 64.8% | 98.4% |
| fresh_paired_mixed | 10.9% | 35.2% | 99.6% | 18.0% | 47.3% | 72.1% | 99.6% |
| hard_standard_mixed | 10.9% | 10.4% | 94.8% | 0.0% | 4.7% | 56.0% | 94.8% |
| hard_paraphrase_mixed | 14.1% | 14.6% | 97.9% | 0.0% | 2.6% | 52.0% | 97.9% |
| harder_standard_mixed | 10.9% | 9.4% | 96.4% | 0.0% | 0.0% | 44.1% | 96.4% |
| harder_paraphrase_mixed | 9.4% | 8.9% | 88.0% | 0.0% | 0.0% | 40.6% | 88.0% |
| domain_arithmetic | 0.0% | 12.5% | 100.0% | 9.4% | 50.0% | 66.7% | 100.0% |
| domain_calendar | 9.4% | 40.6% | 100.0% | 31.2% | 50.0% | 78.1% | 100.0% |
| domain_unit | 3.1% | 9.4% | 100.0% | 6.2% | 50.0% | 69.8% | 100.0% |
| domain_list | 3.1% | 53.1% | 100.0% | 0.0% | 6.2% | 69.3% | 100.0% |
| domain_boolean | 40.6% | 37.5% | 100.0% | 21.9% | 50.0% | 68.2% | 100.0% |
| domain_lookup | 0.0% | 12.5% | 84.4% | 3.1% | 18.8% | 61.5% | 84.4% |
Domain Breakdown
| Domain | n | Direct | Hidden VM | Repair |
|---|---|---|---|---|
| arithmetic | 44.00 | 0.0% | 22.7% | 97.7% |
| calendar | 44.00 | 18.2% | 40.9% | 100.0% |
| unit | 42.00 | 0.0% | 26.2% | 100.0% |
| list | 42.00 | 4.8% | 69.0% | 100.0% |
| boolean | 42.00 | 42.9% | 31.0% | 100.0% |
| lookup | 42.00 | 0.0% | 21.4% | 100.0% |
Training Dynamics
Fresh paired hidden VM accuracy moved from 18.4% at initialization to 35.2% after the full treatment. Verified local repair on the same split reaches 99.6%. The matched trace-only control scores 71.9% on fresh paired, 55.2% on hard length 8, and 30.2% on harder length 10.
Repair Target Quality
The repair-target pass built 512.00 training targets from 512.00 source examples. Verified repairs were found for 100.0% of source examples, changed-program repairs for 11.3%, with an average of 151.44 local candidates per source example.
Run Summary
| Run | Variant | Direct | Hidden VM | Repair | Program exact | State prefix |
|---|---|---|---|---|---|---|
| main_curriculum_repair_s512 | trace | 10.9% | 35.2% | 99.6% | 18.0% | 72.1% |
| main_curriculum_trace_s512 | trace | 10.9% | 71.9% | 98.8% | 55.9% | 77.3% |
| pilot_curriculum_repair_keep_l6_s256 | trace | 12.5% | 61.5% | 100.0% | 45.8% | 70.8% |
| pilot_curriculum_repair_l6_s256 | trace | 13.5% | 45.8% | 100.0% | 18.8% | 55.0% |
Interpretation
The primary measurement is fresh paired mixed-domain accuracy. Direct logits score 10.9%, while the repair-distilled hidden VM scores 35.2% (+24.2 pp). The matched trace-only curriculum control scores 71.9%, so the repair-distillation treatment changes fresh paired accuracy by -36.7 pp. Verified local repair scores 99.6%, showing that nearby executable headroom remains high even when the argmax compiler is poor.
Program-exact accuracy after repair distillation is 18.0% and state-prefix accuracy is 72.1%. The trace-only control is the best deployable model in this experiment, not the repair-distilled treatment. The repair target pass found verified targets for 100.0% of source examples, but only 11.3% were changed-program repairs; the subsequent no-selection repair phase moved the compiler away from its stable token-local policy.
The hard-length splits are the decisive stress test. The trace-only control trains up to length 6 and reaches 55.2% at length 8 and 30.2% at length 10. The repair-distilled run falls to 10.4% and 9.4% on those same splits.
Decision
This experiment should be read as a split result. The length curriculum is useful and should be retained. Target-aware local repair is also highly informative: it shows that the model's top-k neighborhood usually contains a correct executable program. The failed part is naive repair distillation from final-answer-verified programs. The next experiment should not simply add more repair steps; it should either distill canonical traces selected by a state-aware verifier, or train a verifier/reranker for inference-time candidate selection before trying to fold repairs back into the compiler.
Limitations
- The domains are synthetic and deterministic.
- Answers are integers in a bounded value vocabulary.
- Trace supervision supplies exact hidden programs during the curriculum phase.
- Repair accuracy is target-aware and should be read as verifier-assisted headroom.
- The runtime is fixed and hand-designed.
- This is one primary run unless additional runs are added.
Artifacts
Small experiment files live in:
experiments/qwen_hidden_vm_curriculum_repair/Large artifacts live in:
large_artifacts/qwen_hidden_vm_curriculum_repair/checkpoints/Primary files:
analysis/summary.mdanalysis/final_metrics.csvanalysis/all_final_metrics.csvanalysis/figures/split_accuracy.pnganalysis/figures/domain_accuracy.pnganalysis/figures/training_curve.pnganalysis/figures/run_summary.pngruns/main_curriculum_repair_s512/metrics.csvruns/main_curriculum_repair_s512/train_log.csvreports/qwen_hidden_vm_curriculum_repair_paper.mdreports/qwen_hidden_vm_curriculum_repair_paper.htmlcheckpoint_manifest.csv
Experiment log 1
Show the running log (1 entry, 2026-06-23)
2026-06-23
Created a standalone hidden VM curriculum-repair experiment.
Design:
- Use the same typed hidden VM families inside this experiment: arithmetic chains, calendar shifts, unit transforms, list aggregation, boolean thresholding, and lookup/adjust rules.
- Train
Qwen/Qwen3-4Bwith QLoRA and a hidden compiler head. - Use a staged length curriculum so the model first sees shorter programs, then length-6 programs.
- Evaluate fresh standard, paraphrase, paired, length-8 hard, and length-10 harder splits.
- Add a verifier-guided local repair pass that edits predicted hidden programs and keeps candidates whose deterministic execution matches the known answer.
- Distill verified repair targets back into the compiler with trace-style value losses, while avoiding stale token-position losses for repaired slots.
Initial planned runs:
smoke_curriculum_repair: tiny end-to-end check.pilot_curriculum_repair_l6: small run to verify curriculum and repair target construction.main_curriculum_trace_s512: trace/curriculum control without repair distillation.main_curriculum_repair_s512: trace/curriculum plus verifier-guided repair distillation.
Smoke run:
python experiments/qwen_hidden_vm_curriculum_repair/src/qwen_hidden_vm_curriculum_repair_experiment.py \
--run_name smoke_curriculum_repair --variant trace \
--train_examples 12 --val_examples 6 --eval_examples 6 --eval_pairs 4 --domain_eval_examples 2 \
--train_steps 2 --repair_steps 1 --train_batch_size 1 --eval_batch_size 2 \
--max_steps 4 --train_max_len 3 --eval_length 3 --hard_length 4 --harder_length 0 --max_length 384 \
--curriculum_schedule 2:1,3:2 --repair_source_examples 12 --repair_train_topk 2 --repair_eval_topk 2 --repair_max_edits 1 \
--log_every 1 --seed 101 --eval_seed 101001Smoke result:
- Completed successfully in 16.8 seconds.
- Verified local repair metrics were produced on every eval split.
- The repair-target phase built 4 targets from 12 source examples, with verified repairs found for 50.0% of source examples and changed-program repairs for 33.3%.
- Checkpoint and run CSVs were written under the new standalone artifact layout.
Pilot 1:
python experiments/qwen_hidden_vm_curriculum_repair/src/qwen_hidden_vm_curriculum_repair_experiment.py \
--run_name pilot_curriculum_repair_l6_s256 --variant trace \
--train_examples 256 --val_examples 72 --eval_examples 72 --eval_pairs 48 --domain_eval_examples 16 \
--train_steps 260 --repair_steps 120 --train_batch_size 2 --eval_batch_size 4 \
--max_steps 10 --train_max_len 6 --eval_length 6 --hard_length 8 --harder_length 10 --max_length 768 \
--curriculum_schedule 4:120,6:260 --repair_source_examples 256 --repair_train_topk 3 --repair_eval_topk 3 --repair_max_edits 2 \
--lr 5e-5 --executor_loss_weight 0.2 --state_loss_weight 0.05 \
--log_every 40 --seed 102 --eval_seed 102001Pilot 1 result:
- Completed in 301.5 seconds.
- Fresh paired hidden-VM executor accuracy: 45.8%; verified local repair: 100.0%.
- Hard length-8 standard executor accuracy: 23.6%; verified local repair: 100.0%.
- Harder length-10 standard executor accuracy: 19.4%; verified local repair: 98.6%.
- The repair target builder found verified candidates for 99.6% of source examples, but
repair_onlywithout unchanged programs produced only 86 training targets from 256 source examples. - Interpretation: local repair headroom is extremely high, but changed-only repair distillation is too narrow and does not by itself solve the argmax compiler.
Pilot 2 change:
- Include unchanged verified programs in the repair target set.
- Add an eval at the transition point before repair distillation, so the repair phase can be measured rather than inferred.
Pilot 2:
python experiments/qwen_hidden_vm_curriculum_repair/src/qwen_hidden_vm_curriculum_repair_experiment.py \
--run_name pilot_curriculum_repair_keep_l6_s256 --variant trace \
--train_examples 256 --val_examples 72 --eval_examples 72 --eval_pairs 48 --domain_eval_examples 16 \
--train_steps 260 --repair_steps 120 --train_batch_size 2 --eval_batch_size 4 \
--max_steps 10 --train_max_len 6 --eval_length 6 --hard_length 8 --harder_length 10 --max_length 768 \
--curriculum_schedule 4:120,6:260 --repair_source_examples 256 --repair_train_topk 3 --repair_eval_topk 3 --repair_max_edits 2 --repair_include_unchanged \
--lr 5e-5 --executor_loss_weight 0.2 --state_loss_weight 0.05 \
--log_every 40 --eval_every 260 --seed 103 --eval_seed 103001Pilot 2 result:
- Completed in 375.0 seconds.
- Transition-point fresh paired executor accuracy before repair distillation: 44.8%.
- Final fresh paired executor accuracy after repair distillation: 61.5%.
- Transition-point hard length-8 standard executor accuracy: 36.1%; final: 62.5%.
- Transition-point harder length-10 standard executor accuracy: 11.1%; final: 44.4%.
- Repair target builder produced 253 targets from 256 source examples, with verified repairs found for 98.8% and changed-program repairs for 26.6%.
- Interpretation: retaining unchanged verified programs makes repair distillation dense enough to improve the argmax compiler. Use this construction for the main repair run.
Main run plan:
main_curriculum_trace_s512: trace/curriculum control, no repair distillation.main_curriculum_repair_s512: same trace/curriculum prefix and seed, then repair distillation with unchanged verified programs included.
Main trace control:
python experiments/qwen_hidden_vm_curriculum_repair/src/qwen_hidden_vm_curriculum_repair_experiment.py \
--run_name main_curriculum_trace_s512 --variant trace \
--train_examples 512 --val_examples 144 --eval_examples 192 --eval_pairs 128 --domain_eval_examples 32 \
--train_steps 700 --repair_steps 0 --train_batch_size 2 --eval_batch_size 8 \
--max_steps 10 --train_max_len 6 --eval_length 6 --hard_length 8 --harder_length 10 --max_length 768 \
--curriculum_schedule 4:240,6:700 --repair_source_examples 512 --repair_train_topk 3 --repair_eval_topk 3 --repair_max_edits 2 --repair_include_unchanged \
--lr 5e-5 --executor_loss_weight 0.2 --state_loss_weight 0.05 \
--log_every 70 --eval_every 700 --seed 104 --eval_seed 104001Main trace result:
- Completed in 605.8 seconds.
- Fresh paired executor accuracy: 71.9%; verified local repair: 98.8%.
- Hard length-8 standard executor accuracy: 55.2%; verified local repair: 100.0%.
- Hard length-8 paraphrase executor accuracy: 41.1%; verified local repair: 99.0%.
- Harder length-10 standard executor accuracy: 30.2%; verified local repair: 99.5%.
- Harder length-10 paraphrase executor accuracy: 16.7%; verified local repair: 99.0%.
Main repair-distillation run:
python experiments/qwen_hidden_vm_curriculum_repair/src/qwen_hidden_vm_curriculum_repair_experiment.py \
--run_name main_curriculum_repair_s512 --variant trace \
--train_examples 512 --val_examples 144 --eval_examples 192 --eval_pairs 128 --domain_eval_examples 32 \
--train_steps 700 --repair_steps 220 --train_batch_size 2 --eval_batch_size 8 \
--max_steps 10 --train_max_len 6 --eval_length 6 --hard_length 8 --harder_length 10 --max_length 768 \
--curriculum_schedule 4:240,6:700 --repair_source_examples 512 --repair_train_topk 3 --repair_eval_topk 3 --repair_max_edits 2 --repair_include_unchanged \
--lr 5e-5 --executor_loss_weight 0.2 --state_loss_weight 0.05 \
--log_every 70 --eval_every 700 --seed 104 --eval_seed 104001Main repair result:
- Completed in 809.1 seconds.
- The curriculum prefix exactly matched the trace control at step 700: fresh paired 71.9%, hard length-8 standard 55.2%, harder length-10 standard 30.2%.
- Repair target builder produced 512 targets from 512 examples, with verified targets found for 100.0% and changed-program repairs for only 11.3%.
- After 220 repair-distillation steps, fresh paired executor accuracy fell to 35.2%.
- Hard length-8 standard fell to 10.4%; hard length-8 paraphrase fell to 14.6%.
- Harder length-10 standard fell to 9.4%; harder length-10 paraphrase fell to 8.9%.
- Verified local repair remained high at evaluation time, but the learned argmax compiler was damaged.
Main interpretation:
The curriculum itself is a positive result: training on lengths 1-6 produces nontrivial hard length-8/10 transfer in the trace-only control. Target-aware local repair reveals enormous headroom, often near 99-100%. But naive repair distillation from final-answer-verified programs is unsafe at scale: because many verified programs are non-canonical and the repair phase disables token-position selection losses, it can move the compiler away from the stable prompt-to-slot policy learned by trace supervision.
Reports to generate:
analysis/summary.mdreports/qwen_hidden_vm_curriculum_repair_paper.mdreports/qwen_hidden_vm_curriculum_repair_paper.html- figures under
analysis/figures/ checkpoint_manifest.csv
Figures 4
Data files 12
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
analysis/all_final_metrics.csv26 kBanalysis/final_metrics.csv6.4 kBruns/main_curriculum_repair_s512/metrics.csv6.4 kBruns/main_curriculum_repair_s512/results.json114 kBruns/main_curriculum_trace_s512/metrics.csv6.4 kBruns/main_curriculum_trace_s512/results.json84 kBruns/pilot_curriculum_repair_keep_l6_s256/metrics.csv6.9 kBruns/pilot_curriculum_repair_keep_l6_s256/results.json113 kBruns/pilot_curriculum_repair_l6_s256/metrics.csv6.9 kBruns/pilot_curriculum_repair_l6_s256/results.json57 kBruns/smoke_curriculum_repair/metrics.csv4.6 kBruns/smoke_curriculum_repair/results.json41 kB
Reproduce
This entry is source code or analysis only — there is no separate run to reproduce.