Qwen Iterative Repair Policy

Finished2026-06-23imported · line YPosttraining and Adaptation Process Control and Tool Use Structured Execution and Compilers GitHub ↗

The one idea you need

A small model turns each prompt into a 24-step arithmetic recipe, but usually botches a step or two. A separate checker reads what each near-miss recipe actually computes, then swaps in one better step at a time — targeted fixes, never full rewrites.

The question

When a small model turns a prompt into a step-by-step arithmetic program but gets a few steps wrong, can a tiny helper repair it by editing single steps?

What we found

Yes. The frozen model alone got about 30% of programs exactly right; editing one step at a time lifted that to 53% on fresh problems, closing roughly 38% of the distance to the best a perfect fixer could reach (89%). The first edit did most of the work, and it slightly beat simply picking the single best candidate outright — with no written reasoning generated.

Why it matters

When a small model outputs a runnable program, don't force it to rewrite its reasoning in words. Score and edit the program directly against what it computes — cheap targeted edits rescued many near-misses without generating any extra full attempts.

Programs exactly right, before and after30% → 53%frozen model alone vs after editing single steps

Best a perfect fixer could reach89%ceiling if the ideal repair were always chosen

Share of the fixable gap closedabout 38%of the distance from the model alone up to that ceiling

Correct after just one edit47%first edit does most of the work; a second adds a few points

On this page

Results at a glance
Overview
Report
Experiment log
Figures
Data files
Reproduce
Related

Results at a glance 2

Editing steps one at a time nearly doubles fully-correct programs

How to read

Four bars on fresh problems; bar height is how often the whole step-by-step recipe runs exactly right (taller is better). Left to right: the model alone, a fixer that picks the single best candidate in one shot, the one-step-at-a-time fixer, then the best a perfect fixer could reach.

Takeaway → The one-step fixer bar rises far above the model-alone bar, roughly doubling correct programs, and edges just past the pick-the-best-in-one-shot fixer.

Data table

condition	exact execution accuracy
base compiler	30.1%
unconstrained learned scorer	52%
iterative repair (K=2)	52.7%
local oracle ceiling	89.1%

Numbers from experiments/qwen_iterative_repair_policy/analysis/fresh_main_summary.csv

Technical framing

Learned iterative repair converts near-miss compiled programs into exact ones (fresh paired prompts) — Iterative one-edit-at-a-time repair lifts fresh paired accuracy from 30.1% to 52.7%, recovering 38% of the gap to the local oracle.

Almost all the gain comes from the very first repair

How to read

Lines show how often the recipe runs exactly right as more repair rounds are allowed, from zero rounds up to three (higher is better); each line is a different fresh problem set. Rising left-to-right means each added round helps.

Takeaway → Every line jumps sharply after the first round then flattens, showing one fix does most of the work and a third round adds nothing.

Data table

repair iterations K	fresh paired	fresh paraphrase	fresh standard
0	30.1%	29.7%	29.7%
1	47.3%	48.4%	43.2%
2	52.7%	51.6%	44.8%
3	52.7%	51%	44.8%

Numbers from experiments/qwen_iterative_repair_policy/analysis/main_final_metrics.csv

Technical framing

Repair iterations K: most of the gain arrives by K=1, saturates at K=2 — One repair step gives the bulk of the improvement; K=3 adds nothing because candidates are capped at two edits from the base program.

In the author’s words from the Report · “Abstract”

On fresh paired length-24 prompts, the base compiler reached 30.1%, the unconstrained learned scorer reached 52.0%, and the iterative one-edit-at-a-time repair loop reached 52.7% at K=2. The local oracle ceiling was 89.1%.

Overview

Standalone experiment for an iterative hidden-program repair policy attached to a frozen Qwen numeric compiler. The policy receives the current compiled program trace, predicts sparse slot edits, and is evaluated over multiple repair iterations.

Small outputs live in this experiment directory. Large checkpoints live under:

large_artifacts/qwen_iterative_repair_policy/checkpoints/

Report

Rendered from reports/qwen_iterative_repair_policy_paper.md

Abstract

This experiment tests whether a frozen Qwen-attached hidden-program compiler can be improved at inference time by a learned iterative repair loop. The compiler emits an executable modular-arithmetic program. A small verifier scores local candidate repairs from execution-trace features. At inference, the repair loop starts from the base compiled program and may move one slot edit at a time for K iterations.

The primary run trained on 384 length-24 programs and selected checkpoints on a paired standard/paraphrase validation split. On fresh paired length-24 prompts, the base compiler reached 30.1%, the unconstrained learned scorer reached 52.0%, and the iterative one-edit-at-a-time repair loop reached 52.7% at K=2. The local oracle ceiling was 89.1%.

Setup

Substrate: frozen Qwen 4B hidden-program compiler localized in this experiment's artifact directory.
Program domain: length-24 modular arithmetic over modulus 97.
Hidden program slots: initial value, operation per step, and argument per step.
Candidate set: top-3 local alternatives with up to two edits around the base compiled program.
Learned component: small transformer verifier over candidate execution traces and candidate metadata.
Iterative rule: at each iteration, move only to a candidate within one slot edit of the current candidate if its learned score is higher.
Primary run: main_iterative_candidate_scorer_s384.
Verifier epochs: 12; selected epoch: 9.

Main Results

Split	Base	Learned	Best K	Iterative	Oracle	Delta	Gap
Fresh standard L24	29.7%	45.3%	2	44.8%	89.6%	+15.1 pp	25.2%
Fresh paraphrase L24	29.7%	50.5%	2	51.6%	88.0%	+21.9 pp	37.5%
Fresh paired L24	30.1%	52.0%	2	52.7%	89.1%	+22.7 pp	38.4%

Fresh accuracy

K Sweep

The K sweep shows most of the gain appears by K=1, with a smaller but real second-step gain on validation, paraphrase, and paired splits. K=3 adds no measurable benefit because the candidate set is capped at two edits.

Accuracy by K

On fresh paired prompts, exact execution moved from 30.1% at K=0 to 47.3% at K=1 and 52.7% at K=2.

Oracle Gap

Oracle gap

At K=2, the iterative repair loop recovered 38.4% of the fresh paired base-to-oracle gap, 25.2% on fresh standard prompts, and 37.5% on fresh paraphrases.

Training Dynamics

Verifier training

The verifier's paired-validation accuracy was noisy, so checkpoint selection was necessary. The selected checkpoint achieved strong fresh transfer despite the small train set.

Iteration Path

Iteration comparison

The direct raw-value repair policy was not robust: one setting damaged fresh splits and a paired-selected setting mostly learned to copy. The candidate scorer was the first robust positive arm because it separated proposal quality from sparse transition control.

Interpretation

The result supports the narrow claim that an iterative hidden-program repair loop can convert near-miss Qwen-compiled programs into exact programs without generating chain-of-thought text. It does not show a universal capability gain: the task is synthetic, the runtime is hand-designed, and the oracle ceiling still leaves substantial unrecovered headroom. The important signal is that iteration over an executable latent representation improved fresh paired exact execution from 30.1% to 52.7%, slightly beating unconstrained learned selection on the paired split.

Limitations

The verifier is trained with offline exact-state labels.
The runtime is specialized to modular arithmetic.
The candidate set is local and capped at two edits.
K=3 cannot expose deeper repairs under this candidate budget.
The base compiler is frozen; this run does not update Qwen weights.

Artifacts

Small files:

experiments/qwen_iterative_repair_policy/runs/main_iterative_candidate_scorer_s384/metrics.csv
experiments/qwen_iterative_repair_policy/runs/main_iterative_candidate_scorer_s384/verifier_train_log.csv
experiments/qwen_iterative_repair_policy/analysis/main_final_metrics.csv
experiments/qwen_iterative_repair_policy/reports/qwen_iterative_repair_policy_paper.md
experiments/qwen_iterative_repair_policy/reports/qwen_iterative_repair_policy_paper.html

Large files:

large_artifacts/qwen_iterative_repair_policy/checkpoints/fixed_compiler_step00800/
large_artifacts/qwen_iterative_repair_policy/checkpoints/main_iterative_candidate_scorer_s384/candidate_trace_verifier.pt

Experiment log 1

Show the running log (1 entry, 2026-06-23)

2026-06-23

Objective: test whether repeated learned repairs over a hidden executable program can recover exact execution from a frozen Qwen-attached compiler. The experiment must be standalone, keep its own logs and artifacts, and produce Markdown and HTML reports with charts.

Initial setup:

Created experiments/qwen_iterative_repair_policy/.
Created large_artifacts/qwen_iterative_repair_policy/checkpoints/.
Localized a frozen compiler checkpoint into the new large-artifact namespace: large_artifacts/qwen_iterative_repair_policy/checkpoints/fixed_compiler_step00800.
Seeded source from the existing Qwen numeric compiler utilities and began replacing one-shot editor evaluation with explicit iterative repair.

Smoke run:

python experiments/qwen_iterative_repair_policy/src/qwen_iterative_repair_policy_experiment.py \
  --run_name smoke_iterative_repair_policy \
  --train_examples 8 --val_examples 4 --eval_examples 4 --eval_pairs 4 \
  --editor_epochs 1 --qwen_batch_size 4 --repair_topk 2 --repair_max_edits 1 \
  --repair_max_pair_arg_slots 4 --editor_d_model 64 --editor_layers 1 \
  --editor_heads 4 --augment_cases_per_group 1 --augment_top_pool 4 \
  --include_target_noedit_case --stats_candidate_traces 2 \
  --train_select_k_values 1,2 --eval_k_values 0,1,2 \
  --max_edits_per_iteration 1 --max_length 384 --seed 201 --eval_seed 201001

Smoke result:

Completed successfully in 12.9 seconds after model load.
Wrote run metrics, train log, results JSON, and an iterative repair policy checkpoint.
Verified K-sweep rows are emitted for K=0,1,2.
Tiny split numbers are not interpreted; this was only a wiring check.

Pilot 1:

python experiments/qwen_iterative_repair_policy/src/qwen_iterative_repair_policy_experiment.py \
  --run_name pilot_iterative_k1_sparse_s96 \
  --train_examples 96 --val_examples 48 --eval_examples 64 --eval_pairs 64 \
  --editor_epochs 6 --qwen_batch_size 8 --repair_topk 3 --repair_max_edits 2 \
  --repair_max_pair_arg_slots 24 --editor_d_model 128 --editor_layers 2 \
  --editor_heads 4 --augment_cases_per_group 2 --augment_top_pool 16 \
  --include_target_noedit_case --stats_candidate_traces 6 \
  --train_select_k_values 1,2,3 --eval_k_values 0,1,2,3 \
  --max_edits_per_iteration 1 --unchanged_value_loss_weight 0.05 \
  --edit_gate_pos_weight 8.0 --changed_slot_weight 4.0 \
  --max_length 384 --seed 202 --eval_seed 202001

Pilot 1 result:

Completed in 580.9 seconds.
Validation improved from 27.1% base to 33.3% at K=1, but fresh transfer failed.
Fresh standard length-24 fell from 31.2% base to 29.7%; fresh paraphrase fell from 23.4% to 15.6-17.2%; fresh paired fell from 32.0% to 27.3%.
Interpretation: the policy learned a validation-specific sparse repair trigger. The next iteration should select checkpoints on a paired validation split and reduce noisy fallback edits when no local oracle candidate exists.

Iteration after pilot 1:

Added optional val_paired_len24 construction with --val_pairs.
Added --selection_split so checkpoint selection can use paired validation.
Changed selection priority to paired both-correct when the selection split is paired.

Pilot 2:

python experiments/qwen_iterative_repair_policy/src/qwen_iterative_repair_policy_experiment.py \
  --run_name pilot_iterative_paired_select_s96 \
  --train_examples 96 --val_examples 48 --val_pairs 48 \
  --selection_split val_paired_len24 --eval_examples 64 --eval_pairs 64 \
  --editor_epochs 8 --qwen_batch_size 8 --repair_topk 3 --repair_max_edits 2 \
  --repair_max_pair_arg_slots 24 --editor_d_model 128 --editor_layers 2 \
  --editor_heads 4 --augment_cases_per_group 3 --augment_top_pool 24 \
  --augment_repairable_only --include_target_noedit_case \
  --stats_candidate_traces 8 --train_select_k_values 1,2,3 \
  --eval_k_values 0,1,2,3 --max_edits_per_iteration 1 \
  --editor_target_mode oracle_or_base --unchanged_value_loss_weight 0.1 \
  --edit_gate_pos_weight 6.0 --changed_slot_weight 5.0 \
  --max_length 384 --seed 203 --eval_seed 203001

Pilot 2 result:

Completed in 960.5 seconds.
Paired validation selected a very conservative checkpoint: validation paired improved from 29.2% base to 31.2% at K=1, but fresh standard, paraphrase, and paired metrics all copied the base exactly.
Interpretation: paired selection prevented the damage seen in pilot 1, but the direct raw-value edit policy did not learn robust edits.

Iteration after pilot 2:

Added a candidate-scorer method. It trains a trace verifier over the local candidate set, then applies it iteratively by moving only to candidates within one edit of the current candidate at each step.
Kept K as an explicit eval axis; K=2 can reach two-edit candidates through one-edit transitions instead of selecting them directly at K=1.

Candidate-scorer smoke:

python experiments/qwen_iterative_repair_policy/src/qwen_iterative_repair_policy_experiment.py \
  --method candidate_scorer --run_name smoke_iterative_candidate_scorer \
  --train_examples 8 --val_examples 4 --eval_examples 4 --eval_pairs 4 \
  --verifier_epochs 1 --qwen_batch_size 4 --repair_topk 2 \
  --repair_max_edits 1 --repair_max_pair_arg_slots 4 --trace_d_model 64 \
  --trace_layers 1 --trace_heads 4 --eval_k_values 0,1,2 \
  --max_edits_per_iteration 1 --max_length 384 --seed 204 --eval_seed 204001

Candidate-scorer smoke result:

Completed successfully in 9.3 seconds after model load.
Verified verifier training, checkpoint writing, and iterative candidate K-sweep metrics.

Candidate-scorer pilot:

python experiments/qwen_iterative_repair_policy/src/qwen_iterative_repair_policy_experiment.py \
  --method candidate_scorer --run_name pilot_iterative_candidate_scorer_s128 \
  --train_examples 128 --val_examples 64 --val_pairs 64 \
  --selection_split val_paired_len24 --eval_examples 96 --eval_pairs 96 \
  --verifier_epochs 8 --qwen_batch_size 8 --repair_topk 3 --repair_max_edits 2 \
  --repair_max_pair_arg_slots 24 --trace_d_model 128 --trace_layers 2 \
  --trace_heads 4 --base_positive_group_weight 0.25 \
  --repairable_group_weight 8.0 --no_positive_group_weight 1.0 \
  --no_positive_base_weight 0.2 --eval_k_values 0,1,2,3 \
  --max_edits_per_iteration 1 --candidate_accept_margin 0.0 \
  --max_length 384 --seed 205 --eval_seed 205001

Candidate-scorer pilot result:

Completed in 326.9 seconds.
Fresh standard length-24 improved from 24.0% base to 35.4% at K=2.
Fresh paraphrase length-24 improved from 30.2% base to 33.3% at K=2.
Fresh paired length-24 improved from 29.2% base to 39.6% at K=1.
Full learned scoring without iterative constraints reached 34.9% fresh paired, so the constrained iterative path gave an additional +4.7 pp on that split.
Interpretation: candidate scoring is the first positive repair-policy arm. The effect is not monotonic in K on all splits, so the main run should report the full K sweep rather than one headline K.

Main run:

python experiments/qwen_iterative_repair_policy/src/qwen_iterative_repair_policy_experiment.py \
  --method candidate_scorer --run_name main_iterative_candidate_scorer_s384 \
  --train_examples 384 --val_examples 128 --val_pairs 128 \
  --selection_split val_paired_len24 --eval_examples 192 --eval_pairs 128 \
  --verifier_epochs 12 --qwen_batch_size 8 --repair_topk 3 \
  --repair_max_edits 2 --repair_max_pair_arg_slots 24 --trace_d_model 128 \
  --trace_layers 2 --trace_heads 4 --base_positive_group_weight 0.25 \
  --repairable_group_weight 8.0 --no_positive_group_weight 1.0 \
  --no_positive_base_weight 0.2 --eval_k_values 0,1,2,3 \
  --max_edits_per_iteration 1 --candidate_accept_margin 0.0 \
  --max_length 384 --seed 206 --eval_seed 206001

Main result:

Completed in 757.3 seconds.
Fresh standard length-24 improved from 29.7% base to 44.8% at K=2 versus an 89.6% oracle.
Fresh paraphrase length-24 improved from 29.7% base to 51.6% at K=2 versus an 88.0% oracle.
Fresh paired length-24 improved from 30.1% base to 52.7% at K=2 versus an 89.1% oracle.
On fresh paired prompts, unconstrained learned scoring reached 52.0%, so the iterative one-edit path slightly exceeded full learned selection while keeping the repair transition sparse.
K=3 did not improve over K=2, consistent with the candidate set being capped at two edits.

Report generation:

python experiments/qwen_iterative_repair_policy/src/analyze_qwen_iterative_repair_policy.py

Generated:

analysis/all_final_metrics.csv
analysis/main_final_metrics.csv
analysis/fresh_main_summary.csv
analysis/verifier_train_logs.csv
analysis/direct_policy_train_logs.csv
analysis/figures/accuracy_by_k_main.png
analysis/figures/fresh_accuracy_bars.png
analysis/figures/oracle_gap_recovered.png
analysis/figures/verifier_training_curve.png
analysis/figures/iteration_path_comparison.png
reports/qwen_iterative_repair_policy_paper.md
reports/qwen_iterative_repair_policy_paper.html
checkpoint_manifest.csv

Figures 5

iteration path comparison · analysis/figures/

oracle gap recovered · analysis/figures/

verifier training curve · analysis/figures/

Data files 16

Result tables and metrics copied from the experiment folder — preview inline or open the raw file.

Reproduce

This entry is source code or analysis only — there is no separate run to reproduce.

Browse the experiment folder on GitHub ↗