Verifier-conditioned recovery banking curriculum
The one idea you need
A coding agent loops: read the code, edit, run the tests, commit. Trained only on flawless runs, it never meets a failed test — like a driver who practiced on empty roads and freezes at the first skid. So it replays its own fixes from real broken states.
The question
If you teach a small coding agent to recover by replaying moments it once fixed, does it learn to recover — and what does that cost?
What we found
Yes — replaying real failures taught recovery, lifting success from about half of cases to more than nine in ten. But the best version also added a little "explain your fix" text, and that supposedly tiny slice actually carried most of the training force, disturbing unrelated knowledge past a safety limit. The run stopped before it could even test whether recovery transfers to new code.
Why it matters
Don't gauge a training slice's impact by its share of the words. A handful of tokens the model finds unnatural can dominate the update and quietly corrupt unrelated skills. Measure the actual size of the update, not the word count.
On this page
Results at a glance 2
How to read
Four bars, left to right: the untrained agent, one trained only on clean successful runs, one trained on real failure moments, and one that also explains its fix. Bar height is the share of 60 controlled recovery cases solved; taller is better.
Takeaway → Each step up adds recovery skill, and the explain-your-fix bar is highest at more than nine in ten — but this is trained-family practice, not transfer to new code.
Data table
| calibration arm | hidden-test recovery success |
|---|---|
| frozen base | 48.3% |
| happy actions | 81.7% |
| recovery actions | 85% |
| recovery actions + plans | 91.7% |
Numbers from experiments/qwen35_4b_verifier_conditioned_recovery_bank/reports/result_receipt.json
Technical framing
Verifier-conditioned banking raises trained-family recovery — The frozen selector chose the plan arm at 91.7%, +10.0 points over the matched happy-path control. That is trained-family calibration only: the arm then failed locality, so no transfer task or Menagerie seed was exposed.
How to read
Three bars show how much each trained version disturbs the model's unrelated outputs; a flat reference line marks the allowed ceiling. Both action-only recipes sit well under it; the version that adds explanation text spikes far above. Lower is better.
Takeaway → Adding a nominally small amount of explanation text roughly triples the disturbance and clears the safety line, so the run halted before any transfer test ran.
Data table
| trained arm | median non-target logit drift | registered ceiling |
|---|---|---|
| happy actions | 0.0833 | 0.15 |
| recovery actions | 0.0982 | 0.15 |
| recovery actions + plans | 0.303 | 0.15 |
Numbers from experiments/qwen35_4b_verifier_conditioned_recovery_bank/reports/result_receipt.json
Technical framing
Plan supervision triples unrelated-logit drift and fails the safety ceiling — Both action-only arms pass. Adding nominally 5% plan-token mass increases merge-delta norm by 29.5% versus recovery action-only, pushes drift to 0.303, and lowers unrelated entropy by 0.106 nats. The run stopped here.
In the author’s words from the Overview · “Results”
The harvest covered 58/72 tasks (80.6%); 57 tasks survived patch minimization and produced 399 replay-clean rows per arm. On the 60-case trained-family calibration block (table on the experiment page). The frozen selector chose recovery_reason: +43.3pp over base, +10.0pp over happy, and +6.7pp over recovery action. It also improved the registered transition composite by +18.9pp over happy. But its centered non-target logit drift was 0.303 versus the 0.15 ceiling, so the run stopped. Exploratory mechanism controls isolated the damage: happy and recovery-action drift were only 0.083 and 0.098 and both passed locality. Their unrelated entropy changes were −0.016 and +0.006 nats, while recovery-reason reduced it by 0.106 nats. … Read the full result →
Overview
Research Program
- Program:
agentic_breadth_installation - Program question: can execution-selected policy compression install useful coding breadth without deleting rare recovery and completion behavior?
- Prior anchors: C5, C50, C52, and the direct predecessor C54.
Question
Can Qwen/Qwen3.5-4B learn a transferable coding-agent recovery policy when supervision is balanced at conditional state→action transitions—especially rejected-patch→changed-patch and failed-test→diagnose/revise—rather than only at global INSPECT/PATCH/VERIFY/COMMIT totals?
Hypothesis
The C54 failure was caused by conditioning collapse: its compact success traces contained no rejected edits or failed tests, so exact operator marginals still taught failed_test→nothing useful. Replay-verified failure-state rows should improve controlled recovery on unseen algorithm families. If the effect is real training rather than a promptable rule, it must beat the frozen incumbent, matched happy-path training, an explicit external recovery scaffold, and matched-compute sampling while retaining normal loops and unrelated logits.
Setup
- Model: only
Qwen/Qwen3.5-4B, revision851bf6e..., warm-started from the frozen C54apex_replaymerged checkpoint. - Substrate: ten fresh procedural Python repository families with visible and host-only hidden executable tests. Each fixture has an unresolved partial repair used only to construct public failed-test states.
- Train/selection: six families; selection uses new trained-family tasks only.
- Transfer: four wholly different families, followed by a new-seed confirmation block.
- Arms:
happy_action,recovery_action, andrecovery_reason(the last adds exactly 5% plan loss to byte-identical recovery rows). - Baselines: frozen start checkpoint, frozen start plus an external recovery reminder, and two independent half-depth trajectories at the same call/token reservation.
- Primary metric: controlled-recovery hidden-test success, paired by task and scenario.
- Retention: normal-start success, verify-after-final-patch, pass→commit, invalid actions, and unrelated-context next-token locality.
- Hidden boundary: the model sees only repository files, issue text, and visible tool/test output. Hidden executable text never leaves the host and no benchmark content is read.
The design is frozen in reports/preregistration.md, with the adversarial review in reports/design_review.md.
Run
CPU invariants:
.venv/bin/python experiments/qwen35_4b_verifier_conditioned_recovery_bank/scripts/run.py --smokeGPU integration smoke:
.venv/bin/python experiments/qwen35_4b_verifier_conditioned_recovery_bank/scripts/run.py --gpu-smokeStaged full run:
.venv/bin/python experiments/qwen35_4b_verifier_conditioned_recovery_bank/scripts/run.py --fullThe orchestrator is resumable. It stops before confirmation and Menagerie whenever a prerequisite gate fails.
Results
Verdict: stopped at the registered locality gate. Menagerie and both transfer blocks remained sealed.
The harvest covered 58/72 tasks (80.6%); 57 tasks survived patch minimization and produced 399 replay-clean rows per arm. On the 60-case trained-family calibration block:
| Arm | Recovery success | Failed-test success | Rejected-patch success | Invalid turns | Mean sampled tokens |
|---|---|---|---|---|---|
| frozen base | 48.3% | 53.3% | 43.3% | 5.7% | 2,340 |
| happy action | 81.7% | 80.0% | 83.3% | 2.0% | 1,377 |
| recovery action | 85.0% | 73.3% | 96.7% | 19.1% | 1,503 |
| recovery reason | 91.7% | 100% | 83.3% | 5.9% | 480 |
The frozen selector chose recovery_reason: +43.3pp over base, +10.0pp over happy, and +6.7pp over recovery action. It also improved the registered transition composite by +18.9pp over happy. But its centered non-target logit drift was 0.303 versus the 0.15 ceiling, so the run stopped.
Exploratory mechanism controls isolated the damage: happy and recovery-action drift were only 0.083 and 0.098 and both passed locality. Their unrelated entropy changes were −0.016 and +0.006 nats, while recovery-reason reduced it by 0.106 nats. Full result and source checksums are in reports/result_receipt.json.
Interpretation
Conditional transition banking contains a strong local signal, but this recipe does not establish capability transfer. Action-only failure-state learning is parameter-local and adds +3.3pp over the already-strong happy control, though it produces too many invalid actions. Concise plan supervision repairs those invalids and adds another +6.7pp, but its realized gradient is far larger than its nominal 5% token mass and violates locality.
Entropy/varentropy explains why. Before training, the correct JSON action-start token was already rank 1 at every action seam, and failure-specific plan starts were also rank 1. The imposed plan starts at ordinary pivots were unnatural: rank ~8,404 at inspect→patch, ~1,163 at patch-ok→verify, ~135 at start→inspect, and 3 at pass→commit. Plan SFT forced every one to rank 1 and near-zero entropy. Future plan dose must be calibrated by realized gradient/surprisal, not weighted token count.
Knowledgebase Update
- Program evidence: updated with the locality-gated negative and action-only positive control.
- Program backlog: queues a new locality-first interpolation experiment; transfer seeds remain untouched.
- Claim ledger/synthesis: deferred until the interpolation follow-up determines whether the signal survives a compliant dose.
Artifacts
src/: constrained repository environment, recovery scenarios, bank, and pinned vLLM runner.scripts/: harvest, bank, training, merge, evaluation, diagnostics, analysis, and staged orchestration.configs/default.yaml: frozen seeds, doses, budgets, and gates.reports/artifact_manifest.yaml: external artifact locations and regeneration commands.
Report
Rendered from reports/report.md
Summary
Stopped negative with a strong mechanism signal. Conditional transition banking raised trained-family controlled recovery from 48.3% to 91.7%, but the selected reasoning arm failed exact-logit locality (0.303 drift versus 0.15). Transfer and Menagerie remained sealed. Exploratory controls show verifier-conditioned action learning itself is local; nominally 5% plan supervision caused the collateral.
Research Program Fit
This experiment executed the agentic_breadth_installation backlog item created by C54: preserve failed_patch→changed_patch, failed_test→revision, and passed_test→commit, not merely aggregate tool marginals. It resolves that transition-balanced banking is learnable and locally promising, while plan-span pressure remains the bottleneck.
Method
The frozen C54 apex checkpoint searched 72 fresh procedural repositories across six training families. Successful model patches were minimized and replayed from deterministic public rejected-patch and failed-visible-test states. Each admitted task contributed seven state→action rows; weighted action mass was exactly equal by operator and within-operator transition.
Three r32 QLoRA arms trained for 120 steps from the same warm start:
happy_action: matched operator/transition action mass in successful-path contexts;recovery_action: actual public failure contexts, zero plan loss;recovery_reason: byte-identical recovery rows with nominal 5% plan-token mass.
The 60-case calibration block was fresh but used the six training families. A frozen lexicographic selector chose one recovery arm. Locality then compared the selected merged checkpoint against apex on 48 frozen unrelated contexts.
Results
Harvest and bank
- 58/72 tasks covered (80.6% versus the 35% bar), with every family at 41.7% or better.
- 143 successful trajectories; 135 preserved pass→commit.
- 57 model repairs survived minimization/collapse.
- 399 rows per arm; 100% replay; every structural, balance, firewall, and length gate passed.
Calibration
| Arm | Overall | Failed test | Rejected patch | Submit | Invalid/turn | Mean tokens |
|---|---|---|---|---|---|---|
| base | 0.483 | 0.533 | 0.433 | 0.283 | 0.057 | 2,340 |
| happy action | 0.817 | 0.800 | 0.833 | 0.767 | 0.020 | 1,377 |
| recovery action | 0.850 | 0.733 | 0.967 | 0.800 | 0.191 | 1,503 |
| recovery reason | 0.917 | 1.000 | 0.833 | 0.950 | 0.059 | 480 |
The selector chose recovery reason: +0.433 versus base, +0.100 versus happy, +0.067 versus action-only, and +0.189 on the transition composite versus happy. Verification and commit were both 1.0.
Locality and uncertainty
| Arm | Merge delta norm sum | Non-target drift | Entropy delta | Locality |
|---|---|---|---|---|
| happy action | 28.03 | 0.083 | −0.016 | pass |
| recovery action | 29.17 | 0.098 | +0.006 | pass |
| recovery reason | 37.78 | 0.303 | −0.106 | fail |
The nominal 5% plan span was not a 5% realized update. Step-10 loss/gradient were 12.52/1.80 for action-only and 43.61/42.12 for reason; the standard clip bounded magnitude but made early update direction plan-dominated.
Teacher-forced seam audit explains the pressure. Before any new training, the target action-start token was rank 1 with very low entropy for all seven transitions. Failure-specific plan starts were also already rank 1. The imposed ordinary-state plan starts were highly off-policy:
- inspect→patch: target plan token rank ~8,404;
- patch-ok→verify: ~1,163;
- start→inspect: ~135;
- passed-test→commit: 3.
Reason training made every plan and action seam rank 1 with near-zero entropy. Action-only made action seams sharper but left the plan distribution largely natural and passed unrelated-context locality.
Controls
The matched happy arm shows that most calibration recovery gain comes from balanced action training, not failure conditioning alone. Recovery conditioning adds +3.3pp overall and strongly reallocates success toward rejected-patch states, but also increases invalid actions. Reason supervision restores valid concise execution and adds +6.7pp, at unacceptable shared-weight cost.
The external scaffold and matched-sampling controls were correctly not funded: the selected checkpoint failed a prerequisite locality gate before any held-out family was touched.
Oracle Versus Deployable Evidence
Host oracles validated fixture truth conditions only. Full training targets came solely from the model's own execution-verified repairs. Hidden executable code/output never entered model context. Calibration hidden tests establish trained-family performance only; because transfer stayed sealed, this is not a breadth or downstream capability claim.
Interpretation
The experiment rejects its frozen headline recipe but leaves a sharper positive: verifier-conditioned action learning is parameter-local at this dose. The failure was placing positive pressure on arbitrary lexical plans whose first tokens were extremely unlikely under natural thinking. Token-mass calibration missed realized CE and gradient scale.
The most efficient next test is locality-first interpolation of the already-trained reason delta. Its 91.7% behavior margin is large, and drift should fall continuously with scale; action-only provides a full-dose locality-pass anchor. This must be a new experiment with the same calibration block for selection and untouched transfer seeds for the first claim-grade test.
Next Experiments
- New experiment: merge the reason LoRA at a frozen scale ladder, run locality before behavior, and select only among passing scales plus the action-only anchor.
- If interpolation retains a calibration advantage, evaluate the frozen winner against base, happy, scaffold, and matched sampling on the untouched four-family transfer blocks.
- Future retraining should calibrate plan pressure by realized gradient or target surprisal and supervise only genuinely useful, non-rank-1 pivots—not all plan starts.
Artifact Manifest
Large artifacts live under large_artifacts/qwen35_4b_verifier_conditioned_recovery_bank. The committed result receipt records compact metrics and SHA-256 provenance for harvest, bank, adapters, merges, evaluations, locality, and uncertainty audits. Detailed trajectories and 4B checkpoints remain external as listed in artifact_manifest.yaml.
Experiment log 2
Show the running log (2 entries, 2026-07-12)
2026-07-12 — intake and design
- Re-read the program index, scorecards, claim ledger, synthesis, model playbook, compute/vLLM guidance, and C54 predecessor artifacts.
- Chose conditional transition balancing over another think-token FTPO round: it directly addresses the observed
failed_test→no changed patchcollapse while retaining executable outcome selection. - Created ten fresh two-defect procedural repository families. Initial and partial workspaces must fail both visible and hidden tests; the oracle must pass both.
- Registered three warm-start arms, an external scaffold control, matched-compute sampling, family-held-out transfer, confirmation, normal-loop retention, locality, and exploratory entropy/varentropy diagnostics.
- CPU fixture and bank smoke passed after strengthening private partial-state checks.
- GPU integration smoke established that the local C54 composite can be QLoRA-trained, explicitly merged, and loaded by the pinned vLLM runner.
- Integration smoke caught and fixed a binary-file search crash caused by test-created
__pycache__; the public repository tools now expose only source/README files.
No result-bearing harvest, training, or evaluation had run when the preregistration was frozen.
2026-07-12 — full run
- Harvest: 58/72 tasks covered (80.6%); all family coverage gates passed. Patch minimization admitted 57 tasks and 399 rows/arm at 100% replay.
- All arms trained for 120 steps from the immutable apex warm start.
happy_actionandrecovery_actionhad total merge-delta norms 28.03 and 29.17.recovery_reasonreached 37.78. - The nominal 5% plan mass was not a 5% realized dose. At step 10, recovery action logged loss/gradient 12.52/1.80; recovery reason logged 43.61/42.12. The plans were much more surprising than the action targets, so clipping made early updates plan-dominated.
- Calibration recovery success: base 0.4833, happy 0.8167, recovery action 0.8500, recovery reason 0.9167. The frozen selector chose reason and passed its mechanism gate.
- Registered reason locality failed: drift 0.3031 > 0.15, unrelated entropy −0.1058 nats. Transfer and Menagerie stopped unopened.
- Exploratory controls: happy drift 0.0833 and recovery action 0.0982, both locality-pass; action-only entropy +0.0060. The plan span, not verifier-conditioned action learning, caused collateral.
- Seam audit: all target action starts were rank 1 before training. Unnatural plan starts at inspect→patch, patch-ok→verify, start→inspect, and pass→commit were ranks ~8,404/~1,163/~135/3 and were driven to rank 1 with near-zero entropy by reason training.
- Decision: preserve this result as a locality-gated negative. Use a new experiment for locality-first interpolation; do not reinterpret action-only or a scaled reason checkpoint inside this frozen run.
Data files 1
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
Reproduce
Smoke test
.venv/bin/python experiments/qwen35_4b_verifier_conditioned_recovery_bank/scripts/run.py --smokeFull run
.venv/bin/python experiments/qwen35_4b_verifier_conditioned_recovery_bank/scripts/run.py --fullRun steps are documented inside the experiment folder (README and scripts).