Research log Small Model Experimentation
GitHub

Repository search-compress-bank coding curriculum

Training on clean wins erases recovery skill

The one idea you need

Imagine training a mechanic only on repairs that went perfectly the first time: inspect, fix, test, done. They nail that tidy four-step routine but never watch a stuck bolt get worked loose — so when a fix is rejected, they just resubmit the same broken move.

The question

If you train a small coding agent only on its clean, fully successful bug-fix runs, does it get broadly better at fixing bugs across new codebases?

What we found

No. It became flawless on the six codebase types it trained on, solving every task in exactly four clean steps. But on four never-seen codebase types its success collapsed from 68% to 35% — worse than just running the untrained model twice, which scored 53%. Stripped of every failed attempt, it never learned what to do when a fix gets rejected.

Why it matters

When distilling agent trajectories, keep the failures and recoveries, not just the shortest winning path. Balance error-conditioned moves — failed-test-then-revise — not just how often each action appears, and test recovery on unseen tasks before trusting it.

Practiced codebase types, before vs after83% → 100%success on the six trained families it memorized
Unseen codebase types, before vs after68% → 35%success on four never-trained families (a 33-point drop)
Retries a fix after a failed test92% → 0%untrained model re-patched 24 of 26 times; trained model 0 of 48
Beaten by running the untrained model twice35% vs 53%same-compute baseline outscored it by 18 points
On this page
  1. Results at a glance
  2. Overview
  3. Report
    1. Outcome
    2. Training and artifact validity
    3. Mechanism diagnosis
    4. Decision and learned lessons
  4. Experiment log
  5. Data files
  6. Reproduce
  7. Related

Results at a glance 2

Training perfected practiced codebases but broke unseen ones

How to read

Grouped bars show the share of coding tasks solved. The left pair is the six practiced codebase types, the right pair four unseen types. The lighter bar is before training, the darker bar after clean-wins training. Taller is better.

0%25%50%75%100%trained family (48 tasks)trained family (48 tasks)83.3%100%4 unseen families (72 tasks)4 unseen families (72 tasks)68.1%34.7%

Takeaway → The practiced pair climbs to a perfect 100%, but the unseen pair drops by a third — the gain was memorized cloning, not transferable coding skill.

Data table
task familybefore training (frozen)after compact-bank training
trained family (48 tasks)83.3%100%
4 unseen families (72 tasks)68.1%34.7%

Numbers from reports/result_receipt.json

Technical framing

Compact-bank training helped the trained family but broke transfer — Trained-family success rose to perfect, but on four wholly unseen repository families it fell by a third — family-specific policy cloning, not transferable skill.

Every training example kept all four coding steps equally

How to read

Bars count target training rows per step — inspect, patch, verify, commit — in a 20-repository setup check. All four sit at exactly 20 rows, showing compression kept each step in equal amounts. This is a data check, not a model result.

0510152020inspect20patch20verify20commit

Takeaway → Perfectly level bars confirm the compression balanced how often each step appears — yet equal counts still failed, because they never say when to recover after a failure.

Data table
operatorcanonical target rows
inspect20
patch20
verify20
commit20

Numbers from experiments/qwen35_4b_repo_search_compress_bank/reports/smoke_receipt.json

Technical framing

Replay compression preserves all four coding operators in smoke — This is a deterministic substrate smoke, not a model result. All 20 test repositories replayed one target for each operator with equal loss mass and no hidden-test leakage.

In the author’s words from the Overview · “Result”

PRIMARY GATE FAILED. The compact arm improved trained-family success from 40/48 to 48/48 (+16.7 points) and reduced the learned path to exactly four valid calls on every task, but this was family-specific policy cloning rather than transferable coding competence. On four wholly held-out repository families it fell from 49/72 to 25/72 (−33.3 points; paired 95% CI [−44.4, −22.2]) and also lost to matched-compute sampling by 18.1 points. Median unrelated-context centered-logit drift was 0.386 against the frozen 0.15 ceiling. The transition audit localizes the failure. After a failed visible test, the apex control patched again on 24/26 transitions; compact did so on 0/48, instead re-testing, re-inspecting, or emitting an invalid action. … Read the full result →

Overview

This experiment asks whether Qwen/Qwen3.5-4B can acquire broader coding-agent competence by searching real procedural repositories, verifying candidate repairs with executable tests, compressing successful tool trajectories to their replay-necessary core, and banking compact plans and actions with explicit operator balance.

Research program

Primary program: agentic_breadth_installation. The experiment follows C53's queued scaffold-distillation direction, composes C12/C22 verified banking with C54 compression advantage, and directly guards the semantic-operator collapse found by the interactive-policy curriculum.

The closest near-duplicate is qwen35_4b_think_ftpo_round2: it introduced a six-family procedural repository agent only as a fresh evaluation gate. It did not search repositories for training data, replay-minimize successful edits, bank tool states, balance operators, or train on coding-agent actions.

Hypothesis

Long successful coding trajectories contain a small causal core: issue-directed inspection, necessary edits, post-edit verification, and commit. Executable minimization should discard failed branches and redundant patches; compact state-specific planning should install the useful transition structure; equal INSPECT/PATCH/VERIFY/COMMIT loss mass should preserve the rare terminal operators that broad DAgger erased.

The hypothesis fails if the compact arm cannot beat (a) an identical action-only bank, (b) the regenerated C54 apex policy on family-disjoint repositories, and (c) two shorter apex samples under the same eight-call/token reservation.

Result

PRIMARY GATE FAILED. The compact arm improved trained-family success from 40/48 to 48/48 (+16.7 points) and reduced the learned path to exactly four valid calls on every task, but this was family-specific policy cloning rather than transferable coding competence. On four wholly held-out repository families it fell from 49/72 to 25/72 (−33.3 points; paired 95% CI [−44.4, −22.2]) and also lost to matched-compute sampling by 18.1 points. Median unrelated-context centered-logit drift was 0.386 against the frozen 0.15 ceiling.

The transition audit localizes the failure. After a failed visible test, the apex control patched again on 24/26 transitions; compact did so on 0/48, instead re-testing, re-inspecting, or emitting an invalid action. All 18 recursive-overlay tasks repeated the same exact patch after the patch tool had already rejected it. Operator-marginal balance preserved the happy-path vocabulary, but success-only compression deleted failure-conditioned recovery behavior.

The necessary gate stopped action-only training, confirmation, and Menagerie. Therefore the result rejects the registered compact-bank recipe as a whole; it does not identify compact plan gradients rather than action gradients as the cause. See the final report, primary gate, and result receipt.

Firewall-clean substrate

Ten fresh Python repository generators materialize source files and a visible test. Six families are eligible for search/training; four algorithmically distinct families are never harvested or trained and are used only for transfer. Hidden test programs and oracle edits live in host memory, are never written into the repository, never enter a model message, and are reduced to pass/fail booleans in receipts. The benchmark directory is never read or imported.

The agent has real bounded tools: tree, read, literal search, exact single-replacement patch, subprocess test, and submit. Source/test paths are constrained to the temporary repository; visible tests are readable but immutable.

Arms and controls

  • apex_replay: regenerate C54 apex from its committed training data with this experiment's fixed optimizer budget.
  • compact: the same C54 data plus replay-minimized repository rows with compact state-specific plans.
  • action_only: identical repository contexts, actions, row weights, and optimizer budget, but zero loss on the compact plan span.
  • C53 blend: fixed search teacher and contextual deployment comparator, never a source of private labels.
  • Matched sampling: two independent four-turn apex rollouts versus one eight-turn candidate rollout; both reserve eight model calls and 6,144 sampled tokens per task.

Entropy and varentropy may be recorded later to route which live states deserve more search, but verifier outcomes supply correctness and operator class supplies balancing. They never scale token loss or choose a push-up/push-down target.

Frozen stages

  1. Self-test every generator, path boundary, and hidden-label firewall.
  2. Search 24 tasks from each of six training families with four eight-turn C53 trajectories.
  3. Keep private-test successes, greedily delete replay-unnecessary patches, then reconstruct and replay inspect→patch→test→submit traces.
  4. Train the three registered arms from the pinned base at the same 584 optimizer steps.
  5. Gate trained-family retention, family-disjoint transfer, compact-vs-action-only advantage, matched-call sampling, verification/commit retention, invalid actions, and unrelated-context logit locality.
  6. Confirm on a second transfer block. Only then assign fresh union-checked Menagerie seeds and compare aggregate quick/medium scores through the benchmark CLI.

The run stopped during stage 5. Stages 6 and Menagerie were not authorized.

Exact thresholds, seeds, and interpretation rules are frozen in the preregistration, its pre-harvest token-mass implementation amendment, the memory-feasibility audit, the final exact chunked-loss amendment, and configuration. The adversarial review is in reports/design_review.md.

Run

CPU smoke (no model or benchmark):

.venv/bin/python experiments/qwen35_4b_repo_search_compress_bank/scripts/run.py --smoke

GPU smoke after the preregistration commit:

.venv-vllm/bin/python experiments/qwen35_4b_repo_search_compress_bank/scripts/run.py --gpu-smoke

The staged full command is intentionally gate-stopping:

.venv/bin/python experiments/qwen35_4b_repo_search_compress_bank/scripts/run.py --full

Result-bearing model outputs and weights live under large_artifacts/qwen35_4b_repo_search_compress_bank; small summaries and receipts are committed. No PR is created: accepted work is rebased and pushed directly to main.

Status

PRIMARY GATE FAILED; EXPERIMENT STOPPED. Both licensed 584-step arms trained and merged successfully. The compact arm overfit the trained procedural families, regressed family-disjoint transfer, lost to matched sampling, failed verification/invalid-action retention, and exceeded the locality ceiling. Action-only, confirmation, and Menagerie were correctly cancelled; zero benchmark seeds were consumed.

Report

Rendered from reports/report.md

Status: negative; primary gate failed and all downstream work stopped. Menagerie remained sealed.

Outcome

The registered recipe installed a fast, exact policy for the six training families but damaged the broader repository agent. Compact improved the trained-family block from 40/48 to 48/48 (+16.7 points, paired 95% CI [+6.3, +27.1]), then fell on four wholly held-out algorithm families from 49/72 to 25/72 (−33.3 points, CI [−44.4, −22.2]). Matched-compute sampling reached 38/72, so compact also lost that comparator by 18.1 points (CI [−27.8, −8.3]).

Frozen blockApex replayCompactCompact delta
Six trained families, deep40/48 (0.833)48/48 (1.000)+0.167
Four held-out families, deep49/72 (0.681)25/72 (0.347)−0.333
Four held-out families, matched sampling38/72 (0.528)compact −0.181

Held-out family success exposes the narrow transfer:

FamilyApex replayCompact
dependency_order13/180/18
recursive_overlay0/180/18
retry_schedule18/1818/18
ttl_cache18/187/18

Compact verification given success was 0.88 versus 1.00 for apex, invalid actions rose from 0.093 to 0.260 per turn, and submit rate fell from 0.889 to 0.306. Commit given verification was preserved (1.00 versus 0.939), so this is not loss of the terminal submit operator after a genuinely successful verification.

The unrelated-context locality guard also failed: median centered non-target logit drift was 0.386 against the frozen 0.15 ceiling. Mean entropy changed only −0.0094, so entropy did not reveal the collateral that the centered-logit audit caught.

Training and artifact validity

Both licensed arms completed the exact registered 584 optimizer steps from pinned Qwen/Qwen3.5-4B revision 851bf6e8, at effective batch 16. Apex ran for 10,007 seconds and compact for 9,633 seconds. The exact checkpointed full-vocabulary loss peaked at 48.58 GB in both arms and crossed the former dense-loss OOM path without approximation. Every one of 128 LoRA modules was nonzero and explicitly merged into each composite checkpoint before vLLM evaluation.

The frozen C53 search policy had covered 129/144 fresh training-family repositories and supplied 376 conjunctive visible+private successes. Replay minimization admitted all 129 covered tasks, collapsed every per-file edit to one patch, and produced 516 rows—exactly 129 each for INSPECT, PATCH, VERIFY, and COMMIT. Weighted action-token mass was exactly 36,110 per operator and compact-plan mass 3,125.8 per operator. All 129 canonical task traces replayed before training. The negative therefore survives the intended data, weighting, padding, merge, and inference-backend checks.

Mechanism diagnosis

The bank compressed away the policy states that matter after something goes wrong.

  • On trained tasks, compact produced exactly INSPECT→PATCH→VERIFY→COMMIT on all 48/48 examples, with zero invalid calls, four turns, and only 402 sampled tokens on average. This is a strong family-specific transducer.
  • Among failed-test observations with another turn available, the apex control chose another PATCH on 24/26 next transitions (92.3%). Compact chose another PATCH on 0/48; it chose VERIFY 20 times, INSPECT 17 times, and an invalid output 11 times.
  • After a passing test, both policies retained commit: apex committed on 64/65 next transitions and compact on 22/22.
  • Compact encountered rejected exact patches on all 18 recursive_overlay tasks, re-read the file, then repeated the byte-identical rejected patch multiple times on all 18. It had learned the nominal edit shape without learning how to repair an exact-match failure.
  • On ttl_cache, compact made a plausible partial fix, observed a visible-test failure, then mostly re-tested or re-inspected instead of revising. retry_schedule, whose repair matched the single local happy-path pattern, stayed perfect.

This sharpens the prior semantic-operator lesson. The interactive-policy curriculum failed because rare verify/commit operators disappeared. This experiment balanced those operator marginals exactly and preserved commit after pass, yet still failed because marginal counts do not specify conditional transitions. A looping agent needs a contingency policy: failed patch → re-inspect and change the edit; failed test → diagnose and revise; passed test → submit. Success-only minimization retained only the last of those.

The locality failure is consistent with additional shared-weight collateral, but it is not a complete causal attribution. Action-only training was intentionally conditional on the necessary gate and did not run. The candidate also displaced some C54 replay examples under the fixed 584-step budget (1.801 mixed-data epochs versus 2.0 apex epochs). The supported conclusion is that the registered compact plan-plus-action recipe fails, not that compact plans alone are harmful.

Decision and learned lessons

The primary gate failed 7 of 11 checks: held-out transfer, both paired-bootstrap lower bounds, matched sampling, invalid actions, verification retention, and locality. Trained-family retention, absolute verification, and both commit checks passed. The orchestrator recorded stop_before_action_only_confirmation_and_menagerie; action-only, confirmation, and Menagerie did not run, and zero benchmark seeds were consumed.

Do not repeat this success-only one-patch bank at another dose. A defensible successor requires a new experiment and fresh procedural splits, with one of two strategies:

  1. Bank a verifier-conditioned state machine rather than a minimal success trace. Preserve rejected patches, failed tests, changed second edits, and recovery transitions; balance conditional transitions such as failed_test→PATCH, not only operator totals. Require a frozen perturbation/recovery gate before training.
  2. Keep tool-found scaffolds external and retrieve/execute them conditionally, avoiding a broad shared-weight policy update. Compare this against matched compute and the same held-out algorithm-family transfer.

Either strategy must pass locality and family-disjoint recovery before action-only attribution, confirmation, or Menagerie. The consumed trained_dev and transfer_dev seeds are now analysis data and cannot become future training material or be reused as fresh capability evidence.

The compact machine-readable record is result_receipt.json; paired decisions are in repo_primary_gate.json.

Experiment log 4

Show the running log (4 entries)

Preregistration and implementation

  • Re-audited the program state, with C12/C22, C52, C53, C54, the evaluation-only FTPO round-2 repository harness, and the failed interactive-policy curriculum as closest anchors.
  • Selected executable replay compression plus operator-balanced compact banking; entropy/varentropy are routing diagnostics only.
  • Created ten procedural repository families: six train/search families and four family-disjoint transfer families.
  • Verified on CPU that every family starts visible/hidden broken and becomes visible/hidden correct under its host-only oracle.
  • Implemented constrained real filesystem tools, answer-region JSON parsing, terminal hidden grading, replay patch deletion, canonical trace reconstruction, operator balancing, and firewall checks.
  • Froze seeds, arms, doses, gates, and the conditional benchmark license before result-bearing generation.
  • After preregistration commit 462f6274, the first GPU smoke stopped before model load because the current runner template lacked its older local-checkpoint field. Added exact Qwen3.5-4B local-checkpoint support plus an architecture fingerprint test; no scientific seed or output was consumed by the failed attempt.
  • The corrected GPU smoke loaded the merged C53 checkpoint under vLLM 0.24 and repaired 5/6 one-task-per-family repositories within four turns (implementation evidence only). Its full trajectories remain external and firewall-clean.
  • Corrected smoke scoring to require both visible and private tests: the valid count is 4/6; one apparent fifth success passed private edge cases while regressing the visible suite. Added a regression test for this exact failure.
  • Replaced equal row-count weighting with exact tokenizer-level action-token mass equality and separately equal compact-plan mass. The action-only control now retains the identical teacher-forced compact text and removes only its plan-span gradient.
  • Froze 48 unrelated non-coding contexts for apex→compact centered-logit locality and implemented the full matched-step trainer, explicit composite merge, paired repository analysis, and gate-stopping continuation path while the registered harvest ran.

Registered harvest and bank

  • Search teacher covered 129/144 tasks (89.6%; per-family range 58.3–100%) from 576 trajectories and 1,225,314 sampled tokens. There were 376 conjunctive visible+private successes, 370 explicit post-patch verifications, and 351 commits after a pass.
  • Replay minimization admitted all 129 covered tasks. Seven source trajectories needed two patch calls, but per-file initial→final collapse reduced every canonical trace to one patch; all canonical visible/private/submission replays passed.
  • Built 516 rows: exactly 129 each for INSPECT/PATCH/VERIFY/COMMIT. Exact weighted action-token mass is 36,110 per operator and compact-plan mass 3,125.8 per operator. Longest target sequence is 879 tokens.
  • Tokenizer preflight encoded all 4,669 C54 rows and all 516 repository rows with zero skips. Apex replay is exactly 2.0 dataset epochs at 584 steps; both candidates are exactly 1.8011 union epochs at the same 584 steps.
  • Every registered pre-training gate passed; training authorized. Menagerie remains sealed.

Training feasibility recovery

  • Apex replay with batch 4 × accumulation 4 stopped at optimizer step 52 on a 3,193-token batch: the 9.54 GiB logits allocation exceeded 9.09 GiB free. Loss/gradients were finite, but no adapter or checkpoint was saved.
  • Froze the compute-equivalent recovery before rerun: batch 2 × accumulation 8, effective batch 16, 584 steps, 9,344 examples, three apex padding duplicates, and exactly two control epochs. Enabled expandable CUDA segments. All arms share the corrected geometry.
  • The 2 × 8 fallback completed three unsaved steps but measured 19–23 s/step. Root cause was dense cross-entropy's full 248k-vocabulary FP32 temporary, not forward activations. Restored the original 4 × 4 geometry with exact gradient-checkpointed 128-position loss chunks; CPU loss/gradient equivalence passes at 1e-6.
  • A two-step canary on the eight longest 3,193-token rows passed at batch four, peaking at 48,375,846,912 CUDA bytes. This directly covers the formerly failing allocation path; the smoke adapter is not eligible for evaluation.

Result-bearing training and necessary gate

  • Apex replay completed 584/584 optimizer steps in 10,007 seconds; compact completed 584/584 in 9,633 seconds. Both used exact checkpointed full-vocabulary cross-entropy, effective batch 16, and approximately 48.58 GB peak CUDA memory. All 128 LoRA modules were nonzero and explicitly merged for both arms.
  • Trained-family deep evaluation: apex 40/48, compact 48/48 (+16.7pp; paired CI [+6.3,+27.1]). Compact used the exact four-operator canonical path on all 48 tasks, with zero invalid calls and 402 versus 1,643 mean sampled tokens.
  • Family-disjoint transfer: apex 49/72, compact 25/72 (−33.3pp; CI [−44.4,−22.2]). Matched two-by-four-turn apex sampling scored 38/72, still +18.1pp over compact. Compact invalid calls rose 9.3%→26.0%, verification retention fell 1.00→0.88, and median unrelated-context centered-logit drift was 0.386 versus the 0.15 ceiling.
  • Transition audit: after a failed visible test, apex patched again on 24/26 transitions; compact did so on 0/48. After a pass, commit remained intact (64/65 apex, 22/22 compact). All 18 recursive-overlay compact trajectories repeated the identical already-rejected patch. Exact operator marginals preserved the happy path but not verifier-conditioned recovery.
  • The primary gate stopped action-only training, confirmation, and Menagerie. Zero benchmark seeds were assigned or consumed. The action-only cancellation leaves plan-gradient attribution unresolved by design.
  • Fixed a staged-run footgun found on the negative path: locality correctly wrote a failing receipt and returned status 4, but the orchestrator previously raised before consolidated analysis. Expected gate status is now explicitly accepted, summarized, and returned cleanly; unexpected nonzero statuses still raise, with a regression test.

Data files 1

Result tables and metrics copied from the experiment folder — preview inline or open the raw file.

Reproduce

Smoke test

.venv/bin/python experiments/qwen35_4b_repo_search_compress_bank/scripts/run.py --smoke

Full run

.venv/bin/python experiments/qwen35_4b_repo_search_compress_bank/scripts/run.py --full

Run steps are documented inside the experiment folder (README and scripts).

Browse the experiment folder on GitHub ↗