Qwen35 4B Repair + Why Stack

Finished2026-07-18Agentic Breadth Installation Structured Execution and Compilers GitHub ↗

The one idea you need

The team is trying to turn a small model that can write one correct function into one that can carry a coding task across many steps. Several earlier training sets each nudged the model a little, and two of them helped in DIFFERENT ways. One set taught the model to notice a failing check and fix its own code — a debug-and-retry behavior — and it lifted the model's multi-step coding score. A different set taught the model to write the reason each line of correct code is correct (as short inline notes the grader throws away), and it lifted the model's single-function correctness on the coding exams. Neither gain was large enough on its own to be sure it was real rather than luck. Because the two gains point at different weaknesses and neither hurt the other, the natural next move is to STACK them: shuffle the two training sets together into one and train the model once on the combination. This experiment builds and freezes that combined set (a deterministic 1008-example mix of the two 504-example sets, re-checked to contain no exam answers) and the exact training and scoring recipe; the training and evaluation runs themselves are gated behind reviews and have not happened yet.

The question

If one training set nudged the model's multi-step 'debug and retry' coding and a different one nudged its single-function correctness, does training on BOTH at once capture both gains — or were the two small wins just noise?

What we found

The obvious way to combine our two promising curricula - just train one model on both - backfired. We put the 504 self-repair rows and 504 WHY-comment rows into one 1008-row training set and trained a single adapter. Instead of getting both gains, we got neither: the WHY model's +5 HumanEval collapsed to +1, and the repair model's agentic 10/35 collapsed to 7/35 (basically base). This is a known trap (we hit it in the earlier menagerie work too): when you train one small model on a mixture, each ingredient gets half the concentration and washes out. So the complementary effects are real enough to conflict for capacity, but they don't ADD this way. The right way to combine two separately-trained specialists is in weight space - keep each at full strength and add their learned changes together (task vectors) - which is the cheap next test, no retraining needed.

Why it matters

This is the program's test of whether small, separate coding gains ADD UP. The mission is to install real multi-step coding ability into one small model without a bigger teacher. Four earlier attempts landed short of a clear win; two of them nudged different weaknesses. If stacking two believed-in but individually-weak ingredients produces a clear, confirmed gain, it validates a cheap, composable recipe for building capability a piece at a time. If it stays flat, it is honest evidence the earlier nudges were noise — which saves the team from stacking more of them.

Stack agentic7/35self_repair alone was 10/35 - diluted

Stack HumanEval+1why_comment alone was +5 - diluted

VerdictNULLmixture dilution - both effects washed out

Nexttask vectorscombine in weight space at full strength

On this page

Results at a glance
Overview
Report
Experiment log
Reproduce
Related

Results at a glance 1

The stack (corpus-union) DILUTES both effects — mixture dilution for coding Test of whether the two complementary positive curricula (self_repair -> agentic 10/35; why_comment -> HumanEval +5) COMBINE when trained as one adapter on the union of both corpora (1008 rows interleaved). Result: MIXTURE DILUTION. The union washed out BOTH effects - why_comment's +5 HumanEval collapsed to +1, self_repair's 10/35 agentic collapsed to 7/35 (at/below base). One adapter trained on both curricula at half concentration splits its capacity, the same mixture-dilution law the menagerie program found. Complementary effects DO NOT combine via corpus-mixing. The correct combination is weight-space at full magnitude - task vectors (base + repair_delta + why_delta) - tested next as a cheap no-training follow-on.

agentic count / HumanEval delta · arm →

Data table

arm	agentic (of 35)	HumanEval delta (problems)
base	8	0
self_repair	10	3
why_comment	8	5
stack (union)	7	1

Numbers from experiments/qwen35_4b_repair_why_stack/reports/measure_review.md

In the author’s words from the Overview · “Results”

When run, runs/measure/transfer_summary.json records all four pass@1 numbers (base/treatment x HE/MBPP, counts + fractions), the per-problem paired deltas, and the frozen, tightened verdict. Separate deployable evidence (transfer, a real code improvement because the grader ignores comments) from the retention guard. Construction facts already established (model-free): Union corpus sha 2462c93ea2a8dcfbd9413e1c6115ed1456ad438e5dabfdc01e924be6148ddbe5, 1008 rows, 504 self_repair + 504 why_comment, deterministically interleaved (shuffle seed 93570), reproducible from the two sha-pinned source copies. Source shas verified: self_repair 920cb228…, why_comment 040be350…. … Read the full result →

Overview

Research Program

Program: agentic_breadth_installation (cognitive-core coding sub-program)
Program question: can real, transferable coding capability be INSTALLED into base Qwen/Qwen3.5-4B by designed, contamination-free curricula, proven by transfer to held-out coding — and specifically, do the two individually weak-positive ingredients (self-repair loop behavior; why-comment causal reasoning) COMBINE to capture BOTH of their target-specific gains and clear significance?
Prior anchors: base coding baselines from experiments/qwen35_4b_coding_fitness_harness — HumanEval 76.2% (125/164, strong function coder), MBPP 56.5% (113/200), agentic duet-eval 23% (8/35, weak agent). Bet #1 (qwen35_4b_exec_trace_install, execution-tracing) was NULL. Bet #2 (qwen35_4b_self_repair_install, self-repair) was a WEAK POSITIVE on the LOOP target (agentic 8/35 -> 10/35, HumanEval +3). Bet #4 (qwen35_4b_why_comment_install, why-comment) was a WEAK POSITIVE on the FUNCTION target (HumanEval +5, agentic flat). The two help DIFFERENT targets.

Question

If self-repair (a loop behavior) and why-comment (causal reasoning) each produced a real but weak, target-SPECIFIC coding gain — repair on the agentic loop, why on per-function correctness — does STACKING them (training one fresh adapter on the UNION of the two committed curricula) capture BOTH gains (HumanEval ~+5 AND agentic ~10/35) and cross the significance line the individual bets could not?

Hypothesis

The two ingredients are COMPLEMENTARY, not redundant: why-comment teaches the 4B the generating reason of correct code (function correctness), self-repair teaches the detect-and-fix loop (multi-step agentic behavior). Because they target disjoint failure modes and neither regressed the other's target, their effects should ADD rather than interfere. If, after stacking, BOTH the HumanEval gain (~+5, from WHY) and the agentic gain (~10/35, from repair) appear, the stack works AND the individual weak signals are confirmed real. If the stack is flat, the two weak signals were likely noise. Honest prior on a MEANINGFUL install (>= 3-problem HumanEval/MBPP gain with retention): ~40% — above each parent's prior because we are combining two believed-in positive ingredients, but the double dose carries a real interference/forgetting risk and neither parent was individually significant.

Setup

Model: only Qwen/Qwen3.5-4B (rev 851bf6e8…); ONE fresh r32/a64 QLoRA adapter trained from the base_reserialized composite in a single stage.
Dataset/task source: data/sft_repair_why_stack.jsonl — the deterministic UNION of the two already-built, already-verified, already-committed source corpora, COPIED into data/source_corpora/ (sha-pinned) and combined by scripts/build_corpus.py (shuffle seed 93570): 504 self_repair rows + 504 why_comment rows = 1008 rows, deterministically interleaved (combined sha 2462c93e…). NO new generation.
Train/eval split: training is the spec -> #WHY:-commented correct code + spec -> repair-episode union; evaluation is the held-out spec -> code HumanEval (164) + MBPP (200) — deliberately disjoint surfaces, with the grader ignoring comments. The agentic duet-eval is the primary real target, run manually as a follow-on.
Baseline: base Qwen/Qwen3.5-4B on the same shared harness.
Controls: contamination firewall re-run on the UNION (whole-word banned-benchmark-name audit, 0 hits over 1008 rows; distinctive code n-gram overlap, 0 — inherited from the two clean parents and re-verified present-only); comments inert to the grader; the base composite authenticated fail-closed (tree + weights) before training.
Primary metric: greedy pass@1 on HumanEval + MBPP (shared fitness harness, experiments/qwen35_4b_coding_fitness_harness/scripts/eval_pass1.py, referenced not copied).
Oracle-only metrics: none gate here; the agentic duet-eval is a follow-on confirm, not part of this cell's frozen consequence.
Hidden-label boundary: the frozen, TIGHTENED two-directional consequence (INSTALLED_CODING requires a >= 3-problem gain / RETENTION_FAIL / NULL) is read once from the four pass@1 numbers; benchmarks are executed, never read as data.

Run

Smoke (no GPU, no writes):

python scripts/run.py --smoke

Rebuild the union corpus (deterministic, CPU-only; reproduces the combined sha):

python scripts/build_corpus.py              # writes corpus + receipt
python scripts/build_corpus.py --verify-corpus  # fail-closed verify only

GPU stages (each gated behind a staged adversarial review; see reports/preregistration.md for the exact commands and checkpoint order):

python scripts/run.py --stage train     # r32/a64, 4 epochs, seed 93571
python scripts/run.py --stage merge     # vendored composite merger
python scripts/run.py --stage measure   # shared HumanEval+MBPP harness, both arms

Agentic confirm (manual follow-on on the merged composite — the PRIMARY real target; base 8/35, self_repair 10/35, why_comment 8/35): run the duet-eval gen4 harness with --model-override set to large_artifacts/qwen35_4b_repair_why_stack/merged/repair_why_stack, exactly as bets #2 and #4's measure reviews documented for their composites.

Results

Not yet run. The train/merge/measure stages are gated behind staged reviews. When run, runs/measure/transfer_summary.json records all four pass@1 numbers (base/treatment x HE/MBPP, counts + fractions), the per-problem paired deltas, and the frozen, tightened verdict. Separate deployable evidence (transfer, a real code improvement because the grader ignores comments) from the retention guard.

Construction facts already established (model-free):

Union corpus sha 2462c93ea2a8dcfbd9413e1c6115ed1456ad438e5dabfdc01e924be6148ddbe5, 1008 rows, 504 self_repair + 504 why_comment, deterministically interleaved (shuffle seed 93570), reproducible from the two sha-pinned source copies.
Source shas verified: self_repair 920cb228…, why_comment 040be350….
Contamination on the UNION: 0 whole-word banned-benchmark-name hits over all 1008 rows; 0 distinctive shared 7-grams between the union's executable code and the benchmark code (present-only HF-cache aid; inherited-clean by set union of the two audited-clean parents).
53 unit tests green; run.py --smoke green; boundary drills refuse.

Interpretation

Pending the sealed measurement. INSTALLED_CODING makes the stack the program's reference composite and funds the agentic duet-eval confirm — the real test of whether the two weak ingredients' signals are additive and real. RETENTION_FAIL realizes the double-dose generation shift (reconsider dose/mix). NULL — no

= 3-problem gain — with a flat agentic confirm would price the two individual

weak signals as likely noise rather than additive real effects; a preserved boundary finding, not a re-roll.

Knowledgebase Update

Program evidence updated: pending measurement.
Program backlog updated: pending measurement.
Claim ledger updated: pending measurement.

Artifacts

scripts/build_corpus.py — deterministic UNION builder (source-sha verify + seeded interleave + banned-name audit + receipt); contamination.py; vendored train_think.py / merge_adapter.py; fail-closed train_trial.py; measure_transfer.py; run.py.
data/sft_repair_why_stack.jsonl — the 1008-row union + stack_corpus_receipt.json; data/source_corpora/ — the two sha-pinned source corpus copies; data/provenance/base_reserialized.json; data/contamination/.
configs/, reports/, tests/
reports/artifact_manifest.yaml

Report

Rendered from reports/report.md

Design-frozen report. The model-free construction (the union corpus, its deterministic build script + receipt, the contamination re-audit, the vendored pipeline, the tests, the lineage package) is complete and verified; the train/merge/measure GPU stages are gated behind staged adversarial reviews and have not run. Results will be appended to the Results section once the sealed measurement is read.

Summary

Lifecycle 36 — the STACK of the cognitive-core coding program's two positive coding bets. Bet #2 (self-repair, a LOOP behavior) was a weak positive on the agentic target (agentic 8/35 -> 10/35, HumanEval +3); bet #4 (why-comment, WHY causal reasoning) was a weak positive on the FUNCTION target (HumanEval +5, agentic flat). They are COMPLEMENTARY and target-specific, and neither regressed the other's target. This cell tests whether STACKING them — training one fresh r32/a64 LoRA on the UNION of the two already-committed 504-row curricula, deterministically interleaved into 1008 rows — captures BOTH gains (HumanEval ~+5 AND agentic ~10/35) and clears the significance the individual weak bets could not. It merges onto base and is measured for TRANSFER + RETENTION on the shared HumanEval + MBPP fitness harness under a frozen, TIGHTENED two-directional consequence (INSTALLED_CODING requires a >= 3-problem gain).

This is LEANER than a fresh curriculum: both source corpora are already built, verified, and committed. This cell COMBINES them (no new generation), so the union carries no contamination risk beyond that of its two parents — verified, not assumed. The why-comment rows are inert to execution grading, so a pass@1 gain is an unconfounded CODE improvement.

Research Program Fit

The program's target is the base 4B's agentic/multi-step cognition gap: HumanEval 76.2% (strong function coder) vs duet-eval 23% (weak agent). Bet #1 (a passive skill) reshuffled without raising; bet #2 (a loop behavior) nudged the agentic loop; bet #4 (causal reasoning) nudged per-function correctness. The two positive bets pointed at DISJOINT failure modes. This stack attacks the additivity question: does combining two believed-in, target-specific positive ingredients capture BOTH gains at once? HumanEval/MBPP serve as the fast transfer + retention signal; the agentic duet-eval (base 8/35, self_repair 10/35, why_comment 8/35) is the PRIMARY real target, run manually as a follow-on (not gated by this cell).

Method

Union build (scripts/build_corpus.py, shuffle seed 93570). The two source corpora are COPIED into data/source_corpora/ (sha-pinned) and combined by a deterministic, fail-closed builder: each source sha is verified against its pin BEFORE combining (abort on mismatch); the 504 self_repair + 504 why_comment non-blank JSONL lines are concatenated in the frozen order (self_repair, then why_comment) EXACTLY as their bytes appear (no re-serialization, so each row's encoding is preserved); the 1008 lines are then deterministically shuffled with random.Random(93570) so the two kinds INTERLEAVE. Combined sha 2462c93ea2a8dcfbd9413e1c6115ed1456ad438e5dabfdc01e924be6148ddbe5; the sha is a pure function of the two source shas + the shuffle seed and is verified stable across two independent rebuilds. data/stack_corpus_receipt.json documents the source shas, combine order, shuffle seed, final sha, and row count by kind (504 self_repair + 504 why_comment).
Contamination re-audit (scripts/contamination.py). The committed banned set of all 668 HumanEval + MBPP function names (663 after the language whitelist; name set byte-identical to both parents' fixtures) — zero whole-word hits over all 1008 rows (prompt + think + answer). A present-only code n-gram aid — zero distinctive shared 7-grams between the union's executable code (docstrings + comments stripped) and benchmark solution code; the union's code 7-grams are the union of the two clean parents' code 7-grams, so 0 by set union, re-verified over the combined corpus. No new generation -> no new contamination risk beyond the union.
Install (scripts/train_trial.py -> vendored scripts/train_think.py). One fresh r32/a64 adapter, 4 EPOCHS (the converged recipe for the high-entropy why-comment rows the union contains), lr 1e-5, batch 1, grad-accum 8, max-length 4096, w_think 0.2, w_close 0.2, seed 93571 (126 optimizer steps/epoch), from the base_reserialized composite. The base is authenticated FAIL-CLOSED (in-cell provenance copy + full tree manifest + full 9 GB weights hash) before training.
Merge (vendored scripts/merge_adapter.py) with --base-model = the base composite -> merged/repair_why_stack.
Measure (scripts/measure_transfer.py -> SHARED harness, referenced not copied). Base and repair_why_stack, HumanEval 164 + MBPP 200, greedy pass@1, identical vLLM path; all four numbers (counts + fractions) + per-problem paired deltas + the frozen, tightened verdict recorded. The grader ignores comments, so a pass@1 gain is an unconfounded CODE improvement.

Results

Pending the sealed measurement. runs/measure/transfer_summary.json will carry pass_at_1{base,repair_why_stack}{humaneval,mbpp}, the pass counts, the McNemar b/c paired deltas per dataset, and the frozen consequence. Deployable transfer evidence is a >= 3-problem pass@1 gain; the retention guard is the paired dataset staying within 0.02. The agentic duet-eval confirm (the LOOP direction) is the manual follow-on that decides whether both weak signals are additive and real.

Construction facts already established (model-free):

Union corpus sha 2462c93ea2a8dcfbd9413e1c6115ed1456ad438e5dabfdc01e924be6148ddbe5, 1008 rows, 504 self_repair + 504 why_comment, deterministically interleaved (shuffle seed 93570); reproducible from the two sha-pinned source copies.
Source shas verified: self_repair 920cb228…, why_comment 040be350….
Contamination on the UNION: 0 banned-name whole-word hits over 1008 rows; 0 distinctive shared 7-grams (present-only HF-cache aid ran; 61 shared spans, all pure control-flow idioms).
53 unit tests green (present-only cache aids RUN with the HF cache; the union is re-audited by kind, by banned name, and by distinctive n-gram); run.py --smoke green; boundary drills refuse.

Controls

Contamination firewall re-run on the UNION (banned-name audit + distinctive code n-gram overlap), both zero, so a benchmark movement cannot be memorization.
Comments are inert to the execution grader, so any pass@1 gain is a CODE gain, not a grader artifact.
Base composite authenticated fail-closed (tree + weights) before training and merge; a swapped composite aborts.
Identical measurement path for both arms (the shared harness), so base and treatment pass@1 are directly comparable.
Tightened consequence rule: a noise-level (<3-problem) bump reads NULL, fixing bet #1's letter-of-the-law false positive.
The union is a deterministic, source-sha-pinned combination (no new generation), so the training data is a verifiable function of the two audited parents.

Oracle Versus Deployable Evidence

Deployable evidence = a >= 3-problem HumanEval/MBPP pass@1 gain (real, held-out spec -> code generation, comments ignored by the grader). The retention guard (the other dataset within 0.02) is a control on the double-dose forgetting risk, not a capability claim. The agentic duet-eval is the eventual deployable target and the real test of the stack's LOOP direction, but is a manual follow-on confirm, not gated here. No metric here uses hidden labels beyond the one-shot frozen verdict read.

Interpretation

Pending measurement. INSTALLED_CODING: the two complementary ingredients combined; the stack becomes the program reference and funds the agentic duet-eval confirm — the real test of whether BOTH weak signals are additive and real. RETENTION_FAIL: the double-dose generation shift is realized; reconsider dose/mix. NULL — no

= 3-problem gain — with a flat agentic confirm prices the two individual weak

signals as likely noise rather than additive real effects; a preserved boundary finding, not a re-roll.

Next Experiments

If INSTALLED_CODING: run the agentic duet-eval confirm on the repair_why_stack composite; a confirmed agentic gain alongside the HumanEval gain makes the stack the program reference.
If RETENTION_FAIL: reconsider the dose / mix ratio before any confirm.
If NULL: do NOT re-roll this union; a flat stack (fast AND agentic) is strong evidence the two weak signals were noise — advance to the think-block WHY variant (bet #3, still queued) or a different mechanism.

Artifact Manifest

See artifact_manifest.yaml — the trained adapter and merged composite live under large_artifacts/ (omitted from git); the union corpus + build script + receipt, the two source corpus copies, the contamination fixture, the base provenance copy, and the receipts are in-repo and reproducibility-critical.

Experiment log 3

Show the running log (3 entries, 2026-07-18)

2026-07-18 — model-free construction frozen

Lifecycle 36, the STACK of the cognitive-core coding program's two positive coding bets. Mission: install real coding capability into base Qwen/Qwen3.5-4B via a designed, contamination-free curriculum, proven by transfer. Meta-context: bet #2 (self-repair, a LOOP behavior) was a WEAK POSITIVE on the agentic target (agentic 8/35 -> 10/35, HumanEval +3); bet #4 (why-comment, WHY causal reasoning) was a WEAK POSITIVE on the FUNCTION target (HumanEval +5, agentic flat). The cross-bet finding: the two are COMPLEMENTARY and target-specific, and neither regresses the other's target.

The bet: STACK the two positive ingredients — train one fresh r32/a64 LoRA on the UNION of the two already-committed 504-row curricula — and test whether the combined effect captures BOTH gains (HumanEval ~+5 from WHY AND agentic ~10/35 from repair) and clears the significance the individual weak bets could not. This is LEANER than a fresh curriculum: both source corpora are already built, verified, and committed; this cell COMBINES them, it does not regenerate. No new generation -> no new contamination risk beyond the union.

Built and verified (no GPU, no commit):

scripts/build_corpus.py (union-build shuffle seed 93570) — the deterministic, fail-closed UNION builder. Verifies BOTH source copies' sha against their pins BEFORE combining (self_repair 920cb228…, why_comment 040be350…), concatenates their 504 + 504 non-blank JSONL lines in the frozen order (self_repair, then why_comment) EXACTLY as their bytes appear (no re-serialization), then shuffles the 1008 lines with random.Random(93570) so the two kinds INTERLEAVE. Combined sha 2462c93ea2a8dcfbd9413e1c6115ed1456ad438e5dabfdc01e924be6148ddbe5; the sha is a pure function of the two source shas + the shuffle seed and is verified stable across two independent rebuilds. data/stack_corpus_receipt.json documents source shas, combine order, shuffle seed, final sha, and row count by kind (504 self_repair + 504 why_comment). --verify-corpus re-derives it twice in memory and fails closed against the committed corpus + receipt + pin (used by run.py --smoke).
data/source_corpora/sft_self_repair.jsonl + sft_why_comment.jsonl — the two COPIED, sha-pinned source corpora (the standalone reproduction inputs, per the standalone-experiments doctrine: lineage = copied ordered SFT datasets, not cross-experiment references).
scripts/contamination.py + data/contamination/banned_function_names.json (668 benchmark function names, 663 after whitelist; name set byte-identical to both parents' fixtures). Re-audited on the UNION: 0 whole-word hits over all 1008 rows (prompt + think + answer); 0 distinctive shared 7-grams between the union's executable code (docstrings + comments stripped) and benchmark code (present-only HF-cache aid; 61 shared spans, all pure control-flow idioms). The union's code 7-grams are the union of the two clean parents' code 7-grams, so 0 by set union.
Vendored scripts/train_think.py (sha e0eca2a2…) and scripts/merge_adapter.py (sha cb9af8b4…), byte-identical to the sibling cells' trainer/merger.
scripts/train_trial.py — fail-closed base authentication (in-cell provenance copy + tree manifest + full 9 GB weights hash); recipe r32/a64, 4 epochs, lr 1e-5, seed 93571 (126 optimizer steps/epoch). The 4-epoch recipe is inherited: the why_comment rows in the union are the high-entropy #WHY: target that undertrained at 1 epoch (why bet #4 was retrained at 4 epochs).
scripts/measure_transfer.py — invokes the shared fitness harness for both arms x both datasets; the grader IGNORES comments; frozen, TIGHTENED INSTALLED_CODING (>= 3-problem gain) / RETENTION_FAIL / NULL consequence, identical to bets #2/#4.
scripts/run.py — checkpointed --smoke | --stage train | --stage merge | --stage measure; each GPU stage gated behind a staged adversarial review.
53 unit tests green (present-only HF-cache aids RUN with the cache; the union is re-audited by kind, by banned name, and by distinctive n-gram; source shas verified; deterministic-shuffle 2-build identity; consequence-rule truth table; base auth fail-closed); run.py --smoke green; boundary drills refuse.

Grep-fresh note: union-build seed 93570 and training seed 93571 are fresh repo-wide as SEEDS (the only textual matches are incidental substrings inside unrelated floats). No training-seed collision.

Two-directional reading (frozen): the stack succeeds only if it captures BOTH targets. The fast HumanEval/MBPP gate tests the FUNCTION direction (the why_comment gain); the agentic duet-eval (base 8/35, self_repair 10/35, why_comment 8/35) tests the LOOP direction (the self_repair gain) and is the PRIMARY real target, run manually as a follow-on. If BOTH the HumanEval gain (~+5) AND the agentic gain (~10/35) appear, the stack works and the two individual weak signals are confirmed real; if flat, they were likely noise. Honest prior on a MEANINGFUL install: ~40%.

GPU stages (train/merge/measure) are pending their staged reviews.

2026-07-18 — Transfer measurement: MIXTURE DILUTION (naive stack fails)

Corpus-union stack vs base: HumanEval +1 (why_comment alone was +5), MBPP -3, agentic 7/35 (self_repair alone was 10/35). Frozen rule: NULL.
The union DILUTED BOTH component effects - one adapter on both curricula at half concentration splits capacity. Confirms the menagerie mixture-dilution law for coding: complementary effects do NOT combine via corpus-mixing.
CORRECT combination is weight-space at full magnitude: task vectors (base + repair_delta + why_delta). Testing that next (cheap, no training).

2026-07-18 — Task-vector combination ALSO fails (worse than dilution)

Tested the weight-space combination (base + (self_repair-base) + (why_comment-base), 128/723 tensors changed, full-magnitude both deltas): HumanEval 126/164 (+1, same as the union - WHY gain gone), agentic 3/35 (WELL BELOW base 8/35). Summing two full-strength deltas INTERFERES/overshoots, hurting agentic badly.
CONCLUSION: the two weak positives do NOT combine by ANY method - corpus-union DILUTES (both -> ~base), task-vector INTERFERES (agentic degrades to 3/35). Combined with neither being individually significant, this is strong evidence the individual effects (why_comment +5 HE, self_repair +2 agentic) are FRAGILE / at the noise floor. The SFT-curriculum thread has been thoroughly explored (6 bets) without a robust, combinable, significant coding gain.

Reproduce

Smoke test

python scripts/run.py --smoke

Full run

python scripts/run.py --stage train && python scripts/run.py --stage merge && python scripts/run.py --stage measure

Run steps are documented inside the experiment folder (README and scripts).

Browse the experiment folder on GitHub ↗

Qwen35 4B Repair + Why Stack

Results at a glance 1

Overview

Research Program

Question

Hypothesis

Setup

Run

Results

Interpretation

Knowledgebase Update

Artifacts

Report

Summary

Research Program Fit

Method

Results

Controls

Oracle Versus Deployable Evidence

Interpretation

Next Experiments

Artifact Manifest

Experiment log 3

2026-07-18 — model-free construction frozen

2026-07-18 — Transfer measurement: MIXTURE DILUTION (naive stack fails)

2026-07-18 — Task-vector combination ALSO fails (worse than dilution)

Reproduce

Related

Related experiments