Research log Small Model Experimentation
GitHub

Qwen35 4b Zero Root Lineage Rebuild

Finished2026-07-16Agentic Breadth InstallationPosttraining and AdaptationStructured Execution and CompilersGitHub ↗
The mystery ancestor priced: ninety percent of the win was the documented recipe

The one idea you need

The program's best model was built by six well-documented training steps stacked on top of one mystery ingredient: an old starter adapter nobody kept a recipe for. This experiment rebuilds the model using only the six documented steps, starting from a truly blank adapter, to see whether the mystery ingredient actually mattered.

The question

If the six documented training steps are replayed from a blank adapter (no undocumented starter), does the rebuilt model still match the original on the sealed ten-skill benchmark?

What we found

["The rebuild answered the provenance question with numbers. Retracing the six documented training steps from a truly blank starting adapter — same datasets, same seeds, same settings — produced a model with about ninety percent of the original's benchmark gain: 4.9 times the base score, seven of ten families strictly won, nothing lost. The undocumented ancestor adapter turns out to be worth about 0.04 aggregate, concentrated in exactly the three families that match its old training era — and it actually held BACK three other families, where the clean rebuild scores higher. So the formal verdict is 'degraded': the original's perfect sweeps lean on that ancestor's margin, and the clean-provenance upgrade is not available. But the honest headline is the reverse: the program's documented, contamination-free recipe does almost all of the work on its own, and a fully clean lineage extended with the proven skill-converter dose is now a mapped path.", ' ', '[', 'E', 'r', 'r', 'a', 't', 'u', 'm', ' ', '2', '0', '2', '6', '-', '0', '7', '-', '1', '6', ':', ' ', 's', 'w', 'e', 'e', 'p', '-', 'r', 'a', 't', 'e', ' ', 'f', 'r', 'a', 'm', 'i', 'n', 'g', 's', ' ', 'c', 'i', 't', 'i', 'n', 'g', ' ', '~', '5', '0', '%', ' ', 'r', 'e', 'f', 'l', 'e', 'c', 't', 'e', 'd', ' ', 'a', ' ', 'f', 'a', 'v', 'o', 'r', 'a', 'b', 'l', 'e', ' ', 'w', 'i', 'n', 'd', 'o', 'w', ';', ' ', 't', 'h', 'e', ' ', 'a', 'l', 'l', '-', 'e', 'v', 'e', 'n', 't', 's', ' ', 'r', 'a', 't', 'e', ' ', 'i', 's', ' ', '2', '/', '6', ' ', '—', ' ', 's', 'e', 'e', ' ', 't', 'h', 'e', ' ', 's', 'w', 'e', 'e', 'p', '-', 'r', 'a', 't', 'e', ' ', 'c', 'o', 'n', 's', 'o', 'l', 'i', 'd', 'a', 't', 'i', 'o', 'n', ' ', 'c', 'e', 'l', 'l', '.', ']']

Why it matters

The headline result rests on a model whose first ingredient has no recorded origin. If the rebuild matches, the whole result becomes reproducible from committed recipes end-to-end; if it degrades, the program finally learns how much it owes to an ingredient it cannot reproduce.

Zero-root aggregate0.3462vs original 0.3824, base 0.0713
Transfer carried by documented stages~90%7/10 strict wins, zero losses
The ancestor's worth~0.036concentrated in sirens/rites/mirage
Families the ancestor suppressed3chronicle, siftstack, stockade
On this page
  1. Results at a glance
  2. Overview
  3. Report
    1. Summary
    2. Research Program Fit
    3. Method
    4. Results
    5. Controls
    6. Oracle Versus Deployable Evidence
    7. Interpretation
    8. Next Experiments
    9. Artifact Manifest
  4. Experiment log
  5. Data files
  6. Reproduce
  7. Related

Results at a glance 1

Original versus clean-rebuild scores on the ten families

How to read

Grouped bars per family: base, the original model, and the zero-root rebuild on the same sealed seed.

00.20.40.60.8chroniclechronicle00.40.6lockpicklockpick00.10.1mendersmenders000miragemirage00.80.5ritesrites00.20siftstacksiftstack00.50.6sirenssirens0.40.70.4stockadestockade0.0970.1570.355toolsmithtoolsmith0.1170.80.74warrenwarren0.10.1670.167

Takeaway → The rebuild matches or beats the original on seven families; the ancestor's edge lives in three.

Data table
public benchmark familybasehygiene_explore_originalzero_root_rebuild
chronicle00.40.6
lockpick00.10.1
menders000
mirage00.80.5
rites00.20
siftstack00.50.6
sirens0.40.70.4
stockade0.0970.1570.355
toolsmith0.1170.80.74
warren0.10.1670.167

Numbers from experiments/qwen35_4b_zero_root_lineage_rebuild/runs/benchmark/zero_root_readout.json

Technical framing

Per-family scores at medium tier, sealed seed 78159 (tb 1024) — ZERO_ROOT_DEGRADED, mildly: the six documented contamination-free stages replayed from a fresh zero-initialized adapter carry ~90% of the transfer (0.3462 vs the original's 0.3824 over base 0.0713; 7/10 strict wins vs base with ZERO losses). The undocumented C53-era prefix contributes ~0.036 aggregate concentrated in sirens/rites/mirage — coherent with its gym-era profile — and SUPPRESSED chronicle/siftstack/stockade where the rebuild wins. The original read 9/10 on this seed (menders tie, the fifth data point on the ~50% sweep rate). The provenance scope is now a recorded number; a zero-root lineage + the statechain converter is the mapped clean path.

In the author’s words from the Report · “Summary”

The provenance question closed with a measured answer: ZERO_ROOT_DEGRADED, mildly. Replaying the six documented contamination-free stages from a fresh zero-initialized adapter (same datasets, seeds 42/43/44/47/51/55, trainer variants) produces a composite at 0.3462 aggregate versus the original's 0.3824 over base 0.0713 on the same sealed seed — ~90% of the transfer, 7/10 strict family wins, zero losses — while the undocumented C53-era prefix contributes ~0.036 aggregate concentrated in sirens/rites/mirage and actually suppressed chronicle/siftstack/stockade. The original read 9/10 on this seed (menders tie). … Read the full result →

Overview

Lifecycle 22 — the provenance question elevated to the program's strongest remaining bet: the six documented contamination-free training stages of the hygiene_explore composite, replayed from a FRESH zero-initialized adapter on the official base (removing the undocumented C53-era gym-line root adapter), then measured ONCE at medium against the original.

Research Program

  • Program: agentic_breadth_installation
  • Program question: can synthetic curricula install general capability that lifts the held-out aggregate without a negative family — and is the demonstrated position carried by the DOCUMENTED stages alone?
  • Prior anchors: the goal-gate confirmation (AGGREGATE_ONLY; two 10/10 sweeps across four sealed medium seeds 78,154/78,157; aggregate 4/4 at ~0.33-0.38 vs base ~0.06-0.11); the standalone lineage package committed in that cell, whose root adapter carries a HARD provenance boundary (no committed creation receipt anywhere in the repo); the C53-era record that the blend root alone carried ~0.44 quick aggregate — load-bearing, but HOW load-bearing at medium is unmeasured.

Question

The original composite's lineage is carried by ONE warm-started LoRA adapter: [undocumented C53 'blend' root] → replay_refresh(42) → designed160(43) → close_xi(44) → replay_after_close(47) → designed_fresh(51) → hygiene_explore(55). Every stage is documented, contamination-free, and byte-reconstructable — except the root. If the six documented stages are replayed from a FRESH zero-initialized adapter (same datasets, same fixed seeds, same trainer variants, same hyperparameters, raw pinned HF base everywhere), does the result still carry the demonstrated medium position, or is the undocumented prefix load-bearing?

Hypothesis

The six stages total ~6,700 training rows over ~3 epochs-equivalent of designed curricula; the mechanism they install (think-channel structure + close-weight shaping) is what the goal-gate events measured. If that is right, the zero-root replay lands within one family of the original on the same seed (ZERO_ROOT_COMPARABLE) and the headline model is contamination-clean end-to-end. If the gym-era root's ~0.44 quick aggregate was carrying medium capability the stages merely preserved, the zero-root arm degrades and the recorded contrast IS the root's contribution.

Setup

  • Model: Qwen/Qwen3.5-4B @ 851bf6e8… (raw pinned HF revision for every training stage and the final merge; never a merged composite as training base).
  • Stage replays: scripts/rebuild_zero_root.py — stage 1 trains a FRESH rank-32/alpha-64 adapter (trainer's default fresh path, NO --warm-start; LoRA-B zero-init so the delta starts at zero) on stage01_replay_refresh.jsonl at seed 42 with the exact recorded stage-1 hyperparameters (lr 1e-5, batch 1, accum 8, maxlen 4096, epochs 1, w_think 0.2, NO w_close, stage-1/2 trainer variant); stages 2-6 warm-start each from the PREVIOUS zero-root stage with their recorded per-stage recipes (stage 3: train_think_close with target_close_kinds=[u_execute,u_induct], target_w_close=1.0). Per-stage receipts → runs/lineage/stageNN_<name>.json; the manifest's recorded adapter hashes are CONTRAST fields only (different root ⇒ different bytes, never verification).
  • Lineage package: byte-identical copy of qwen35_4b_goal_gate_confirmation's committed package (manifest sha 1f49cd8b…, six datasets, three trainers, merger). The blend root is deliberately NOT vendored — its omission IS the design, and the receipt/tests fail closed if a copy appears under this cell's artifact storage.
  • Merge: stage-6 zero-root adapter onto the raw base via the copied merger → large_artifacts/qwen35_4b_zero_root_lineage_rebuild/merged/zero_root_hygiene_explore; receipt runs/lineage/merge.json pins adapter shas, full output tree sha, weights sha.
  • Benchmark (ONE event): medium, tb 1024, ONE fresh sealed seed 78,159 (grep-fresh audited in the design receipt), THREE arms in frozen order: base (b654e033…/26d8ee48…), hygiene_explore_original (e2112344…/9eb653d7…), zero_root_hygiene_explore (pinned post-merge via fail-closed TODO-pins in scripts/run_benchmark.py, filled from the committed merge receipt). Hardened single-seed runner: verdict + design-receipt code-pin check at the seed-consuming boundary, write-ahead opened/closed ledger whose closed record sha-pins the summary AND all three per-arm receipts, byte-equal crash reconciliation, clean-slate unopened seed, finiteness guards, implementation signature anchored to the discovery/confirmation block.
  • Training seeds 42/43/44/47/51/55 are INHERITED STAGE CONSTANTS (deliberate reuse — they are what "same recipe" means); the only fresh seed in this cell is 78,159.
  • Readings (no promotion): per-family table + aggregates for all three arms; goal gate vs base for BOTH composites (forensics-identical strict-win partition); the PREFIX CONTRIBUTION contrast (zero-root minus original, per family and aggregate — "the gym-era root's contribution at medium, one seed, cross-arm same-seed paired"); budget integrity; menders/rites/warren margins (the statechain→rites conversion question does not apply — no statechain stage).
  • Consequence (ordered, total): ZERO_ROOT_COMPARABLE iff the zero-root aggregate strictly beats base AND its goal-gate strict wins ≥ (original's strict wins on this seed − 1) — "the documented stages alone carry the demonstrated position; the headline model is contamination-clean end-to-end"; ZERO_ROOT_DEGRADED otherwise — "the undocumented prefix is load-bearing at medium; its contribution is the recorded contrast".
  • Hidden-label boundary: nothing under benchmarks/ is ever read; gateway receipts only.

Run

Smoke (no GPU, no writes):

PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -B experiments/qwen35_4b_zero_root_lineage_rebuild/scripts/run.py --smoke

Stages (each requires its committed-at-HEAD prerequisites + literal review verdict on clean pushed main):

# ~2.5-3h GPU: six stage replays + the merge; commit runs/lineage/ after
.venv/bin/python -B experiments/qwen35_4b_zero_root_lineage_rebuild/scripts/run.py --stage rebuild   # needs PASS_REBUILD in reports/compute_review.md
# fill the three TODO-pins in scripts/run_benchmark.py from runs/lineage/merge.json, commit, review, then:
.venv/bin/python -B experiments/qwen35_4b_zero_root_lineage_rebuild/scripts/run.py --stage benchmark # needs PASS_BENCHMARK_EVENT in reports/benchmark_design_review.md

Results

Pending: the rebuild and benchmark stages have not run. The terminal artifact will be runs/benchmark/zero_root_readout.json with the frozen consequence.

Interpretation

Pending the sealed event.

Knowledgebase Update

  • Program evidence updated: pending.
  • Program backlog updated: this cell IS the queued zero-root rebuild from the goal-gate confirmation's backlog.
  • Claim ledger updated: pending.

Artifacts

  • data/design_receipt.json: the frozen design — package pins, stage plan (zero-root rewiring), root-omission block, seed-78,159 freshness audit, arm pins, consequence partition, code pins. run_benchmark.py is pinned by a NORMALIZED HASH: exactly the three TODO-pin value slots are canonicalized to a fixed placeholder before hashing, so every other byte (every guard call site included) is frozen pre- and post-fill; the digest and the normalization rule live in the receipt and are re-verified at the seed-consuming boundary.
  • data/lineage/: byte-identical copies of the manifest + six stage datasets.
  • scripts/lineage_trainers/, scripts/merge_adapter.py: byte-identical trainer/merger copies.
  • scripts/rebuild_zero_root.py: the six-stage zero-root replay + merge (receipts to runs/lineage/).
  • scripts/run_benchmark.py, scripts/check_benchmark.py: the hardened single-seed three-arm event and its provenance-anchored readout.
  • reports/artifact_manifest.yaml: external artifact declarations (adapters + merged composite live under large_artifacts/).

Erratum (2026-07-16)

The sweep-rate framing in this document ("two full sweeps across four independent sealed seeds", ~50%) reflects the 78,154–78,157 window and omits the earlier 78,150 reading (8/10, menders+rites ties). Over ALL six recorded goal-gate readings the rate is 2/6 (exact 95% CI [0.04, 0.78]), with menders blocking every miss. See experiments/qwen35_4b_sweep_rate_consolidation for the consolidated record; the per-seed facts in this document are unchanged.

Report

Rendered from reports/report.md

Summary

The provenance question closed with a measured answer: ZERO_ROOT_DEGRADED, mildly. Replaying the six documented contamination-free stages from a fresh zero-initialized adapter (same datasets, seeds 42/43/44/47/51/55, trainer variants) produces a composite at 0.3462 aggregate versus the original's 0.3824 over base 0.0713 on the same sealed seed — ~90% of the transfer, 7/10 strict family wins, zero losses — while the undocumented C53-era prefix contributes ~0.036 aggregate concentrated in sirens/rites/mirage and actually suppressed chronicle/siftstack/stockade. The original read 9/10 on this seed (menders tie). The headline model's recorded sweeps lean on the prefix margin, so the contamination-clean upgrade is not available; every prior reading involving this lineage now carries a recorded, quantified scope instead of a suspicion.

Research Program Fit

agentic_breadth_installation — the provenance question elevated to a measurement. The goal-gate confirmation demonstrated the position (AGGREGATE_ONLY; two 10/10 sweeps across four sealed seeds) but the composite's lineage root has no committed creation receipt. If the documented stages alone carry the position, the headline model is contamination-clean end-to-end; if not, the recorded contrast IS the root's contribution.

Method

  • Byte-identical copied lineage package (manifest 1f49cd8b…, six datasets, three trainers, merger); the blend root deliberately NOT vendored — its omission is the design and fails closed if present.
  • rebuild_zero_root.py: stage 1 fresh rank-32/alpha-64 (no --warm-start, LoRA-B zero-init) at seed 42 with the exact recorded stage-1 recipe; stages 2-6 warm-start from the previous zero-root stage with their recorded recipes (stage 3 targeted close overrides). Per-stage receipts committed to runs/lineage/; the original chain's adapter hashes recorded as CONTRAST fields only.
  • Merge of the stage-6 zero-root adapter onto the raw base via the copied merger; the merge receipt pins the full output tree.
  • ONE sealed benchmark event: medium, tb1024, seed 78159 (grep-fresh audited), three arms in frozen order (base, hygiene_explore_original, zero_root_hygiene_explore), single-seed write-ahead ledger whose closed record sha-pins the summary and all three gateway receipts, TODO-pinned zero-root arm filled post-merge, implementation signature anchored to the discovery/confirmation block.
  • Frozen consequence (ordered, total): ZERO_ROOT_COMPARABLE iff the zero-root aggregate strictly beats base AND its goal-gate strict wins >= original's strict wins on this seed − 1; ZERO_ROOT_DEGRADED otherwise.

Results

PENDING — terminal artifact will be runs/benchmark/zero_root_readout.json.

Controls

  • Same-seed cross-arm pairing: all three arms run the identical sealed seed, tier, and think budget in one event.
  • The original composite arm is the exact published tree the goal-gate events measured (tree 9eb653d7…, weights e2112344…).
  • The base arm is the frozen reserialized base (tree 26d8ee48…).
  • Budget integrity is a reading, never a gate; any over-budget arm invalidates the paired comparison scope, scores still recorded.

Oracle Versus Deployable Evidence

No oracle anywhere: the measurement is the deployable benchmark surface through the trusted gateway (receipts only; nothing under benchmarks/ is ever read).

Interpretation

PENDING the sealed event.

Next Experiments

  • If ZERO_ROOT_COMPARABLE: the zero-root composite becomes the contamination-clean headline lineage; the menders dose-scale intake proceeds on a clean base.
  • If ZERO_ROOT_DEGRADED: the recorded prefix-contribution contrast becomes the program's measurement of the gym-era root; decide whether to reconstruct the root's function via documented curricula.

Artifact Manifest

See artifact_manifest.yaml — adapters and the merged composite live under large_artifacts/ (receipts committed in runs/lineage/); the complete stage-replay package is carried in-repo.

Experiment log 6

Show the running log (6 entries, 2026-07-16)

2026-07-16 — Model-free design freeze

  • Opened as the map-completion's strongest remaining bet and the owner's provenance question made measurable: the six documented stages replayed from a zero root (the undocumented C53-era blend adapter omitted fail-closed as the treatment), merged, and measured once at medium beside the original at sealed seed 78,159.
  • The lineage package copied byte-identically; the stage plan test-pinned against the manifest with the chain rewired to the zero root; the hardened single-seed runner carries the receipt-pinned ledger and the discovery-pinned implementation signature. 84 tests green; smoke green.

2026-07-16 — Review fix: normalized-hash pin for run_benchmark.py

  • The adversarial review confirmed a MAJOR, mutation-verified live: the original substring-contract mechanism pinned 23 constants/def-lines but ZERO call sites — deleting require_todo_pins_filled(), require_verdict(...), the require_clean_pushed_main block, append_ledger(opened_record()), the ledger_plan call, or neutralizing require_zero_root_provenance() left gen --check green. A drifted runner could have consumed sealed seed 78159 with no verdict gate, no pin refusal, and no ledger.
  • Fix (the stronger option): a NORMALIZED-HASH code pin. A deterministic regex canonicalizes exactly the three TODO-pin VALUE slots — the two "zero_root_hygiene_explore": None,-shaped dict entries and the ZERO_ROOT_MERGE_RECEIPT_SHA256 = None constant, each also matching the post-fill quoted 64-hex — to the fixed placeholder __ZERO_ROOT_TODO_PIN__ (fail-closed on any slot-count mismatch), and the sha256 of the canonicalized bytes is frozen as RUN_BENCHMARK_NORMALIZED_SHA256 = a2d87408efe346a9… in gen_design_receipt.py and in the receipt (run_benchmark_normalized_pin block: digest + the full normalization rule). Every byte outside the three value slots is byte-frozen pre- and post-fill; six call-site substring contracts remain as belt-and-braces diagnostics only. Runner and generator docstrings updated to state the mechanism exactly.
  • Verified: receipt regenerated (sha 7b822095…), --check twice byte-identical; LIVE drills — deleting the require_todo_pins_filled() call and the require_verdict(...) call each made --check refuse (exit 2, normalized-hash mismatch), and simulating the post-merge pin fill with real 64-hex values PASSED --check byte-identically; 13 new unit tests (96 total, green) cover None-vs-filled hash equality, all six reviewer mutations as regressions, one-byte drift anywhere, and fail-closed slot drift; smoke green; stage refusals intact.

2026-07-16 — Review: the pin mechanism hardened; rebuild authorized

  • The runner lens mutation-drilled the deferred substring contracts and proved them vacuous for control flow (deleting the verdict gate and pin refusal left every check green). Replaced with the normalized-hash pin: three canonicalized fill slots, everything else byte-frozen, the six mutations now regression tests. Fidelity lens clean. 96 tests green; PASS_EXPENSIVE_RUN and PASS_REBUILD granted.

2026-07-16 — The six-stage replay: two honest bumps, then clean

  • Stage 1 completed on the first invocation (42 min; the fresh zero-root adapter trains slower than warm continuations). Stage 2 then OOMed at step 78/190 (CUDA fragmentation under the gauntlet-era trainer variant); the rebuild script fail-closed on the orphaned empty output dir as designed. Resumed via direct rebuild invocation (stage 1 skipped by verified receipt) with PYTORCH_CUDA_ALLOC_CONF= expandable_segments:True — an allocator setting, not a training hyperparameter; recorded here for the reproduction path.
  • Stages 2–6 and the merge then ran clean: walls 23/5/5/22/22 min; zero-root composite tree 414f5829…, weights 6e9aad25…, merge receipt e906caea…. The three runner pins were filled from the committed merge receipt and the normalized-hash pin verified byte-stable post-fill (gen --check exit 0) — the review-installed mechanism working as designed.

2026-07-16 — Post-fill fixture repair (test-only)

  • One tamper-fixture in the normalized-pin test suite hardcoded the pre-fill = None slot form and broke after the legitimate post-merge pin fill; the previous commit's smoke failure was this and was masked by a command chain — recorded honestly. Fixed test-only with a _none_baseline() canonicalizer so the mutation fixtures are fill-state-agnostic; no pinned code changed (the normalized hash and the design receipt are untouched; gen --check still passes). 96 tests green; smoke green.

2026-07-16 — The event and closure: the prefix priced

  • The three-arm event at sealed 78,159 ran clean end-to-end. Verdict ZERO_ROOT_DEGRADED per the frozen rule (7 strict wins vs the required 8): the documented stages alone carry ~90% of the transfer (0.3462 vs 0.3824 over base 0.0713; zero losses), and the undocumented prefix's contribution is ~0.036 aggregate concentrated in sirens/rites/mirage — coherent with its gym-era profile. The zero-root model beats the original on chronicle/siftstack/stockade (the prefix also suppressed capability). The original read 9/10 (menders tie), the fifth data point on the ~50% sweep rate.
  • The provenance scope is now a recorded number, not a suspicion; the contamination-clean upgrade is not available; a zero-root lineage extended with the statechain converter is the clean future line.

Data files 1

Result tables and metrics copied from the experiment folder — preview inline or open the raw file.

Reproduce

Smoke test

PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -B experiments/qwen35_4b_zero_root_lineage_rebuild/scripts/run.py --smoke

Full run

.venv/bin/python -B experiments/qwen35_4b_zero_root_lineage_rebuild/scripts/run.py --stage rebuild && ... --stage benchmark (each behind its committed review verdict)

Run steps are documented inside the experiment folder (README and scripts).

Browse the experiment folder on GitHub ↗