State-Formation Branch Handoff Recovery
The one idea you need
A retired error slot is like a reserved parking space: once a verified replacement arrives, later checks must recognize the new vehicle without forgetting the old incident.
The question
Can later full-size setup work continue after a successful replacement occupies the file location that previously held a preserved failure?
What we found
Yes, the narrow handoff passed every safety check, then all three full-size setup controls reached 48 of 48 with their learned path and zero of 48 with that path disabled.
Why it matters
Practitioners can continue the promised compact-versus-full-size comparison without rewriting published history or weakening the original safety checks. Later work still waits for each completed setup checkpoint to be published.
On this page
Results at a glance 1
How to read
The six horizontal groups cover the false rejection, retirement, failure archive, missing retired copy, successful setup record, and invocation history. Bar height is the pass indicator; higher is better.
Takeaway → All six bars reach the pass mark before model work, showing the handoff recognizes the verified replacement without losing failure safeguards.
Data table
| frozen handoff control | control pass |
|---|---|
| false rejection reproduced | 1 |
| retirement exact | 1 |
| failure archive exact | 1 |
| retired mirror absent | 1 |
| successful G0 exact | 1 |
| G0 invocation exact | 1 |
Numbers from experiments/qwen35_4b_state_formation_branch_handoff_recovery/runs/smoke.json
Technical framing
All six handoff lineage controls pass before downstream model work — Operational evidence only. Smoke loads no model, starts no training/evaluation, and opens no benchmark or sealed contrast.
In the author’s words from the Overview · “Results”
Unit controls additionally reject exact failed bytes at the successful slot, any changed successful-G0 bytes, a reappearing mirror, and unregistered invocation shapes. Smoke loaded no model, started no training/evaluation, opened no benchmark or contrast path, and changed no scientific interpretation. No positive-control or downstream scientific result is claimed here. At that checkpoint, use remained blocked until publication and both workflows were green; the later paragraphs record that completed progression. Handoff commit 627254f1…d892 passed both workflows. The exact full-rank seed-7411 positive control then completed once through this wrapper. … Read the full result →
Overview
This operational successor lets immutable producer v11 continue after the first branch recovery successfully replaces the retired failed-G0 receipt at its canonical path. It changes no scientific arm, model, data, metric, threshold, or branch rule.
The handoff question is answered and the setup matrix has passed. Continued Stage-B training through this frozen wrapper belongs to the canonical producer and does not make this recovery experiment active.
Research Program
- Program:
structured_execution_and_compilers. - Direct producer:
qwen35_4b_state_formation_capacity_adjudication. - Closest experiment:
qwen35_4b_state_formation_branch_recovery, whose exact-prefix seam produced the valid full-rank seed-7411 G0 but whose frozen retirement guard treats any later occupant of that pathname as the retired failure.
Question
Can downstream producer stages continue without weakening failure preservation once a successful recovered G0 legitimately occupies the canonical pathname formerly used by the archived failure?
Hypothesis
The defect is limited to invocation orchestration: the first wrapper asks whether the pathname exists, not whether it still contains the exact retired failure. A new wrapper can reuse its frozen, tested path seam while requiring the archived failure and retirement receipts, rejecting a reappearing mirror or exact failed bytes, and accepting only the exact successful G0 plus its STARTED/COMPLETE lineage.
Setup
- Model-bearing stages remain only
Qwen/Qwen3.5-4Brevision851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a, on producer v11's Transformers backend. - Smoke loads no model and runs no training or evaluation.
- Successful G0: file SHA-256
cdc90cd…c68f, identitye1f1c906…f89dc, statusMODEL_SMOKE_PASS, full-rank joint seed 7411. - Retired failure: SHA-256
47305826…2c71; it must remain in the first recovery archive and must not reappear at either producer failure path. - First recovery: source contract
55d0a455…56f3, smoke SHA-2568bf5bb36…6849. - Allowed stages and argument shapes remain the first recovery's exact registered set. The producer CLI retains canonical path, source snapshot, authorization, setup/control, training, evaluation, lock, and receipt gates.
- Hidden-label boundary: smoke opens no contrast or benchmark payload. The wrapper cannot analyze.
Recovery boundary
The wrapper imports the exact frozen first-recovery source and reuses its safe reads, immutable writes, producer-context checks, argument-shape policy, stage-specific output validation, and src.analysis._canonical_expected_path context manager. It does not call the first wrapper's pathname-only invoke_producer; it owns new immutable STARTED/COMPLETE receipts and directly calls the same exact producer CLI under the same path seam.
Every invocation first rehashes the retirement, archived failure, successful G0, first-recovery G0 STARTED/COMPLETE receipts, authorization, first-recovery smoke/source, producer source/model, and the handoff's own frozen smoke/source. A producer output cannot be adopted unless its handoff STARTED receipt already exists.
Run
Freeze and smoke without loading the model:
.venv/bin/python -B -m unittest discover \
-s experiments/qwen35_4b_state_formation_branch_handoff_recovery/tests -v
.venv/bin/python -B \
experiments/qwen35_4b_state_formation_branch_handoff_recovery/scripts/run.py --smokeCommit and push the successful G0, its first-recovery invocation receipts, and this frozen handoff; require both repository workflows green. Then retry the seed-7411 positive control through this wrapper. Every later branch-authorized model stage uses the same handoff wrapper; producer analysis continues through the already-frozen analysis recovery.
Results
The focused suite passes 8/8. The frozen smoke emitted BRANCH_HANDOFF_RECOVERY_SMOKE_PASS at file SHA-256 0e0409de…6e5b, receipt identity d6fecf0a…de4c, and source contract 4d2ffde3…8acd. All six controls pass: the first wrapper's pathname-only false rejection is reproduced; retirement, failure archive, successful G0, and its invocation lineage are exact; and the retired mirror remains absent. Unit controls additionally reject exact failed bytes at the successful slot, any changed successful-G0 bytes, a reappearing mirror, and unregistered invocation shapes.
Smoke loaded no model, started no training/evaluation, opened no benchmark or contrast path, and changed no scientific interpretation. No positive-control or downstream scientific result is claimed here. At that checkpoint, use remained blocked until publication and both workflows were green; the later paragraphs record that completed progression.
Handoff commit 627254f1…d892 passed both workflows. The exact full-rank seed-7411 positive control then completed once through this wrapper. Producer file SHA-256 is a0d17e2e…7a16, producer receipt identity 6708a4d4…9649, and handoff COMPLETE identity af0dd15c…2262. The producer control passed oracle and intact joint state at 48/48, adaptation-disabled joint state at 0/48, exactly 256 updates, gradient accumulation 16, and 4,096 singleton presentations. Both the 892,272,640-parameter full- rank path and shared-state parameters changed; early stopping and checkpoint selection were false. The receipt authorizes later producer work only through its unchanged barriers and records zero result payload, benchmark, contrast, or scientific-evidence access.
After seed-7411 setup commit c507488c…3c06 passed both workflows, seed 7412 independently passed the same path. Its G0 file SHA-256 / identity are 10bf22fc…1d18 / 62ecb79e…951dc: zero K=1 and checkpoint-roundtrip error, all 62 full-rank deltas with complete finite optimizer state, all joint trainable groups finite/nonzero, finite K=12, and 22.1 GiB free. Its positive-control file SHA-256 / identity are 1cbbd823…8510 / 6575e1d2…4554: oracle/intact 48/48 and disabled 0/48 after exact 256 updates, accumulation 16, and 4,096 presentations. Handoff G0/control COMPLETE identities are d61d6441…9246 and 6334d72f…c14a. Both receipts retain zero result, benchmark, contrast, and scientific-evidence access.
After seed-7412 setup commit 92473a53…8530 passed both workflows, seed 7413 completed the matrix. Its G0 file SHA-256 / identity are 021a8444…d635 / 4d2316d3…2ff0: zero K=1 and round-trip error, finite/nonzero joint gradients, complete finite optimizer state for all 62 deltas, finite K=12, and 22.1 GiB free. Its control file SHA-256 / identity are 8a4af0d6…fde8 / 976f28ef…14df: oracle/intact 48/48 and disabled 0/48 after exact 256 updates, accumulation 16, and 4,096 presentations, with changed full-rank/shared state. Handoff G0/control COMPLETE identities are fbfb282f…5aef / ccff1520…58c9. All three pairs now pass, remain setup-only, and open no result, benchmark, contrast, or scientific evidence.
Interpretation
The three valid G0/control pairs establish full-rank feasibility and a working state path at every seed, not a capacity result. This recovery safely handed producer authority through setup without changing scientific logic. Capacity interpretation remains unchanged.
Knowledgebase Update
- Program evidence: no scientific evidence from an invocation repair.
- Program backlog: producer Stage B now owns the mandatory LoRA-state-only and full-rank-joint work.
- Claim ledger: unchanged.
Artifacts
idea_intake.md: nearest-neighbor and novelty decision.reports/design_review.md: adversarial review and stop conditions.configs/default.yaml: exact producer, recovery, failure, G0, and authorization pins.runs/smoke.json: frozen no-model handoff smoke.runs/invocations/: handoff-owned immutable producer invocation provenance.reports/artifact_manifest.yaml: producer-owned large-artifact boundary.
Report
Rendered from reports/report.md
Status
Finished. The first recovery produced a valid full-rank seed-7411 G0, but its frozen pathname-only retirement guard stopped the following positive control before producer or model work. This handoff then carried all three full-rank G0/control pairs through the unchanged producer barriers. Producer receipts remain scientifically authoritative.
Method
Pin the archived failure, retirement, successful G0, first-recovery G0 invocation lineage, first- recovery source/smoke, producer source/model, and authorization. Reuse the exact frozen path seam while replacing only outer invocation orchestration and emitting handoff-owned STARTED/COMPLETE receipts.
Results
- Focused tests: 8/8.
- Smoke status:
BRANCH_HANDOFF_RECOVERY_SMOKE_PASS. - Smoke file SHA-256 / identity:
0e0409de…6e5b/d6fecf0a…de4c. - Frozen source contract:
4d2ffde3…8acd. - Six smoke controls pass; unit controls reject failed bytes, changed G0 bytes, a reappearing mirror, and unregistered invocation shapes.
- Model load, training/evaluation, benchmark/contrast access, and scientific interpretation: zero.
- Full-rank seed-7411 positive control: producer SHA-256
a0d17e2e…7a16, identity6708a4d4…9649; oracle/intact 48/48, adaptation-disabled 0/48, 256 updates, accumulation 16, 4,096 presentations, and changed full-rank/shared-state parameters. - Handoff COMPLETE identity:
af0dd15c…2262; producer result/benchmark/contrast access and scientific evidence remain zero. - Full-rank seed-7412 G0: file SHA-256
10bf22fc…1d18, identity62ecb79e…951dc; zero K=1/ round-trip error, complete finite optimizer state for all 62 deltas, finite K=12, 22.1 GiB free. - Full-rank seed-7412 control: file SHA-256
1cbbd823…8510, identity6575e1d2…4554; oracle/intact 48/48, disabled 0/48, 256 updates, accumulation 16, 4,096 presentations, changed full-rank/shared state, and zero result/benchmark/contrast/scientific access. - Full-rank seed-7413 G0: file SHA-256
021a8444…d635, identity4d2316d3…2ff0; zero K=1/ round-trip error, complete finite optimizer state for all 62 deltas, finite K=12, 22.1 GiB free. - Full-rank seed-7413 control: file SHA-256
8a4af0d6…fde8, identity976f28ef…14df; oracle/intact 48/48, disabled 0/48, 256 updates, accumulation 16, 4,096 presentations, changed full-rank/shared state, and zero result/benchmark/contrast/scientific access. - Complete matrix: all three independent full-rank G0/control pairs pass; this remains setup-only.
Interpretation
No capacity conclusion changes. A passing recovery licenses only the preregistered producer stage that its inputs independently authorize.
Next action
No recovery action remains. The complete setup matrix was published and the producer's Stage-B cells are tracked in qwen35_4b_state_formation_capacity_adjudication; their scientific barriers remain unchanged.
Experiment log 7
Show the running log (7 entries, 2026-07-14)
2026-07-14 — handoff defect isolated
- First recovered full-rank seed-7411 G0 passed and occupied its canonical producer setup path.
- The following positive-control command stopped before wrapper STARTED publication because the first frozen wrapper treated any occupant of that pathname as the retired failure.
- No positive-control producer entry, model load, training/evaluation, benchmark access, contrast access, or scientific evidence occurred.
- Opened this additive operational successor rather than editing the frozen first recovery.
2026-07-14 — design review
- Frozen boundary requires exact archived-failure and retirement lineage, exact successful G0 and first-recovery invocation lineage, absence of the retired mirror, inherited stage shapes, exact producer validation, and new immutable STARTED/COMPLETE receipts.
- Next: focused tests, no-model smoke, commit/push, and both workflows green before model work.
2026-07-14 — frozen handoff smoke passes
- Focused tests pass 8/8, including exact failed-byte, changed-G0, mirror-reappearance, original- false-rejection, and inherited-shape controls.
- Smoke file SHA-256 is
0e0409de…6e5b, receipt identityd6fecf0a…de4c, under frozen source contract4d2ffde3…8acd; all six registered lineage/occupancy controls pass. - Smoke loaded no model, started no training/evaluation, opened no benchmark or contrast path, and changed no scientific interpretation.
- Current authorization: publish this complete checkpoint with the successful G0 and its first- recovery invocation receipts. No downstream model stage may begin until both workflows are green.
2026-07-14 — full-rank seed-7411 positive control passes
- Handoff commit
627254f1…d892passed Validate Repository and Publish Research Site before use. - Producer control file SHA-256 is
a0d17e2e…7a16, identity6708a4d4…9649; handoff STARTED and COMPLETE file hashes areb1b91cb2…ff2bande0887536…13f1, with COMPLETE identityaf0dd15c…2262. - Oracle and intact fixed-final joint state are 48/48; adaptation-disabled joint state is 0/48 after exactly 256 optimizer updates, accumulation 16, and 4,096 singleton presentations. Both the full- rank path and shared state changed; no early stop or checkpoint selection occurred.
- The receipt opens no result payload, benchmark, or contrast and claims no scientific evidence.
- Current authorization: publish this setup checkpoint and require both workflows green before the full-rank seed-7412 G0/control pair. Result training remains blocked on the complete setup matrix.
2026-07-14 — full-rank seed-7412 G0/control pair passes
- Seed-7411 setup commit
c507488c…3c06passed both workflows before seed 7412 began. - G0 file SHA-256 is
10bf22fc…1d18, identity62ecb79e…951dc; handoff G0 STARTED/COMPLETE file hashes are01099e34…aec3/0b48b1e5…7d35, COMPLETE identityd61d6441…9246. - G0 has zero K=1 and checkpoint-roundtrip error, all 62 deltas have complete finite optimizer state, all joint trainable groups have finite nonzero gradients, K=12 is finite, and 22.1 GiB remains.
- Control file SHA-256 is
1cbbd823…8510, identity6575e1d2…4554; handoff STARTED/COMPLETE file hashes areffc30b05…d55c/9535f515…f3a2, COMPLETE identity6334d72f…c14a. - Oracle/intact are 48/48 and disabled is 0/48 after exact 256 updates, accumulation 16, and 4,096 presentations. Full-rank/shared-state parameters changed; no early stop or checkpoint selection.
- Current authorization: publish this setup checkpoint and require both workflows green before seed 7413. Result training remains blocked on the complete setup matrix.
2026-07-14 — full-rank seed-7413 pair passes; setup matrix complete
- Seed-7412 setup commit
92473a53…8530passed both workflows before seed 7413 began. - G0 file SHA-256 / identity are
021a8444…d635/4d2316d3…2ff0; handoff G0 STARTED/COMPLETE file hashes area38ebcba…38ed/11229858…cea3, COMPLETE identityfbfb282f…5aef. - G0 has zero K=1/round-trip error, finite/nonzero joint gradients, complete finite optimizer state for all 62 deltas, finite K=12, and 22.1 GiB free.
- Control file SHA-256 / identity are
8a4af0d6…fde8/976f28ef…14df; handoff control STARTED/ COMPLETE file hashes ared5dbe18e…afa5/8cd6ff71…3c89, COMPLETE identityccff1520…58c9. - Oracle/intact are 48/48 and disabled is 0/48 after exact 256 updates, accumulation 16, and 4,096 presentations. Full-rank/shared-state parameters changed; no early stop or checkpoint selection.
- All three full-rank setup pairs now pass with zero result, benchmark, contrast, or scientific evidence. Publish/green the complete matrix before any Stage-B training cell.
2026-07-14 — handoff scope closed
- The branch handoff and all three full-rank G0/control setup pairs completed, the complete matrix was published, both workflows passed, and the producer subsequently launched its first mandatory Stage-B cell. This answers the recovery experiment's operational question.
- Stage-B execution belongs to the canonical capacity-adjudication producer. Continued use of the frozen wrapper is a producer dependency, not unfinished recovery work.
Reproduce
Smoke test
.venv/bin/python -B experiments/qwen35_4b_state_formation_branch_handoff_recovery/scripts/run.py --smokeFull run
run only an exact branch-authorized producer command documented in the producer GPU runbook through scripts/run.pyRun steps are documented inside the experiment folder (README and scripts).