Research log Small Model Experimentation
GitHub

State-Formation Analysis Recovery

Finished2026-07-13 → 14Benchmark GeneralizationPosttraining and AdaptationStructured Execution and CompilersGitHub ↗
Recovery path is ready

The one idea you need

A frozen analysis is like a sealed calculator: repair the cable that feeds it, while leaving every button and formula untouched.

The question

Can the existing state-analysis run read its own saved files after a strict path check blocked them?

What we found

Yes, the narrow recovery check passed. It accepts only the one registered file location, still rejects unsafe shortcuts, and has not yet examined any model result.

Why it matters

Practitioners can continue the original comparison without retraining or quietly changing how success is judged. The next evidence will still come from the frozen analysis.

Safety checks4 of 4path-boundary controls passed
Results examined0scientific result rows opened
Sealed cases opened0future comparison rows touched
On this page
  1. Results at a glance
  2. Overview
  3. Report
    1. Summary
    2. Research Program Fit
    3. Method
    4. Results
    5. Controls
    6. Oracle Versus Deployable Evidence
    7. Interpretation
    8. Next Experiments
    9. Artifact Manifest
  4. Experiment log
  5. Reproduce
  6. Related

Results at a glance 1

Every recovery safety check passed

How to read

The horizontal groups are the four path-safety checks, and bar height is the pass indicator. Taller bars mean the recovery preserved that safeguard.

00.250.50.751defect reproduceddefect reproduced1canonical equivalencecanonical equivalence1unrelated alias rejectedunrelated alias rejected1prefix traversal rejectedprefix traversal rejected1

Takeaway → All four bars reach the pass mark, showing the file-path repair stayed narrow before any result was read.

Data table
frozen recovery controlpre-result control pass
defect reproduced1
canonical equivalence1
unrelated alias rejected1
prefix traversal rejected1

Numbers from experiments/qwen35_4b_state_formation_analysis_recovery/runs/smoke.json

Technical framing

All four path-boundary controls pass before scientific analysis — Operational smoke only. The receipt also records zero result rows, benchmark paths, sealed contrast rows, and scientific analysis calls; it is not evidence about LoRA or state formation.

In the author’s words from the Overview · “Results”

Zero of 57 required formation cells passed the 0.40 threshold; the maximum intact required accuracy was 0.0234375, all three formation categories missed, and adaptation contrast was uncertain. This licenses the producer's registered Stage B, not any new recovery-defined action. A producer-side metadata projection had accidentally exposed seed-7412 values before the already-fixed seed 7413 ran unchanged; no analyzer or choice followed, but terminal interpretation must disclose imperfect operator blinding.

Overview

This experiment recovers the exact frozen v11 state-formation analyzer through one source-bound path seam, without changing any LoRA/full-rank scientific logic or inspecting a result value.

The operational recovery question is answered and this experiment is closed. Later invocations of the frozen seam are infrastructure use; they do not keep the recovery experiment in progress.

Research Program

Question

Can the exact immutable source-v11 analyzer consume its own source-bound producer receipts after its registered ../../large_artifacts/... path was rejected by the analyzer's lexical-canonical guard, without changing any scientific function, result, threshold, or branch rule?

This is an operational recovery experiment, not a new scientific arm. The producer remains the result-bearing experiment. Its source cannot be edited: doing so would invalidate every completed setup, training, and evaluation receipt.

Hypothesis

The failure is confined to one path-construction boundary. Accepting only the producer's exact registered external prefix and lexically clean descendants, while delegating every other path to the original helper, should let the unchanged v11 training graph reopen. Canonical inputs must resolve identically under the old and recovered helpers. Any broader alias acceptance, source mismatch, or scientific-code change falsifies the recovery.

Setup

  • Model identity carried from the producer: only Qwen/Qwen3.5-4B, revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a; this recovery loads no model.
  • Backend identity: the producer's frozen transformers backend; this recovery performs CPU-only receipt analysis.
  • Dataset/task source: no new data. Inputs are only the producer's exact source-v11 receipts and procedurally generated evaluation outputs, reopened by their embedded SHA-256 identities.
  • Baseline: unmodified v11 _canonical_expected_path, which reproducibly rejects its own registered external prefix before opening a result row.
  • Mechanism-falsifying controls: exact canonical-path equivalence; rejection of unrelated .. aliases; rejection of traversal beneath the registered prefix; automatic restoration of the original function; exact producer source/config/implementation pins.
  • Primary metric: RECOVERED_V11_ANALYSIS_COMPLETE with an output byte hash and original receipt identity matching the exact v11 analyzer output.
  • Oracle-only metrics: all state-formation values remain the producer's preregistered oracle-side measurement. This recovery defines no metric or threshold.
  • Hidden-label boundary: smoke and design review open zero result rows, benchmark paths, or sealed contrast rows. Scientific analysis runs only after all three LoRA trigger evaluations exist.

Recovery boundary

The immutable producer computes this registered path:

experiments/qwen35_4b_state_formation_capacity_adjudication/
../../large_artifacts/qwen35_4b_state_formation_capacity_adjudication

Its helper rejects any absolute path whose raw spelling differs from abspath(raw), so it rejects that path before its later repository/no-symlink validation. The recovery seam recognizes only this exact raw prefix. A descendant is accepted only when every suffix component is clean; it is then normalized once and passed to v11's unchanged canonical_repo_path. All other paths still call the original helper.

The original analysis output stays at the producer's canonical analysis/ location because every later branch authorization requires that exact path. This experiment writes an immutable sidecar that binds its SHA-256, original receipt identity, frozen recovery smoke, and exact seam. A STARTED receipt makes the narrow output-write/sidecar-write crash window recoverable without silently adopting an unrelated result.

Run

Tests and non-result smoke:

.venv/bin/python -B -m unittest discover \
  -s experiments/qwen35_4b_state_formation_analysis_recovery/tests -v
.venv/bin/python -B \
  experiments/qwen35_4b_state_formation_analysis_recovery/scripts/run.py --smoke

After all three LoRA trigger evaluations are complete:

.venv/bin/python -B \
  experiments/qwen35_4b_state_formation_analysis_recovery/scripts/run.py \
  --phase lora_joint

The same runner exposes only the five preregistered producer phases: lora_joint, lora_control, stage_b_seal, fullrank_joint, and fullrank_control.

Results

The non-result smoke passed at file SHA-256 02f6f9275f9c30fddb2f49d4b061237e4e11985d92569642d60cd107b05243f7, receipt identity 30353be5429d4987509715cfa56a6187f24a80ab353b9b774908071de7ed2f8f, and recovery source contract 6ab26016b3de397307c7c8def9c685315b6660370ee98af1a757da11fe1ee94b. All four path controls pass; the receipt records zero result rows, benchmark paths, sealed contrast rows, and scientific analysis calls.

After seed-7413 publication and green workflows, the recovery ran lora_joint exactly once. The unchanged producer v11 analyzer emitted LORA_JOINT_MISS_CONTROLS_REQUIRED, next stage run_lora_state_only_and_fullrank_joint, at producer output SHA-256 cb9fee75…818a and receipt identity b973bc01…a862. The recovery sidecar identity is d068482a…f40e. Zero of 57 required formation cells passed the 0.40 threshold; the maximum intact required accuracy was 0.0234375, all three formation categories missed, and adaptation contrast was uncertain. This licenses the producer's registered Stage B, not any new recovery-defined action. A producer-side metadata projection had accidentally exposed seed-7412 values before the already-fixed seed 7413 ran unchanged; no analyzer or choice followed, but terminal interpretation must disclose imperfect operator blinding.

Interpretation

The recovery has now demonstrated both a narrow path repair and exact execution of the original decision tree. Scientific interpretation remains producer-owned: the LoRA joint recipe missed, so LoRA state-only plus full-rank joint are compulsory before attributing the miss to capacity.

Knowledgebase Update

  • Program evidence: record the producer Stage-A miss and its mandatory rank-causal controls.
  • Program backlog: replace the pending adjudication with the exact Stage-B execution branch.
  • Claim ledger: no claim is available from an operational repair.
  • Reusable artifact: an exact-prefix, source-bound recovery pattern for immutable analyzers.

Artifacts

  • idea_intake.md: novelty and non-duplication decision.
  • configs/default.yaml: exact producer and model/config/source pins.
  • src/recovery.py: isolated importer, narrow seam, smoke, and analysis wrapper.
  • tests/test_recovery.py: adversarial scope and restoration tests.
  • reports/design_review.md: frozen go/no-go review.
  • runs/smoke.json: source-bound non-result smoke receipt once executed.
  • analysis/<phase>_recovery.json: result-output sidecars, created only when licensed.
  • reports/artifact_manifest.yaml: producer-owned external inputs and regeneration path.

Report

Rendered from reports/report.md

Summary

The immutable producer's analyzer had one mechanical path-boundary defect: it rejected its own registered ../../large_artifacts/... prefix before any result row was interpreted. A separately source-bound consumer pins source v11, temporarily repairs only that exact prefix, and has now completed the authorized lora_joint phase without changing scientific code.

Research Program Fit

This preserves the valid capacity adjudication in structured_execution_and_compilers. It is not a new state-formation method or data arm and therefore contributes no program evidence by itself.

Method

The consumer imports exact producer source-contract v11 under an isolated package name, pins the complete source/config/model/backend identities, reproduces the original failure, and installs an exact-prefix seam. Clean suffixes are normalized once and passed to the original repository/no- symlink validator. All nonmatching paths call the original helper. The exact original analyzer writes its canonical receipt; a recovery-owned sidecar binds that output and the frozen seam.

Results

The permitted pre-result outcome passed. RECOVERY_SMOKE_PASS has file SHA-256 02f6f9275f9c30fddb2f49d4b061237e4e11985d92569642d60cd107b05243f7, receipt identity 30353be5429d4987509715cfa56a6187f24a80ab353b9b774908071de7ed2f8f, and recovery source contract 6ab26016b3de397307c7c8def9c685315b6660370ee98af1a757da11fe1ee94b. All four frozen controls pass, with zero scientific rows, benchmark paths, or sealed contrast rows opened.

After the complete producer matrix was published and both workflows passed, the recovered analysis emitted RECOVERED_V11_ANALYSIS_COMPLETE. Producer output SHA-256 is cb9fee75…818a, producer receipt identity b973bc01…a862, and recovery sidecar identity d068482a…f40e. The producer verdict is LORA_JOINT_MISS_CONTROLS_REQUIRED: 0/57 required cells passed the 0.40 threshold, maximum intact required accuracy was 0.0234375, all formation categories missed, and adaptation contrast was uncertain. The exact next stage is LoRA state-only plus direct-full-shape joint. A producer-side metadata projection accidentally printed seed-7412 values before the mandatory seed 7413 ran unchanged; it did not invoke this analyzer or alter a registered choice, but interpretation must disclose imperfect operator blinding.

Controls

  • Original v11 rejection reproduced for its own registered prefix.
  • Canonical-path equivalence between original and recovered helpers.
  • Registered-prefix traversal rejected.
  • Unrelated lexical alias rejected by the original helper.
  • Original helper restored after normal and exceptional context exits.
  • Exact source-contract, implementation, analysis-file, config, model, revision, and backend pins.

Oracle Versus Deployable Evidence

The recovery manipulates only artifact addressing. Its successful execution is operational evidence; the state metrics and Stage-A negative remain producer-owned oracle evidence, not deployable answer capability evidence.

Interpretation

The original decision tree remains the sole source of the scientific verdict. It now mandates the capacity controls because Stage A cannot distinguish a LoRA limit from a joint-objective/recipe miss.

Next Experiments

The recovery is complete and licenses no new experiment by itself. Producer-owned full-rank setup and Stage-B execution continue under qwen35_4b_state_formation_capacity_adjudication and its immutable receipts.

Artifact Manifest

See artifact_manifest.yaml. External artifacts remain owned and hash-bound by the producer.

Experiment log 6

Show the running log (6 entries, 2026-07-13 → 14)

Scaffold

Created as a separate recovery experiment after the immutable source-v11 producer completed all three LoRA joint training cells and seed 7411 trigger evaluation. Before any scientific values were inspected, an independent production _load_evaluation reopen reproduced:

RuntimeError: expected path is not lexical-canonical: .../experiments/
qwen35_4b_state_formation_capacity_adjudication/../../large_artifacts/
qwen35_4b_state_formation_capacity_adjudication/lora_joint_seed7411

The error occurs before evaluation rows are consumed. Editing the producer would invalidate its source-bound artifacts, so the recovery is additive and separately reviewed.

2026-07-13 design

  • Kept the producer at exact source-contract v11, SHA-256 5a8ed26ddb9446c728191ca8e7849ae44cff92a700e24b237dac522cf4286666.
  • Pinned exact analysis.py SHA-256 876888987d816fe29ae93fde0053fb91ed58301d16bf429d5cabaa809c23a2b0.
  • Rejected copying or modifying scientific summarizers.
  • Authorized one temporary helper replacement that recognizes only the registered external prefix and clean descendants, then calls the original repository/no-symlink checker.
  • Required canonical equivalence, unrelated-alias and traversal rejection, finally restoration, a source-bound smoke, original lock/source snapshot, a STARTED receipt, and an immutable output sidecar.
  • Kept all result values, benchmark paths, and sealed contrast rows unopened during design/smoke.

2026-07-13 frozen smoke

The focused recovery suite passed 10/10. The non-result smoke then returned RECOVERY_SMOKE_PASS:

  • file SHA-256: 02f6f9275f9c30fddb2f49d4b061237e4e11985d92569642d60cd107b05243f7
  • receipt identity: 30353be5429d4987509715cfa56a6187f24a80ab353b9b774908071de7ed2f8f
  • recovery source contract: 6ab26016b3de397307c7c8def9c685315b6660370ee98af1a757da11fe1ee94b
  • exact producer source v11 and analyzer hashes matched
  • canonical equivalence, defect reproduction, unrelated-alias rejection, and prefix-traversal rejection all passed
  • result rows, benchmark paths, sealed contrast rows, and scientific analysis calls: zero

Any change to the frozen config, design review, runner, source, or tests now invalidates this smoke. The next licensed action is publication and green CI, followed by LoRA trigger evaluation seeds 7412 and 7413. Running a recovery analysis phase remains prohibited until that three-seed matrix is complete.

2026-07-14 — complete trigger matrix awaits publication before recovered analysis

  • Recovery commits a6360cc1 and e35e071e passed both repository workflows before the remaining producer evaluations.
  • Producer seeds 7412 and 7413 each completed once without retry, making the fixed three-seed LoRA trigger matrix complete. All evaluation receipts bind exact producer source/config/checkpoints, only the three trigger payloads, and zero benchmark or sealed-contrast access.
  • An overbroad producer-side metadata projection exposed seed-7412 per-split values before seed 7413. No recovery analyzer, classifier, branch, retry, source edit, seed choice, or checkpoint choice followed; seed 7413 was already compulsory and ran unchanged. This imperfect operator blinding must remain visible in the final report.
  • The frozen recovery contract is unchanged. Recovered lora_joint analysis remains prohibited until the seed-7413 producer checkpoint is committed, pushed, and both workflows are green.

2026-07-14 — exact v11 LoRA-joint analysis recovered

  • Producer checkpoint b326f6cd passed both workflows before the recovery phase began.
  • --phase lora_joint completed once without resume. It changed only _canonical_expected_path; the scientific functions remained exact immutable producer v11.
  • Producer output SHA-256 is cb9fee75…818a, producer receipt identity b973bc01…a862, recovery sidecar SHA-256 aa43077b…6b7e, and sidecar identity d068482a…f40e.
  • Producer status/verdict is LORA_JOINT_MISS_CONTROLS_REQUIRED; the exact next stage is run_lora_state_only_and_fullrank_joint. No sealed contrast was opened.

2026-07-14 — recovery scope closed

  • The narrow path repair and exact v11 analyzer successfully consumed the producer receipts and emitted the authoritative result. This answers the operational recovery question with authenticated receipts.
  • Stage B remains active only in qwen35_4b_state_formation_capacity_adjudication; reuse of this frozen seam is infrastructure, not unfinished recovery research.

Reproduce

Smoke test

.venv/bin/python -B experiments/qwen35_4b_state_formation_analysis_recovery/scripts/run.py --smoke

Full run

.venv/bin/python -B experiments/qwen35_4b_state_formation_analysis_recovery/scripts/run.py --phase lora_joint

Run steps are documented inside the experiment folder (README and scripts).

Browse the experiment folder on GitHub ↗