Research log Small Model Experimentation
GitHub

State-Carry Versus State-Bag Counterfactual

Finished2026-07-12Posttraining and AdaptationStructured Execution and CompilersGitHub ↗
LoRA did not form the running state this test needed

The one idea you need

Imagine tracking a piece moving across a board. One method keeps a single running note, updating and feeding it into each next turn. The other re-reads the opening position many times independently and pools its guesses. Same total effort — does chaining the note win?

The question

When a small model repeats the same computation to solve a step-by-step puzzle, does chaining its memory forward beat pooling many fresh, independent passes?

What we found

The first matched test did not answer that architecture question because the low-rank update failed to learn the required running state. Carrying memory improved overall accuracy by only 4.3 points, with uncertainty spanning a slight loss to a 9.4-point gain. More decisively, the model decoded the complete joint hidden state on just 0.46% of registered steps, far below the preregistered 40% floor, and swapping memory between examples did not make answers follow the donor. The three-run confirmation therefore stopped exactly as planned.

Why it matters

This was a useful negative checkpoint rather than evidence that serial memory itself was ineffective. A later full-size pilot also showed almost no readable state, but it failed other promotion checks at the same time and did not share bit-identical state-module initialization with this run. A cleaner multi-run comparison is therefore required before blaming or clearing LoRA.

Carry minus parallel-pool accuracy+4.3 pointspaired 95% interval: -0.8 to +9.4 points
Joint hidden states decoded0.46%the preregistered state-formation floor was 40%
Unseen task-family gain+5.1 pointspaired 95% interval: +0.8 to +9.8 points, but without a readable full state
Answer gain from following donor memory+0.8 pointspaired 95% interval: -2.3 to +3.9 points
On this page
  1. Results at a glance
  2. Overview
  3. Report
    1. Status
    2. Question
    3. Validity
    4. Result
    5. Scope and Next Experiment
    6. Artifact Manifest
  4. Experiment log
  5. Data files
  6. Reproduce
  7. Related

Results at a glance 1

Carry and parallel pooling received exactly the same repeated-compute budget

How to read

Each pair of bars shows the total decoder-layer work at one, four, eight, or twelve middle-block passes. The Carry and pooled comparison bars overlap at every depth.

02k4k6k8k16.38k16.38kK=128.67k28.67kK=445.06k45.06kK=861.44k61.44kK=12

Takeaway → The negative pilot cannot be explained by giving one arm less computation. Whether low-rank training caused the missing state remains the question for an initialization- and randomness-matched successor.

Data table
middle-block applications KState-CarryState-Bag
K=116.38k16.38k
K=428.67k28.67k
K=845.06k45.06k
K=1261.44k61.44k

Numbers from experiments/qwen35_4b_state_carry_vs_state_bag/analysis/setup_compute_geometry.csv

Technical framing

Design audit: serial Carry and reset-state Bag receive identical compute — Setup geometry only, not a model result. Both arms run the same base 32 layers and the same eight-layer middle block K-1 extra times; their only intended difference is whether each extra call inherits the preceding state.

In the author’s words from the Report · “Result”

Both node and checksum strata had positive Carry-minus-Bag point estimates (+0.0234375 and +0.0625). Those answer-level and holdout signs cannot rescue a state that did not encode the registered node+phase+checksum trajectory. The only failed promotion check was joint_state_sufficient; the analyzer therefore emitted PILOT_MECHANISM_MISS and promote: false.

Overview

Terminal LoRA status: PILOT_MECHANISM_MISS. The fixed-source, independently seeded rank-32 LoRA pilot was valid and complete, but it failed the preregistered deep-state-formation gate. Confirmation and sample-more were therefore not run. This result mandates a fresh full-rank extra-R-delta successor. That successor produced a complete raw pilot, but a post-result science audit found that it did not isolate capacity; the LoRA-rank question remains open.

Research Program

Question

When parameters, data, optimization, readout, and decoder-layer token applications are matched, does organizing repeated Qwen computation as one serially carried state produce a representation that is more capable than aggregating the same number of independent shallow states?

This experiment is deliberately not satisfied by “looping helps.” A positive requires State-Carry to beat a separately trained State-Bag twin, improve when test-time recurrence exceeds the trained K=4 horizon, accurately track the registered joint state, transfer to the joint family+surface holdout, lose its benefit when the carry edge is cut, and behave causally under state swaps.

Pilot Result

The source-bound seed-7401 Carry/Bag pilot completed all registered cells and diagnostics with matched initialization, data order, prompt tokens, decoder-layer-token applications, and exact post-checkpoint K=1 parity. The analyzer emitted PILOT_MECHANISM_MISS because Carry's mean joint node+phase+checksum step accuracy was 0.0045948, far below the 0.40 promotion threshold; node step accuracy was 0.0641912. This is a valid failure to form the registered deep joint state, not a mechanics, data-integrity, or infeasible-gate stop.

The answer-level signs were insufficient to override that failure. Matched-depth Carry minus Bag was +0.04296875 on 256 tasks (pilot 95% interval [-0.0078125, 0.09375]), and unseen-K Carry minus K=4 was +0.01171875 ([-0.03515625, 0.05859375]). Both query strata were positive and the joint holdout diagnostic was +0.05078125 ([0.0078125, 0.09765625]), but donor following changed by only +0.0078125 under swaps ([-0.0234375, 0.0390625]) and remained below recipient preservation by 0.0546875.

No seeds 7411–7413 were trained or evaluated. The same-checkpoint edge cut and the explicit-CoT sample-more comparator were not licensed. These missing stages are consequences of the registered pilot stop, not missing evidence for a full-run claim.

Architecture

Qwen/Qwen3.5-4B is split at complete native hybrid motifs:

Prelude P: layers  0..11
Loop R:    layers 12..19  (two [GDN,GDN,GDN,attention] motifs)
Coda C:    layers 20..31

Eight existing-vocabulary <|fim_pad|> tokens form a causal state bottleneck before the natural-language query. The untouched first P→R→C pass has recurrence LoRA disabled; therefore K=1 must reproduce the standard model's logits on the identical token sequence.

For extra calls:

State-Carry: z_t = R(prompt_memory, z_{t-1}, step=t)
State-Bag:   b_t = R(prompt_memory, z_1,     step=t)

Both arms use the same tied Qwen layers, loop-only LoRA, sinusoidal step signal, damping, state initializer, last-plus-mean aggregator, auxiliary state heads, ordered training rows, seeds, and number of calls. Non-state activations are reset after every extra call; only the state slots can cross recurrence depth. Checkpoints receipt the exact ordered-row digest, runtime-source contract, environment lock, phase, and fixed final step.

See docs/architecture.md for the exact forward, docs/research_handoff.md for the reasoning behind every choice, and reports/implementation_review.md for the final pre-GPU audit.

Substrate

Every item is a fresh, exactly executed finite world with randomly skinned node names. The hidden state is (node, phase, checksum); the next edge and state update depend on the preceding state. Two transition families and depths 1–4 are trained. A disjoint pilot seed and structurally firewalled pilot splits gate promotion; the three confirmatory seeds and all depth-5–12, third-family, held-out-rendering, and joint-holdout rows remain untouched until confirmation. Node/checksum queries are balanced within each family×template×depth cell.

The workspace tokens occur before Query:. The recurrent state knows the world and requested transition count but not whether it will later be asked for the terminal node or checksum. Shared auxiliary heads query node, phase, and checksum after every iteration, pressuring one state to remain jointly sufficient rather than answer-specific.

No file under benchmarks/ is read or imported.

Primary Arms and Controls

  1. Separately trained continuous-state Carry and Bag twins.
  2. A trained Carry checkpoint evaluated with its edge cut into Bag mode.
  3. Bidirectional counterfactual donor-state swaps in geometry-matched pairs sharing the same world, initial node, rule, and answer interface.
  4. A standard autoregressive explicit-state-trace LoRA trained on the same procedural rows, followed by matched-layer-token sample-more evaluation with frozen sampling/allocation parameters and termination/interface validity gates.

The earlier outcome-dependent mixed semantic-echo branch was removed during this review. It is a distinct architecture with missing shuffled/wrong-task controls and, under the repository lifecycle, must be a fresh successor experiment if continuous-state results license it.

Rank-32 LoRA remains the first adaptation because it touches every linear projection in both repeated motifs while preserving a cheap, exactly disabled K=1 path; the carried workspace itself remains full-width and receives dense joint-state supervision. A valid negative that fails to establish deep state formation does not settle whether LoRA was too restrictive. Preregistration section 10 mandates creating and executing a fresh zero-initialized full-rank extra-R-delta successor in that case, preserving the same model and exact base path. Mechanics/data-integrity failures and mathematically invalid gates require repair rather than a capacity test. A readable-but-unused state or a sample-more-only loss does not trigger full rank because LoRA has then already formed the deeper representation; the former instead licenses the separately controlled interface successor.

Terminal Disposition

Do not advance this LoRA experiment to G2, edge-cut confirmation, or G4. Preserve its valid negative and the earlier invalidated analysis-dispatch attempt. The authorized capacity test was completed in qwen35_4b_state_carry_vs_state_bag_fullrank_delta. Its exact-row, zero-initialized 892M-parameter pilot recorded 0.00277 joint-state accuracy versus 0.40, but it simultaneously failed non-capacity promotion requirements: Carry minus Bag was -0.015625, and neither query stratum was positive. Under that successor's preregistered ladder, the correct disposition is therefore PILOT_PROMOTION_BLOCKED, not an isolated state-formation miss. The raw result is retained, but it does not close LoRA capacity.

A fresh RNG-matched three-seed state-formation adjudication is mandatory. It must compare rank-32 LoRA and full-rank extra-R deltas under paired randomness and the held-fixed state-formation contract, so adaptation capacity is not entangled with seed noise or simultaneous answer/query-gate failures.

Primary Metrics

  • paired State-Carry minus State-Bag accuracy at K=semantic depth, depths 5–12, with a crossed task×training-seed bootstrap;
  • machine-verified initialization and training-compute equality for every seed pair;
  • paired Carry gain from trained K=4 to unseen K=semantic depth;
  • exact joint node+phase+checksum trajectory accuracy;
  • same-checkpoint intact-minus-edge-cut accuracy with positive evidence in every seed;
  • bidirectional pre/post donor-following under geometry-matched state swaps;
  • joint family+surface holdout non-reversal and both node/checksum query strata;
  • quick/overthinking retention at K=1,4,8,12;
  • matched-budget explicit-CoT majority accuracy and oracle pass@N, with natural-close reporting and preregistered parse/cap-contact gates.

The fail-closed verdict ladder is defined in reports/preregistration.md and implemented in src/analysis.py.

Artifacts

  • Source and tests are committed here.
  • Generated full JSONL is deterministic across Python hash seeds and gitignored under data/generated/; its manifest binds the generator/source contract and hashes are produced at runtime.
  • Realized pilot adapters, loop state, checkpoint/run identities, and hashes live under large_artifacts/qwen35_4b_state_carry_vs_state_bag/pilot_{carry,bag}_seed7401/ and are declared in reports/artifact_manifest.yaml.
  • Small evaluation rows, summaries, and paired analysis remain under runs/ and analysis/ unless size requires manifesting them.

Report

Rendered from reports/report.md

Status

PILOT_MECHANISM_MISS: the fixed-source rank-32 LoRA pilot is complete and valid. It failed the preregistered joint-state-formation gate, so the experiment stopped before confirmation. The earlier analysis-dispatch attempt remains preserved but is not pooled with this result.

Question

Does a serially inherited internal state produce deeper, causally useful representations than an equal-compute collection of independent shallow states?

Validity

The canonical G0 receipt passed exact direct-model and Carry/Bag K=1 parity (0.0), both-arm gradient checks, and finite worst-format K=12 forwards. The pilot used only seed 7401 and the dedicated pilot validation, depth, joint-holdout, and counterfactual splits. Carry and Bag each trained for the fixed 300 steps. Their initialization, ordered-row digest, 2,594,937 prompt tokens, and 145,316,472 decoder-layer-token applications matched exactly. Both final checkpoints repeated K=1 parity at 0.0. All registered pilot cells, K=4 diagnostics, joint-holdout cells, and 64 bidirectional swap pairs were complete; the configured gain was mathematically reachable.

The analysis receipt is source/config/data/lock bound:

  • verdict receipt: c9fec5b584bef5aa3de40844a8552395f3c3f0f95f79285700a9e4e375aed0ef;
  • config: 70e4a2d6df7acb0c5a21c7c945c66499a0ede8e98321c7b56da1c080c819744b;
  • source: ef2dd25107cb306490e30dba8ac3035c8c69c76173cec62390557cf1add7a28d;
  • data manifest: 2cf9a4d008d0990873928424170ab5daf0a53473a9f97e542d8739ac6de92879;
  • training lock: 05546fe977583116d6169ea0dfa7b27e1184dd4a2b61d556dfb3f889d5b2b7b1.

Result

Pilot endpointResultRegistered interpretation
Carry joint-state step accuracy0.0045948Fail versus 0.40
Carry node step accuracy0.0641912Chance-like diagnostic
Carry minus Bag, matched depth+0.04296875, CI [-0.0078125, 0.09375]Positive point estimate; uncertain
Positive depth cells5 / 8Diagnostic only at pilot
Carry unseen-K gain over K=4+0.01171875, CI [-0.03515625, 0.05859375]Complete; uncertain
Joint family+surface holdout+0.05078125, CI [0.0078125, 0.09765625]Pilot diagnostic passed
Swap donor-follow gain+0.0078125, CI [-0.0234375, 0.0390625]Causal diagnostic failed
Donor follow minus recipient preserve-0.0546875Causal diagnostic failed
Carry answer-mode rate1.0Interface valid

Both node and checksum strata had positive Carry-minus-Bag point estimates (+0.0234375 and +0.0625). Those answer-level and holdout signs cannot rescue a state that did not encode the registered node+phase+checksum trajectory. The only failed promotion check was joint_state_sufficient; the analyzer therefore emitted PILOT_MECHANISM_MISS and promote: false.

Scope and Next Experiment

No confirmation seed (7411–7413), same-checkpoint edge cut, or explicit-CoT sample-more comparator was run, because the pilot stop prohibited them. This result says the registered rank-32 extra-call LoRA recipe did not form the required deep joint state in its valid pilot. It does not distinguish a serial-state limitation from insufficient low-rank plasticity.

The mandatory successor was executed in qwen35_4b_state_carry_vs_state_bag_fullrank_delta. It replaced extra-call LoRA with 892,272,640 zero-initialized full-rank deltas while holding the first pass, coda, exact K=1 logits, rows, schedule, Carry/Bag comparison, and gates fixed. Its raw pilot was complete and recorded joint-state accuracy 0.0027686, Carry minus Bag -0.015625, unseen-K gain -0.0078125, and noncausal swaps.

That run does not close LoRA capacity. It simultaneously failed the non-capacity promotion requirements that Carry be positive and that both query strata be positive (node 0.0; checksum -0.03125). The successor preregistration assigns PILOT_PROMOTION_BLOCKED whenever a complete pilot fails a non-capacity promotion requirement; PILOT_STATE_FORMATION_MISS can isolate capacity only when joint-state sufficiency is the specific failure. The post-result science-audit disposition is therefore PILOT_PROMOTION_BLOCKED, notwithstanding the raw analyzer label preserved in that experiment's summary.

The required next step is a fresh RNG-matched three-seed state-formation adjudication comparing rank-32 LoRA with full-rank extra-R deltas. Until paired multi-seed evidence separates adaptation capacity from seed variation and simultaneous answer/query failures, the low-rank concern remains unresolved. Neither existing experiment should proceed to confirmation, edge-cut, or sample-more.

Artifact Manifest

See artifact_manifest.yaml.

Experiment log 10

Show the running log (10 entries, 2026-07-12 → 13)

2026-07-12 — Intake and Design Freeze

  • Attached the new experiment to structured_execution_and_compilers; test_time_reasoning_budget was rejected as primary because its charter excludes new architectures.
  • Named qwen_fastweight_hook as the closest negative near-duplicate.
  • Reconstructed repository evidence C11–C54 and current recurrence/latent-state literature as a failure map.
  • Froze the central contrast: one inherited state versus equal-compute independent reset states.
  • Froze Qwen layers 12–19 as two complete hybrid motifs, eight state slots, K=4 training, K=5–12 extrapolation, three seeds, and fail-closed verdicts.

No model was loaded or called.

2026-07-12 — Implementation

  • Removed the scaffold's vLLM runner because hidden-state intervention requires Transformers and backend mixing would invalidate comparisons.
  • Added deterministic random-world substrate, three transition families, three renderings, exact trajectories, structural fingerprints, held-out depth/family/template splits, and matched counterfactual pairs.
  • Added a manual pinned-Qwen forward with untouched K=1, recurrence-only loop LoRA, state-only cross-loop communication, Carry/Bag edge switch, state sufficiency heads, fixed-point loss, and optional semantic echo.
  • Added model identity/layer/tokenizer/LoRA-locality/parity/gradient gates.
  • Added paired pilot/full training, matched-depth evaluation, K curves, edge cuts, donor swaps, explicit textual trace training, compute-matched sample-more, paired bootstrap, and terminal verdict assignment.
  • Hardened minimum-depth generation after adversarial review: both full joint-state repeats and earlier occurrences of the terminal queried field are rejected, including in donor-swap pairs.
  • Added research handoff, literature map, architecture contract, GPU runbook, agent goal, preregistration, and adversarial design review.

2026-07-12 — Local Validation

  • CPU smoke: CPU_SMOKE_PASS; three families at depths 1/4/8; counterfactual pair has distinct consequences; Carry/Bag compute receipts identical; no benchmark files read.
  • Deterministic smoke data build: zero structural cross-split duplicates; exact row counts and hashes.
  • Unit suite: 25 tests pass after initial implementation and adversarial hardening.
  • Python compilation: every experiment source and script compiles without importing unavailable GPU packages.

No Qwen model was loaded or called. Live model smoke remains the first task on the 48 GiB Ada environment.

Review Revisions

  • Avoided duplicate registration of the PEFT base model inside the recurrence wrapper.
  • Removed latent workspace placeholders from the explicit-text comparator.
  • Made answer tokens context-prefix-stable rather than assuming standalone tokenization.
  • Corrected the explicit-CoT target to close Qwen's think channel before the final answer.
  • Isolated analysis by config hash so mixed echo cannot pool with continuous results.
  • Reallocated evaluation compute toward full matched-depth and K=4 comparisons; nonprimary K curves are smaller diagnostics.
  • Preserved both composite and text-only Qwen3.5 config identifiers while keeping model ID/revision absolute.
  • Strengthened counterfactual swaps so paired prompts share world, label mapping, table order, query, and choice order; only the initial state and consequence differ.
  • Replaced flat item bootstraps with hierarchical seed-then-task resampling, machine-enforced positive breadth on six of eight depths, and an explicit state-sufficiency verdict gate.
  • Made sample-more fail closed on compute overspend and reject truncated thoughts that never naturally reach the answer channel.
  • Corrected auxiliary node supervision from an unobservable random generator ID to the node's visible table-row position; counterfactual pairs share that coordinate system.
  • Made gzip archives byte-reproducible and removed the counterfactual exception from the global structural-duplicate firewall; the complete default-size corpus now builds cleanly.
  • Fixed portable row receipts, pilot/full isolation, primary-cell completeness enforcement, and edge-cut analysis; corrupted rows, datasets, adapters, and loop states now fail hash checks.
  • Added initial-value and cumulative training-compute receipts that analysis enforces for every Carry/Bag seed pair.
  • Matched the explicit-CoT optimizer schedule and upgraded deployment analysis to a three-seed task-paired hierarchical comparison against oracle pass@N, with actual sampled-token and synchronized timing receipts.

2026-07-12 — Final Adversarial Pre-Run Revision

No model was loaded or called. Three independent read-only reviews covered scientific design, implementation, statistics, artifacts, and GPU operations; the primary agent then re-read every experiment file and traced config→data→model→training→evaluation→analysis.

  • Fixed cross-process corpus nondeterminism from set iteration and added a multi-PYTHONHASHSEED regression.
  • Moved G1 to seed 7401 and dedicated pilot-only depth, joint, and counterfactual splits; confirmation keeps seeds 7411–7413 and all scored rows untouched.
  • Replaced the nested seed/task bootstrap with a crossed bootstrap over the common task×training-seed matrix and added strict duplicate/key/corpus checks.
  • Made exact checkpoint phase, fixed-final step, seed, critical-source digest, environment-lock digest, tensor identity, and ordered training-row digest mandatory.
  • Converted the same-checkpoint edge cut from artifact availability into a positive causal gate with complete cells, per-seed direction, and crossed uncertainty.
  • Balanced and retained query type, made joint state accuracy mandatory, and added a joint family+surface holdout gate.
  • Geometry-matched counterfactual pairs at a shared initial node, evaluated both swap directions, hashed raw interventions, and added pre/post donor-following evidence.
  • Hardened the explicit-CoT comparator with frozen sampling/allocation, raw generations, exact full coverage, compute rechecks, and close/parse/cap plus Carry answer-mode gates.
  • Removed the conditional mixed semantic-echo variant; any interface follow-up now requires a separate experiment and its missing shuffled/wrong-task controls.
  • Declared training non-resumable rather than allowing approximate recovery, reduced checkpoints to fixed finals, made shell loops fail-fast, and added a worst-format K=12 G0/resource receipt.

The prior CPU-smoke receipt predates these changes and is historical only. Unit/static validation is rerun after the patch; a fresh CPU smoke/data manifest remains the first operational step before G0.

The user additionally required the low-rank capacity ambiguity to be resolved rather than left as a caveat. Rank-32 LoRA remains first. A valid miss that fails to establish deep state formation mandates creating and executing a new zero-initialized full-rank extra-loop-delta successor that preserves the exact K=1 base path. Mechanics/data failures and infeasible gates require repair/review; a readable but unused state routes to the controlled interface successor, and a sample-more-only loss triggers neither because LoRA has then already formed the representation.

2026-07-12 — Integrated Audit Closure

  • Added a dedicated pilot-validation seed/split and removed the last pre-promotion read of confirmatory validation rows.
  • Pinned an exact confirmatory-config digest; every model-bearing entry rejects smoke/reduced geometry, and nonconfirmatory analysis cannot emit evidence.
  • Removed dead mixed-interface scalars and the orphaned static-LoRA arm.
  • Made pilot promotion require complete K=4, joint-holdout, and bidirectional-swap diagnostics without requiring favorable diagnostic signs.
  • Implemented every documented verdict label, full receipt identities, current lock/source checks, exact immutable row pairing, and correct pair-clustered swap inference.
  • Reanalyze raw sample-more allocations, parses, labels, totals, and by-depth natural-close/parse/cap rates.
  • Final validation: 42 tests pass, Python compilation passes, and git diff --check is clean.

No fresh CPU smoke/data generation, model load, GPU call, training, evaluation, or benchmark access occurred.

2026-07-12 — First Live G0 Attempt

  • Rebuilt the exact pinned Transformers environment and compiled causal-conv1d==1.6.2.post1.
  • Fresh CPU smoke, all 41 tests, and the complete source-bound corpus passed before model loading.
  • The first live G0 loaded only Qwen/Qwen3.5-4B at the pinned revision, then stopped before issuing a receipt or starting training: the smoke harness reused a K=4-encoded target tensor for its K=1 Carry/Bag equality forward, producing a 1-versus-4 state-loss shape error.
  • Fixed the harness to encode and use a dedicated K=1 batch for both equality and direct-model parity; added a static regression assertion. Because runtime source is identity-bound, the corpus and G0 receipt must be regenerated/reissued before proceeding.
  • Reissued the full source-bound corpus and reran G0. MODEL_SMOKE_PASS: K=1 direct parity 0.0, Carry/Bag K=1 difference 0.0, identical 16,800,796-parameter/value receipts, nonzero finite LoRA, state, step, and sufficiency gradients in both arms, finite worst-format K=12, and 11.21 GiB peak allocation. No scientific claim is licensed; seed-7401 pilot is next.
  • Completed both fixed 300-step seed-7401 pilot trainings and their dedicated evaluations. Training receipts matched exactly. Before a promotion verdict was written, analysis failed closed because _deployment_comparison applied the full seed set (7411–7413) to pilot seed 7401. This is an analysis phase-dispatch bug, so the otherwise complete attempt is preserved but invalidated; its chance-like state metrics do not license either a LoRA conclusion or the capacity successor.
  • Fixed pilot analysis to prohibit and skip both deployment/sample-more comparators, and added a synthetic end-to-end pilot regression that would fail if full deployment logic is entered. The source-bound contract requires fresh data, G0, training, and evaluation rather than receipt edits.
  • Reissued the source-bound CPU/data receipts and passed the fixed-source G0 retry. The canonical MODEL_SMOKE_PASS records exact direct and Carry/Bag K=1 parity, identical 16,800,796-parameter receipts, nonzero finite gradients in every registered trainable component, finite K=12 forwards, and 11.02 GiB peak allocation. This is a mechanics gate only; the fresh seed-7401 pilot pair is next.

2026-07-13 — Fixed-Source LoRA Pilot Verdict

  • Completed fresh canonical Carry and Bag pilot trainings at the sole registered pilot seed 7401, each for the fixed 300 steps. Initialization and training receipts matched exactly: ordered-row digest 97813bf9a2c7b81cf55db1a405e8e999e7e4bf953b2d50434a007140019b0e4f, 2,594,937 prompt tokens, and 145,316,472 decoder-layer-token applications per arm.
  • Evaluated the fixed final checkpoints on all 256 pilot depth tasks at K=4 and matched depth, all 256 pilot joint-holdout tasks, and all 64 counterfactual pairs in both directions. Both checkpoints retained exact direct-model K=1 parity (0.0). All source, config, lock, data, phase, step, seed, checkpoint, raw-row, and swap hashes passed reanalysis.
  • The analyzer emitted terminal LoRA verdict PILOT_MECHANISM_MISS. Seven of eight promotion checks passed; the only failure was joint-state sufficiency. Carry joint node+phase+checksum step accuracy was 0.0045947759645059705 against the frozen 0.40 threshold, while node step accuracy was 0.06419115958851762.
  • Answer-level Carry minus Bag was +0.04296875 with pilot 95% interval [-0.0078125, 0.09375]; unseen-K gain over K=4 was +0.01171875 [-0.03515625, 0.05859375]. The joint holdout was positive (+0.05078125, [0.0078125, 0.09765625]), but swaps were noncausal: donor-follow gain +0.0078125 [-0.0234375, 0.0390625] and donor-follow minus recipient-preserve -0.0546875.
  • Stopped this experiment at G1 as preregistered. No confirmation seeds, edge-cut confirmation, text baseline, or sample-more calls were run. The earlier analysis-dispatch attempt remains preserved and invalidated; it is not pooled with the canonical result.
  • This is a valid deep-state-formation failure rather than a mechanics/data/infeasible-gate stop. Preregistration section 10 therefore mandates creating and executing a fresh successor with zero-initialized full-rank deltas active only on extra R applications in layers 12–19, preserving the frozen base first pass/coda and exact K=1 path.

2026-07-13 — mandatory capacity successor raw result

  • The fresh qwen35_4b_state_carry_vs_state_bag_fullrank_delta experiment completed its exact-row, held-fixed seed-7401 pilot with 892,272,640 direct full-rank delta parameters.
  • Its analyzer emitted PILOT_STATE_FORMATION_MISS: joint state accuracy 0.00277 versus the 0.40 gate, Carry minus Bag -0.0156, negative unseen-K scaling, and noncausal swaps.
  • The raw artifacts and label are preserved. The post-result science audit below retracts the initial interpretation that this single pilot closed LoRA rank.

2026-07-13 — post-result capacity audit correction

  • The full-rank pilot failed three promotion checks simultaneously: joint-state sufficiency, positive Carry minus Bag, and positive effects in both query strata. Carry minus Bag was -0.015625; node was 0.0 and checksum was -0.03125.
  • Under the successor's frozen verdict ladder, any complete pilot that fails a non-capacity promotion requirement has disposition PILOT_PROMOTION_BLOCKED. PILOT_STATE_FORMATION_MISS isolates the capacity branch only when joint-state sufficiency is the specific failure. The raw analyzer label therefore overstates what this mixed failure can conclude.
  • Retracted the claim that the full-rank run closed LoRA capacity. Its corrected scientific disposition is PILOT_PROMOTION_BLOCKED; the metrics, summaries, and checkpoint references remain valid raw evidence.
  • A fresh RNG-matched three-seed state-formation adjudication is mandatory, pairing rank-32 LoRA and full-rank extra-R deltas while holding the state-formation contract fixed. This is required to separate adaptation capacity from seed variation and simultaneous answer/query-gate failures.
  • Neither completed pilot is licensed to advance to confirmation, edge-cut, or sample-more, and the capacity question must not be marked closed before the paired multi-seed adjudication.

Data files 3

Result tables and metrics copied from the experiment folder — preview inline or open the raw file.

Reproduce

Smoke test

python3 experiments/qwen35_4b_state_carry_vs_state_bag/scripts/run.py --smoke

Full run

phase-gated and intentionally non-monolithic: follow experiments/qwen35_4b_state_carry_vs_state_bag/docs/gpu_runbook.md sections 1-9; the CLI enforces G0, pilot promotion, fixed-final phases, causal gates, and G4 eligibility

Run steps are documented inside the experiment folder (README and scripts).

Browse the experiment folder on GitHub ↗