Qwen3.5-4B Materialized Residual Sibling Search Fresh Replication
The one idea you need
The previous test returned one batch of answers but lost them before saving. Its rules forbid drawing that same batch again, so this successor starts with entirely new puzzles and random draws while keeping the scientific comparison unchanged.
The question
Can concrete intermediate states help the small model finish multi-step rules better than names alone or equally costly ordinary attempts?
What we found
The generation test cannot answer the question because every reasoning trace reached its length cap and the materialized arm parsed only 12 of 52 outputs, with zero successes. The separate one-token ranking test is a clean negative: materialized recall among viable first moves was 0.257, below names alone, shuffled states, listwise choice, and a surface-only rule.
Why it matters
The full transaction record is finally durable, so the cheap ranking path can be retired confidently. But the free-generation null must stay narrow: it diagnoses a broken answer mode, not proof that concrete intermediate states are useless. A successor now qualifies the answer interface before mechanics.
On this page
Results at a glance 1
How to read
Each bar is the average share of publicly viable first moves recovered among the top four choices. Compare materialized states with names alone, shuffled states, listwise choice, a surface-only rule, and random ranking.
Takeaway → Materialized ranking beats random but remains below all five structured comparisons, so this cheap ranking route does not justify a larger search.
Data table
| one-token viability ranker | mean recall at 4 |
|---|---|
| materialized | 0.257 |
| name only | 0.281 |
| shuffled | 0.323 |
| listwise | 0.271 |
| surface | 0.375 |
| random | 0.108 |
Numbers from experiments/qwen35_4b_materialized_residual_sibling_search_fresh_replication/runs/mechanics/summary.json
Technical framing
Materialized viability ranking loses to every structured comparator — All 2,304 model score rows authenticated. Materialized beats realized random by 0.1493, narrowly below the +0.15 gate, but loses to every structured comparator and misses every absolute support floor.
In the author’s words from the Overview · “Results”
The corrected construction passed in 93.6 seconds and froze 264 tasks: 24 mechanics, 48 qualification, and 192 confirmation. It found 3,526 eligible exact-depth-three behaviors, preserved the 88/88/44/44 live-sibling balance, and achieved 0.952 simulated compound power at the registered alternative. The authenticated parent comparison found zero shared task IDs, identity-free public instances, all-mechanics prompts, or terminal materialized prompts. Finite-DSL reuse is reported rather than hidden: 56 functions, 41 concrete triples, and 181 suffixes. The real tokenizer receipt records model EOS 248044, tokenizer EOS 248046, ignore_eos=true, and explicit stop [248044]. All 44 experiment tests pass. … Read the full result →
Overview
This separately registered recovery replication preserves the parent's frozen materialized-residual science while regenerating tasks, request IDs, and sampling seeds and hardening durable generation receipts. The authenticated mechanics run is terminal: generation failed its ABI and the parse-immune cheap materialized ranker failed its gate. Qualification and confirmation were never opened.
Research Program
- Primary:
structured_execution_and_compilers. - Secondary targets after construction review:
evidence_conditioned_selection,interpretability_and_diagnostics, andtest_time_reasoning_budget. - Scientific parent:
qwen35_4b_materialized_residual_sibling_search. - Immediate reason for a new experiment: the parent is sealed by a terminal
STARTEDtransaction after 52 rows returned only in memory. Reusing its task identities or sampling seeds would resample a terminal invocation.
Question
On fresh exact-depth-three tasks, can Qwen3.5-4B complete useful two-operation residuals when an external interpreter materializes every candidate first operation's public consequences, and can that all-sibling explorer beat name-only, token-preserving semantic derangement, and taskwise matched-compute ordinary sampling?
Hypothesis
Concrete public state-to-target relations should reduce a depth-three inverse problem to a depth-two suffix problem. If that mechanism is real, symmetric all-24 materialized completion should improve proposal coverage and visible/probe-only selected hidden accuracy beyond both representation controls and conservative sampled-token/logical-token first-over sampling. Cheap no-think top-four ranking remains secondary and cannot veto the all-24 test.
Setup
- Model: only
Qwen/Qwen3.5-4Bat revision851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a, bf16. - Backend: the pinned experiment-local vLLM runner for every model arm; no backend mixing and no training in this experiment.
- Scientific design: copy the parent's frozen arms, controls, stop gates, and matched-compute estimands without outcome-dependent relaxation.
- Freshness: regenerate every procedural task under seed block
2026072700--2026072709, change every request-ID namespace, and use the new mechanics sampling seed2026072702. Paired arms share identities only within this fresh successor. - Transaction repair:
STARTED -> generate -> durable GENERATED bundle -> re-read/authenticate -> COMPLETE. Failed quarantine bytes remain sealed and downstream analysis requiresCOMPLETE. - Termination receipt: independently lock model EOS
248044, tokenizer<|im_end|>EOS248046,ignore_eos=true, and explicit stop token[248044]before any model request. - Hidden boundary: no hidden output may influence prompts, stopping, resource matching, pool construction, selected IDs, or escalation. No benchmark content is read or used.
Controls and metrics
- Primary treatment: one strict two-operation suffix from all 24 materialized sibling states.
- Mechanism controls: all-24 name-only siblings and all-24 task-hash-deranged state/target alignments with the same token multiset.
- Baseline: candidate-blind full-program sampling matched taskwise at the first conservative sampled-token and logical-model-token overtake points.
- Dominance references: exact public viability and exhaustive CPU enumeration.
- Primary metrics: hidden-correct proposal coverage and visible/probe-only selected hidden accuracy.
- Claim-grade bar: untouched confirmation must beat every registered primary comparator under the parent's paired McNemar/Holm family and bootstrap gate.
Run
The historical identity-only scaffold receipt is preserved at runs/scaffold/summary.json. Two independent reviews now authorize this model-free construction command:
.venv/bin/python experiments/qwen35_4b_materialized_residual_sibling_search_fresh_replication/scripts/run.py --stage smokeThe crash-safe mechanics implementation now exists and passed three independent reviews. Its model-free preparation command is:
VLLM_ENABLE_V1_MULTIPROCESSING=0 .venv-vllm/bin/python experiments/qwen35_4b_materialized_residual_sibling_search_fresh_replication/scripts/run_mechanics.py --stage prepareThat command first passed and froze the original preoutcome. A later lock attempt failed closed on an incorrect historical source commit before creating a lock, raw directory, runner, model load, GPU initialization, or model request. After three independent adversarial reviews, the same command passed again as an append-only V2 recovery without changing any original prepared payload. The reviewed V2 code and receipts were published on main, both CI workflows passed, and the separate schema-V4 implementation lock was independently audited, committed, pushed, and cleared by both workflows. The one authorized live command was then:
VLLM_ENABLE_V1_MULTIPROCESSING=0 .venv-vllm/bin/python experiments/qwen35_4b_materialized_residual_sibling_search_fresh_replication/scripts/run_mechanics.py --stage runResults
The corrected construction passed in 93.6 seconds and froze 264 tasks: 24 mechanics, 48 qualification, and 192 confirmation. It found 3,526 eligible exact-depth-three behaviors, preserved the 88/88/44/44 live-sibling balance, and achieved 0.952 simulated compound power at the registered alternative. The authenticated parent comparison found zero shared task IDs, identity-free public instances, all-mechanics prompts, or terminal materialized prompts. Finite-DSL reuse is reported rather than hidden: 56 functions, 41 concrete triples, and 181 suffixes. The real tokenizer receipt records model EOS 248044, tokenizer EOS 248046, ignore_eos=true, and explicit stop [248044]. All 44 experiment tests pass. No benchmark was read, no model was loaded, and no model call occurred. This is readiness/provenance evidence, not a capability result. The manifest's locked runner hash exactly matches pushed construction commit e43c701e; a post-construction guard now verifies the frozen manifest/summary instead of rewriting them on rerun.
The mechanics preparation then froze 1,984 requests across nine arms, 676 unique canonical request identities/seed keys, 576 surface-control scores, 576 random-control scores, and 24 leave-one-task-out folds. All nine required parent/fresh intersections are zero, including the union of derived stage-one and stage-two seeds and the parent terminal rendered-token sequences. The preoutcome SHA-256 is 80647e830ccb90026b30b00ea674d22aa247eba925b4ebe38d6ddad8b49e0d0e. The full fresh suite now passes 108 tests and 96 subtests. Preparation records zero model loads and zero model calls; no live mechanics artifact exists.
The first lock attempt then failed closed because the generated construction manifest was looked up at pre-construction commit e43c701e rather than published construction commit 9fc288eb. Its zero-model incident receipt has SHA-256 056fd507e83eedbc45648bcb73b4972faa61d0d164b5e57a472dfa667583c5aa. The append-only V2 repair assigns the three generated construction outputs to 9fc288eb and all other frozen design/code files to e43c701e. Its preoutcome SHA-256 is 04d8ba59d212adac3193d88c19a38f58298fa18cbdd41321bf9e312bea72fe72. It proves exact equality with the original 12-file payload table and is byte-idempotent across reruns. The reviewed implementation lock has SHA-256 c1d1c25c3989ad9a748720ee733fc59245c08b7a4e9a88663d52fb5dd460f6e9 and binds implementation commit a8343ef9, all 31 critical files, both preoutcomes, the incident, and the exact 12/3 frozen-source split.
The live run durably completed all nine invocation chains and authenticated all 1,984 request rows. The terminal summary SHA-256 is 14cd0ba20521c4f57eb267becd612794fe4ed6105cd26d8a22a5577cb577209c. Three independent result audits reproduced every score file and gate and returned PASS_FOR_RESULT_PUBLICATION.
Mechanics A is MECHANICS_INTERFACE_INVALID. Materialized, name-only, shuffled, echo, and direct parsed 12/52, 7/52, 12/52, 20/52, and 7/24 rows, while contacting their answer caps 37/52, 42/52, 40/52, 28/52, and 17/24 times. Every thought hit its registered cap. Materialized, name-only, shuffled, and direct produced zero visible successes; echo produced 20/52.
Mechanics B is the clean CHEAP_SIBLING_RANKING_FAIL. Materialized recall@4 was 0.25694, below name-only 0.28125, shuffled 0.32292, listwise 0.27083, and surface 0.37500. It beat random by 0.14931, just below the +0.15 gate, while also missing the absolute recall, hit, task-support, and operation-support floors. Qualification and the top-four branch are unauthorized. Hidden, qualification, confirmation, and benchmark read receipts are all empty.
Interpretation
The generation result is an interface-invalid null, not a broad refutation of materialized residualization. Its descriptive zero successes among 12 parsed materialized rows are weak negative evidence only. The parse-immune ranking arm is a clean negative for this specific cheap behavioral viability ranker and retires the top-four branch. A successor must first select and freeze an echo-qualified answer seam on calibration tasks, then test residualization on disjoint tasks; this experiment cannot be rescued by more cap, parser changes, or threshold tuning.
Knowledgebase Update
- Program evidence, scorecards, backlog, and synthesis updated with the terminal interface/ranking split.
- Claim ledger updated: no. The repository's required claim re-grade remains outstanding, and this result does not license a broad mechanism claim.
Artifacts
configs/default.yamldata/procedural/manifest.jsonruns/scaffold/summary.jsonruns/smoke/summary.jsonruns/smoke/publication_receipt.jsonruns/mechanics/prepared/preoutcome_receipt.jsonruns/mechanics/lock_attempt_1_incident.jsonruns/mechanics/prepared/preoutcome_receipt_v2.jsonruns/mechanics/implementation_lock.jsonruns/mechanics/raw/runs/mechanics/scored/runs/mechanics/summary.jsonidea_intake.mdreports/artifact_manifest.yamlreports/mechanics_result_review.mdreports/report.mdsrc/scripts/tests/
Report
Rendered from reports/report.md
Summary
The separately registered recovery completed a fresh, durable mechanics run. Generation terminated at MECHANICS_INTERFACE_INVALID; the parse-immune cheap ranker terminated at CHEAP_SIBLING_RANKING_FAIL. Qualification and confirmation remained unopened. This is a clean negative for the cheap ranking mechanism and an interface-invalid generation null, not a capability gain or a broad refutation of materialized residualization.
Research Program Fit
The question remains whether externally materialized candidate consequences help Qwen3.5-4B compose residual programs and beat taskwise matched sampling. The parent design is the exact near-duplicate; this new directory is mandatory because the parent invocation may not be resampled.
Method
Preserve the parent's arms, controls, estimands, and stop gates while replacing every procedural task identity, request-ID namespace, and sampling seed. Add a durable GENERATED boundary before semantic authentication and bind the exact model/tokenizer EOS distinction into model-free and live receipts.
Results
The identity-only scaffold smoke passes. Initial independent design review then returned BLOCK: copied code reused every parent task ID, while the draft's zero parent function/triple/suffix rule made the four-live quota impossible. The corrected no-write audit constructs all 264 fresh task instances and reports zero parent task-ID, public-instance, mechanics-prompt, and terminal- prompt overlap. It discloses unavoidable finite-DSL reuse: 56 behavior functions, 41 triples, and 181 suffixes. Both independent follow-up reviews then authorized model-free construction only.
The written construction passed in 93.6 seconds. It froze 264 unique-function tasks (24/48/192 across mechanics/qualification/confirmation), 3,526 eligible exact-depth-three behaviors, exact 88/88/44/44 live-sibling balance, 34 independent exhaustive live audits, and 0.952 simulated compound power at the registered alternative. Required parent intersections are all zero; finite-DSL reuse remains 56/41/181. The real tokenizer receipt authenticates 248044 as the explicit model stop and 248046 as the ignored tokenizer EOS. All 44 experiment tests pass. Manifest SHA-256 is 5d4fb6a000ac4830d2f34e9f5235856ccea42fb400e6b7ee091ff1abad0f45c0. The locked runner hash matches pushed construction commit e43c701e; the subsequent no-rewrite guard verifies rather than regenerates these frozen bytes. Independent scientific and identity audits both passed the written construction for publication and mechanics implementation only.
Three later implementation audits initially blocked preparation. Their demonstrated identity, cross-stage seed, lock-symlink, canonicality, crash- durability, stale-authentication, and recovery-coverage findings were fixed. The final stable transaction accepts only four prefix states, re-durabilizes validated restart bytes before promotion, writes a canonical bundle before authentication, and proves all-nine-invocation recovery without constructing the runner. The full fresh suite passes 108 tests and 96 subtests.
The authorized model-free preparation froze 1,984 requests across nine arms, 676 unique request IDs/seed keys, 576 surface rows, 576 random-control rows, and 24 folds. All nine required parent/fresh intersections are zero, including any-stage derived seeds and terminal rendered token IDs. Preoutcome SHA-256 is 80647e830ccb90026b30b00ea674d22aa247eba925b4ebe38d6ddad8b49e0d0e. It records zero model loads/calls. No live mechanics have run.
The first clean lock attempt failed closed before creating a lock because its source map requested generated construction outputs from pre-construction commit e43c701e rather than publication commit 9fc288eb. The incident records no raw directory, runner, model/GPU activity, request, or sampled output and has SHA-256 056fd507e83eedbc45648bcb73b4972faa61d0d164b5e57a472dfa667583c5aa. Three independent adversarial reviews blocked and then passed an append-only V2 repair. Its split per-file source map is explicit, its V1 table equality is exact, and its preparation-only review gate precedes every write. Two real model-free V2 preparations produced the same receipt SHA-256, 04d8ba59d212adac3193d88c19a38f58298fa18cbdd41321bf9e312bea72fe72, while preserving every V1 payload byte and again recording zero model loads, calls, and sampled outputs. The independently audited implementation lock has SHA-256 c1d1c25c3989ad9a748720ee733fc59245c08b7a4e9a88663d52fb5dd460f6e9. It was committed, pushed, and cleared by repository and site CI before the live run.
All nine live invocations reached canonical COMPLETE. The authentication receipt binds 1,984 request rows, 2,304 model-scored ranking rows, 4,032 requested finite raw-logprob values, exact paired identities/seeds, runtime, preflight, preoutcome, and lock. The terminal summary has SHA-256 14cd0ba20521c4f57eb267becd612794fe4ed6105cd26d8a22a5577cb577209c.
Mechanics A: invalid generation interface
| arm | rows | parsed | cap contacts | visible successes | successful tasks |
|---|---|---|---|---|---|
| materialized suffix | 52 | 12 | 37 | 0 | 0 |
| name-only suffix | 52 | 7 | 42 | 0 | 0 |
| shuffled suffix | 52 | 12 | 40 | 0 | 0 |
| echo control | 52 | 20 | 28 | 20 | 13 |
| direct | 24 | 7 | 17 | 0 | 0 |
Registered suffix gates required at least 47 parses and at most two cap contacts; direct required at least 22 parses and at most one cap contact. All 208 suffix thoughts contacted 512 and all 24 direct thoughts contacted 1,024. Every cap-contact answer was unparsable. Echo succeeded on 20/24 non-cap rows, so parsing and some structured copying work, but the ABI is far below the required ceiling. The registered decision precedence therefore forces MECHANICS_INTERFACE_INVALID rather than a mechanism verdict.
Mechanics B: cheap ranking fails cleanly
| ranker | hit@4 | recall@4 | recall@8 | live-operation support |
|---|---|---|---|---|
| materialized | 0.29167 | 0.25694 | 0.41319 | 6 |
| name-only | 0.50000 | 0.28125 | 0.44097 | 7 |
| shuffled | 0.41667 | 0.32292 | 0.50347 | 9 |
| listwise | 0.41667 | 0.27083 | 0.43750 | 9 |
| surface | 0.54167 | 0.37500 | 0.63542 | 4 |
| random | 0.25000 | 0.10764 | 0.23611 | 5 |
The materialized recall@4 gain over random was 0.14931, narrowly below the registered +0.15 margin, while gains over name-only, shuffled, listwise, and surface were all negative. It also missed the 0.40 recall floor, 0.65 hit floor, 16-task support, and 10-operation support. Every score and raw log probability authenticated, so CHEAP_SIBLING_RANKING_FAIL is a clean negative for this specific cheap viability ranker. Top-four confirmation is forbidden.
Three independent result audits recomputed every scored row and summary byte, verified all nine transactions and identity domains, and reproduced the same summary through an offline restart with runner construction and new writes made fatal. All returned PASS_FOR_RESULT_PUBLICATION. Read receipts for hidden, qualification, confirmation, and benchmark content are empty.
Controls
The planned primary controls are all-24 name-only, all-24 token-preserving state/target derangement, and candidate-blind direct sampling at conservative sampled-token and logical-token first-over points. Exhaustive CPU search is a dominance reference. Top-four ranking is secondary.
Oracle Versus Deployable Evidence
Construction uses exact DSL enumeration and the registered target program as model-free provenance, but no model output or competing-candidate hidden/probe outcome enters task admission. Future prompts, resource matching, pools, and selections remain visible/probe-only; hidden outcomes may score only immutable choices.
Interpretation
The generation stage establishes no positive materialized-residual evidence, but its invalid interface prevents a broad refutation. The descriptive 0/12 successes among parsed materialized rows are weak negative evidence only: conditioning on parse/non-cap status is outcome-selected. The ranking stage is the stronger result. Materialized consequences did not improve the model's one-token viability judgment and were worse than every structured comparator. This retires the cheap ranker and top-four branch.
Next Experiments
Use a new experiment with fresh identities and an echo-gated answer-seam factorial. Compare the current protocol, short no-think structured emission, and an autonomous commit-slot emission on known-answer calibration rows; require at least 90% exact echo/parse and at most 5% cap contact; freeze the winner; then test materialized versus name-only and shuffled residual prompts on disjoint mechanics tasks. Do not rescue this run by raising caps, relaxing parsing, changing ranking thresholds, or opening downstream splits.
Artifact Manifest
artifact_manifest.yaml binds the historical scaffold, construction receipts, both preoutcomes, incident, implementation lock, live preflight, authentication, and terminal summary. All raw/scored JSON artifacts are in git; no external or omitted artifact exists.
Experiment log 2
Show the running log (2 entries)
Scaffold
- 2026-07-13: Reserved a separately registered fresh-identity replication under
structured_execution_and_compilersafter the parent experiment's independent incident review returnedBLOCKon replay. - 2026-07-13: Froze seed block
2026072700--2026072709, required a new request-ID namespace, and retained the parent's scientific arms, controls, estimands, and gates. No model was loaded or called. - 2026-07-13: Added an identity-only scaffold smoke and site metadata so the experiment ID can be pushed before harness copying or model authorization.
- 2026-07-13: Independent pre-construction review returned
BLOCK. All 264 copied task IDs collided with the parent, and rejecting every parent behavior function exhausted the four-live stratum at 5/44 tasks. - 2026-07-13: Replaced the impossible finite-grammar exclusion with measured descriptive reuse; made the versioned namespace operational; authenticated the complete 13-file parent incident lineage; and added fail-closed public- instance, prompt, EOS, symlink, inventory, and hash gates.
- 2026-07-13: A no-write, no-model audit filled 264/264 tasks and measured zero parent task-ID, identity-free public-instance, mechanics-prompt, and terminal prompt overlap. It disclosed 56 shared functions, 41 shared triples, and 181 shared suffixes. Twenty-nine focused tests passed. Follow-up independent review remains pending.
- 2026-07-13: Both follow-up reviewers returned
PASSfor model-free construction only after a cross-arm prompt-union defect and permissive prior scanner were fixed. Mechanics/model access remains sealed. - 2026-07-13: Published the review authorization, waited for green repository and site CI, then ran the model-free construction. It passed with 264 tasks, 3,526 eligible behaviors, 0.952 registered-alternative compound power, zero required parent intersections, 56/41/181 descriptive finite-DSL reuse, zero benchmark reads, zero model loads, and zero model calls. Manifest SHA-256 is
5d4fb6a000ac4830d2f34e9f5235856ccea42fb400e6b7ee091ff1abad0f45c0. - 2026-07-13: All 44 experiment tests passed against the written construction, including a no-rewrite verification of the frozen manifest and summary.
- 2026-07-13: Added a publication receipt proving the construction manifest's locked runner hash equals pushed commit
e43c701e; later documentation and no-rewrite guards are explicitly post-construction and did not rewrite data. - 2026-07-13: Independent scientific and identity audits passed every written construction hash, row count, overlap, EOS, and no-model boundary. They authorize mechanics implementation only, not model execution.
- 2026-07-13: Implemented the parent's byte-identical mechanics scoring and prompt science with fresh canonical request identities, exact parent request/ seed/prompt overlap gates, construction-frozen identity code, lexical lock- path safety, and the corrected 248044/248046 termination contract.
- 2026-07-13: Three independent implementation reviews initially blocked the harness. Demonstrated defects included arbitrary/cross-family request IDs, cross-stage seed collisions, lock symlink resolution, noncanonical bundle promotion, incomplete crash durability, a stale-authentication resampling path, and insufficient restart coverage. Every finding was fixed and mutation-tested before the reviewers returned preparation-only
PASS. - 2026-07-13: The stable transaction now classifies all 16 artifact states, re-durabilizes validated predecessors, recovers all nine generated invocations with runner construction forbidden, and rejects malformed, duplicate-key, hash-drifted, non-prefix, and corrupt-completion states. The full fresh suite passed 108 tests and 96 subtests.
- 2026-07-13: Ran the authorized model-free mechanics preparation. It froze 1,984 requests, 676 unique IDs/keys, zero in all nine registered parent/ terminal intersections, 576+576 control rows, 24 folds, and the exact EOS pair with zero model loads/calls. Preoutcome SHA-256 is
80647e830ccb90026b30b00ea674d22aa247eba925b4ebe38d6ddad8b49e0d0e. - 2026-07-13: The first clean lock attempt failed closed before lock creation: generated construction outputs were incorrectly requested from pre-construction commit
e43c701einstead of publication commit9fc288eb. No raw directory, runner, model/GPU activity, request, or sampled output was created. The append-only incident SHA-256 is056fd507e83eedbc45648bcb73b4972faa61d0d164b5e57a472dfa667583c5aa. - 2026-07-13: Three independent V2 adversarial reviews initially blocked the recovery on write-before-review ordering, lifecycle poisoning, missing exact V1 table equality, and dangling-symlink replacement. All four defects were fixed and mutation-tested before preparation-only
PASSverdicts. - 2026-07-13: Ran the authorized model-free V2 preparation twice. It preserved the original preoutcome and every one of its 12 payload bytes, produced the idempotent V2 preoutcome SHA-256
04d8ba59d212adac3193d88c19a38f58298fa18cbdd41321bf9e312bea72fe72, and again recorded zero model loads/calls and zero sampled outputs. - 2026-07-13: Generated schema-V4 implementation lock
c1d1c25c3989ad9a748720ee733fc59245c08b7a4e9a88663d52fb5dd460f6e9. Three independent audits verified all 31 critical files, the exact 12/3 e43/9fc frozen-source mapping, and zero pre-lock model calls. The exact lock was committed and pushed as55170c9c; repository and site CI passed. - 2026-07-13: Ran the one authorized live mechanics stage. All nine invocation transactions reached canonical
COMPLETE; authentication bound 1,984 rows, 2,304 ranking rows, and 4,032 requested finite raw-logprob values. - 2026-07-13: Automatic decisions were
MECHANICS_INTERFACE_INVALIDandCHEAP_SIBLING_RANKING_FAIL. Generation arms missed parse and cap gates by large margins. The parse-immune materialized ranker reached recall@4 0.25694, below every structured comparator and above random only by 0.14931 versus a +0.15 gate. Qualification and top-four confirmation stayed sealed. - 2026-07-13: Three independent result audits reproduced all scored files and summary SHA-256
14cd0ba20521c4f57eb267becd612794fe4ed6105cd26d8a22a5577cb577209cbyte-for-byte, verified restart idempotence without runner construction, and returnedPASS_FOR_RESULT_PUBLICATION.
Terminal next step
- Publish the complete negative transaction and scoring record.
- Open only a separately registered fresh echo-gated answer-seam successor; never relax this experiment's parser, caps, or ranking thresholds.
Data files 3
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
Reproduce
Smoke test
.venv/bin/python experiments/qwen35_4b_materialized_residual_sibling_search_fresh_replication/scripts/run.py --stage smokeFull run
VLLM_ENABLE_V1_MULTIPROCESSING=0 .venv-vllm/bin/python experiments/qwen35_4b_materialized_residual_sibling_search_fresh_replication/scripts/run_mechanics.py --stage runRun steps are documented inside the experiment folder (README and scripts).