Research log Small Model Experimentation
GitHub

Qwen3.5-4B Tokenizer-EOS Answer Commit Factorial

Boundary works; capability test failed instrumentally

The one idea you need

Qwen emits a chat-end marker before a later model-end marker. Waiting for the later marker made correct short programs look as if they contained extra text. This experiment stopped at the first marker, then tried to use that repaired answer boundary in a harder reasoning comparison.

The question

Does stopping at the first chat-end marker create a strict program answer and unlock the harder residual task?

What we found

The boundary worked cleanly: both no-thinking chat-end conditions produced 48 correct strict answers out of 48, while every matched later-end condition produced zero. The harder transport check also scored 24 of 24. But the final comparison never completed because the analysis code incorrectly demanded that later completed stages be absent.

Why it matters

The end marker is a real part of the usable interface, but this run says nothing about a new reasoning capability. All 4,056 harder-task outputs remain quarantined, and a fresh successor is required.

No-thinking chat-end answers48 / 48strictly correct in both prefix conditions
Matched later-end answers0 / 48failed in every factorial condition
Fresh transport check24 / 24exact answers with zero cap contacts
Quarantined harder-task outputs4,056not scored because visible analysis failed
On this page
  1. Results at a glance
  2. Overview
  3. Report
    1. Summary
    2. Research Program Fit
    3. Method
    4. Results
    5. Controls
    6. Oracle Versus Deployable Evidence
    7. Interpretation
    8. Next Experiments
    9. Artifact Manifest
  4. Experiment log
  5. Data files
  6. Reproduce
  7. Related

Results at a glance 1

Only one synthetic trace is both physically valid and exactly parsed

How to read

Each category is one frozen boundary case. The first bar says whether its stop-token geometry could be real; the second says whether every pre-commit token is exactly the requested answer.

00.250.50.751tokenizer cleantokenizer clean11HF boundary controlHF boundary control10early stopearly stop10interior + terminalinterior + terminal00missing stopmissing stop00extra precommit byteextra precommit byte10

Takeaway → Only the clean first-chat-end case passes exactness. Valid-looking old, early, or extra-byte traces still fail the strict answer grammar.

Data table
frozen synthetic boundary casephysically valid first-stop tracestrict exact precommit answer
tokenizer clean11
HF boundary control10
early stop10
interior + terminal00
missing stop00
extra precommit byte10

Numbers from experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/runs/smoke/summary.json

Technical framing

Strict first-stop smoke accepts only the clean tokenizer-EOS answer — Model-free protocol smoke only. Tokenizer EOS is scoped to the answer stage; no model was loaded or called.

In the author’s words from the Overview · “Results”

After six prospective holds, global independent rereview returned PASS_DESIGN for exact pushed/green commit abd2ffcd. No model or capability result exists; live execution remains sealed behind implementation release review and a committed-green lock. Model-free construction then passed with 48 calibration and 24 mechanics tasks, the frozen 8/8/4/4 strata in each calibration arity and mechanics, every A-X alias once per answer position, zero overlap with all 72 predecessor public fingerprints, zero request/seed/prompt/derived-seed overlap, and a distinct transport namespace. Mechanics gold exists only as tracked AES-256-GCM ciphertext; its key is local and ignored. Model calls and sampled outputs remain zero. … Read the full result →

Overview

Calibration qualified the tokenizer-EOS-only interface; mechanics transport passed 24/24, but visible selection ended in terminal instrument failure after all five transactions. Residual capability remains unadjudicated.

This fresh successor tests whether the prior strict answer-seam failure was caused by waiting past Qwen3.5's tokenizer chat-end token. It registers the first tokenizer EOS only during answer generation, preserves every pre-commit byte under the strict grammar, and compares it with the existing HF-model-EOS boundary on new calibration and residual-mechanics identities.

Research Program

  • Program: structured_execution_and_compilers
  • Program question: can a token-native output boundary expose a deployable structured compiler interface without supplying answer identity?
  • Closest result-bearing predecessor: qwen35_4b_materialized_residual_answer_seam_factorial.
  • Prior evidence: the predecessor was 0/48 strict in every registered arm, but suffix-only removal gave 48/48 frozen-parser exactness in both paired no-think cells; thinking remained 38/48 and 24/48 because of extra closes.

Question

On fresh known-answer rows, does stopping the answer stage at the first tokenizer EOS (248046) create a strict exact program interface that the matched HF-model-EOS (248044) boundary does not—and, only if it qualifies, does that interface permit a disjoint materialized-residual capability test?

Hypothesis

Qwen3.5 naturally commits short structured answers with <|im_end|>. The predecessor treated that token and the following newline as answer content because it waited for <|endoftext|>. Registering first-248046 answer-stage stopping should lift no-think exact/parse from the predecessor's 0/48 to at least 44/48 without changing prompts, answer identity, or thought generation. This is an interface hypothesis, not yet a capability hypothesis.

Setup

  • Model: only Qwen/Qwen3.5-4B revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a, bf16.
  • Backend: pinned repository vLLM for every paired arm and matched-compute baseline.
  • Dataset/task source: fresh procedural exact-depth-three list transforms; never benchmarks/.
  • Frozen fresh splits: 48 known-answer calibration tasks and 24 disjoint residual-mechanics tasks, with new functions, task IDs, record IDs, token sequences, and seed domain.
  • Factorial: answer boundary {first tokenizer EOS, HF model EOS control} x thinking {off, forced close at 512} x answer prefix {freeform, PROGRAM:}.
  • All 192 boundary pairs must share prompts, answer seeds, persisted thought IDs, request adjacency, and sampled token prefix through the earliest stop or cap; one mismatch terminates the experiment before qualification.
  • Every task has one persisted thought transaction reused by all four thinking answer cells. Natural post-close answer content is discarded and cannot bypass the answer-stage comparison.
  • Answer-stage tokenizer EOS never applies to thought generation.
  • Calibration gates: >=44/48 strict exact echoes, >=44/48 parses, <=2/48 answer-cap contacts, plus >=22/24 exact/parse and <=1/24 cap contact in each arity.
  • Parse is membership in a frozen token-ID grammar for any A-X program of the registered arity; exact is equality to the known answer. A stop on sampled token 24 still counts as a cap contact.
  • Eligible winner: first qualifying tokenizer-EOS arm in the frozen no-think PROGRAM:, no-think freeform, think PROGRAM:, think freeform priority.
  • Boundary controls: matched HF EOS, first-stop uniqueness, early/interior/ missing tokenizer EOS, extra pre-commit bytes, and exact stop/token/cost receipt authentication.
  • Conditional mechanics is fully frozen at 24 tasks, 24 candidates per suffix arm, and a 96-row-per-task direct ceiling. Controls are materialized state, name-only, shuffled materialized state/target, candidate-blind direct samples, exhaustive CPU ceiling, and taskwise sampled/logical-token matched- compute prefixes.
  • Every suffix is bound to its semantic candidate first operation before any deduplication or scoring, yielding the same canonical full three-operation proposal type used by direct sampling.
  • Primary capability metric if opened: hidden exact accuracy of a frozen visible-only selector versus all structured controls and matched-compute direct sampling.
  • Hidden-label boundary: mechanics remains sealed until a committed-green calibration decision and second winner-bound lock; hidden labels remain sealed until a committed-green visible-selection receipt.

The prefix cells share tasks and seed derivation and are paired conditions, not independent replications. Only tokenizer-pass/HF-fail in the selected matched cell supports a causal termination-boundary claim. Dual qualification within one matched thinking/prefix pair is impossible under the authenticated-prefix, exactness, and cap gates: shared exact-cap overlap is <=2 globally and <=1 per arity, yielding 44+44-2>48 and 22+22-1>24. Any observed dual qualification terminates as a scoring invariant violation.

Historical reproduction commands

This experiment is finished and its result-bearing mechanics transaction must not be executed or rerun. The commands below document historical provenance only. Use the fresh successor named in the report for any further model call.

Model-free smoke:

python3 -B experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/run.py --smoke

Fresh construction:

python3 -B experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/construct.py

Historical calibration entry points:

experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/calibration_launcher --stage lock
experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/calibration_launcher --stage run
experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/calibration_launcher --stage analyze

Historical winner-bound mechanics entry points (do not execute):

experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/mechanics_launcher --stage lock
experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/mechanics_launcher --stage run
experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/mechanics_launcher --stage analyze-visible
# Hidden scoring was never authorized and must not be executed:
experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/mechanics_launcher --stage score-hidden

calibration_launcher is a 12.6-KiB static x86-64 ELF with no dynamic interpreter. It retains a waiting static parent, opens its exact executable on inherited descriptor 198, discards the inherited environment, and directly execves the pinned Python interpreter with -I -B in a child protected by a parent-death signal. The Python bootstrap requires the live parent executable, the inherited descriptor, and the tracked launcher path to be the same stable inode with the reviewed SHA-256. This kernel-carried provenance cannot be forged by an environment marker. Its source is scripts/calibration_launcher.S; its exact reproducible build command is:

/usr/bin/gcc -nostdlib -static -no-pie -s -Wl,--build-id=none \
  -o experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/calibration_launcher \
  experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/calibration_launcher.S

The mechanics launcher applies the same kernel-carried provenance design to the winner-bound runner. Its exact reproducible build command is:

/usr/bin/gcc -nostdlib -static -no-pie -s -Wl,--build-id=none \
  -o experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/mechanics_launcher \
  experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/mechanics_launcher.S

Direct Python invocation—even with a caller-supplied marker and an open copy of the launcher on descriptor 198—fails because its live parent is not the static launcher. Missing/malformed stages, non-isolated Python, altered launcher bytes, and dynamic-loader variables also fail before local imports.

Results

The model-free protocol smoke passes with zero model calls. It accepts only a clean expected answer followed by one terminal registered stop token. The matched HF boundary preserves tokenizer EOS/newline as strict answer content and therefore fails exactness. Early stops, interior-plus-terminal stops, missing claimed stops, and extra pre-commit bytes all fail exactness or their registered contract as appropriate. A unique final early stop is authenticated but scored; an exact-cap length trace is authenticated with all tokens retained as content.

After six prospective holds, global independent rereview returned PASS_DESIGN for exact pushed/green commit abd2ffcd. No model or capability result exists; live execution remains sealed behind implementation release review and a committed-green lock.

Model-free construction then passed with 48 calibration and 24 mechanics tasks, the frozen 8/8/4/4 strata in each calibration arity and mechanics, every A-X alias once per answer position, zero overlap with all 72 predecessor public fingerprints, zero request/seed/prompt/derived-seed overlap, and a distinct transport namespace. Mechanics gold exists only as tracked AES-256-GCM ciphertext; its key is local and ignored. Model calls and sampled outputs remain zero.

Before any model call, the prospective runner was security-hardened to pin child executables/environment and preserve isolation across recovery re-exec. The tokenizer receipt's sole runner binding was correspondingly refreshed from cbbfae3e... to 4ce61e64...; its grammar, prompts, termination IDs, freshness inventories, and zero-call declarations were unchanged.

A later clean implementation rereview held live calls because an in-Python LD_PRELOAD rejection occurs after interpreter startup and because several zero counters still accepted JSON Boolean aliases. The prospective static launcher now sanitizes before Python exists, and exact integer predicates plus typed-canonical engine comparisons reject those aliases. These are model-free repairs; no calibration request or sampled output has yet occurred.

The next clean rereview held again because the first static-launch repair used a caller-controlled environment marker to assert entry. The parent-plus-open- executable proof above replaces that assertion with kernel process/file state that remains valid across the sanctioned Mamba recovery execve. The repair is still model-free; live authorization remains absent.

The seventh clean review then returned PASS_IMPLEMENTATION for exact pushed-green commit d70756122bc768e82fa4d77a61e05522ef5bca79: 95/95 permitted tests passed, the launcher rebuilt byte-identically, all protected read inventories were empty, and model/GPU calls remained zero. The PASS does not itself authorize generation; its canonical hash-bound receipt and the subsequent implementation lock must each be committed, pushed, and green.

The independently committed receipt and implementation lock both passed their two required workflows. The sealed live calibration then completed 48 shared- thought requests plus 384 paired answer requests. All 192 boundary pairs and the five-invocation transaction chain authenticated. Results were:

CellExactParseCap contactsArity-2 exactArity-3 exact
tokenizer EOS · no-think · PROGRAM:48/4848/48024/2424/24
tokenizer EOS · no-think · freeform48/4848/48024/2424/24
tokenizer EOS · think512 · PROGRAM:38/4838/48018/2420/24
tokenizer EOS · think512 · freeform30/4830/481615/2415/24
HF model EOS · no-think · PROGRAM:0/480/4800/240/24
HF model EOS · no-think · freeform0/480/4800/240/24
HF model EOS · think512 · PROGRAM:0/480/4800/240/24
HF model EOS · think512 · freeform0/480/48160/240/24

The frozen decision is TOKENIZER_EOS_ONLY_INTERFACE_QUALIFIED; the winner is tokenizer_eos_no_think_program_slot, and its matched HF control is hf_model_eos_no_think_program_slot. This opens the preregistered conditional mechanics branch only after this calibration result and a second winner-bound lock are committed, pushed, and green.

The first exact-commit conditional-mechanics review returned HOLD_IMPLEMENTATION for fd06b5053c9327c61775d07061c9a84e070cdcb6. It found four live-stage deadlocks (an incomplete path-audit allowlist, an invalid calibration-verifier call, and two Python-tuple/JSON-list durable comparison failures), partial type/schema authentication, a missing registered direct-pool-exhaustion terminal, and incomplete resource receipts. The prospective repairs add exact calibration/review support reads, a scoped and tested immutable-verifier adapter, JSON-domain receipts, exact typed engine/prompt/seed/token/terminal/cost authentication, omission of unrequested likelihood diagnostics, a durable DIRECT_RESOURCE_MATCH_POOL_EXHAUSTED receipt, and explicit overshoot/row-ID inventories. The model-free suite now passes 124/124. These repairs do not authorize mechanics; a fresh pushed-green exact-commit adversarial review and second lock are still required.

The round-two review of pushed-green commit 3d2f051203c56456fad716e20950b55d5717afd5 verified all seven earlier repairs and held on three new exact-type replay failures: the recorded mechanics preflight authenticated only a runtime subset, visible authorization inherited Python's Boolean/integer equality, and the calibration-era generic transaction envelopes had the same alias at STARTED and bundle schema boundaries. The prospective repair now reconstructs the entire clean preflight from the live runner and exact-compares every JSON-domain field after authenticating recorded CI. Visible authorization also uses recursive exact-type comparison. Because the generic transaction source is an immutable calibration-critical file, mechanics routes through a new additive exact-typed transaction layer rather than altering the calibration anchor. Durable hostile tests cover all ten preflight mutations, visible true versus 1, STARTED/bundle aliases, receipt aliases, and registered row-count aliases. The full model-free suite passes 130/130; mechanics calls and protected reads remain zero. A third fresh pushed-green exact-commit review is still required.

That round-three review of pushed-green 9b527cbfd6934f579b0ebcb07cb4b695c370798b returned HOLD_IMPLEMENTATION despite confirming all round-two closures. It found that production tuple-valued logprob_token_ids would mismatch the JSON-native STARTED sampling after the first call, that hidden scoring reread the visible receipt after authorization, and that the low-level transaction primitive authenticated only a predecessor's state before a fresh successor call. The repairs normalize the complete fresh generated bundle before its first validation/write, pass the exact in-memory authorized visible object directly into scoring, and fully authenticate plus recheck the predecessor chain inside the primitive itself. Tests now use the actual frozen mechanics sampling plan, assert no second visible read, and corrupt a predecessor before a fresh successor. The full model-free suite passes 134/134; mechanics calls and protected reads remain zero. A fourth pushed-green exact-commit review is required before any lock or generation.

Round-four probes confirmed all production-shape and static-corruption repairs, then held on two narrower concurrent-mutation windows: a predecessor could change inside generation or promotion after the pre-call recheck, and visible authorization compared HEAD:path before separately resolving the commit recorded in its receipt. The primitive now rechecks the authenticated predecessor immediately after generation and again before returning from every promotion/recovery path. Visible authorization resolves one commit exactly once and uses that immutable commit ID for its blob comparison and receipt. New callbacks mutate the predecessor during generation and at COMPLETE publication; both are detected before bundle publication or successful return. The full model-free suite passes 136/136 with zero mechanics calls or protected reads. A fifth pushed-green exact-commit review is required.

Round five confirmed every earlier regression family but found that the pre-return predecessor recheck covered only the terminal COMPLETE file. A concurrent change to an already-authenticated predecessor STARTED, bundle, or GENERATED artifact could therefore escape the primitive's successful return until final chain authentication. The primitive now reruns exact authentication over the entire predecessor prefix at each recheck and compares the resulting receipt with the original authenticated snapshot. New regressions change each non-terminal predecessor artifact during generation, publication, or recovery; all fail closed without a successful successor return or recovery resample. The model-free suite passes 139/139 with zero mechanics model/GPU calls or protected reads. A sixth pushed-green exact-commit review is required.

Round six returned PASS_IMPLEMENTATION for exact pushed-green commit df096d330f09847ce844af6255b349b4f707f464. It passed 139/139 tests, rebuilt the static launcher byte-identically, rejected all full-prefix changes across pre-call, callback, publication, and recovery checks, and reconfirmed the production tuple path, hidden object/commit binding, exact typed gates, resource receipts, immutable calibration files, and bootstrap routing. All protected-read arrays and model/GPU counters were zero. The hash-bound review report and canonical receipt must now be committed, pushed, and green before the mechanics lock can be published; this PASS alone does not authorize a model call.

After that receipt was published green, the first lock-only invocation failed closed before creating a lock: calibration decision recomputation retained 32 integer dictionary keys while canonical JSON represented those object keys as strings. There were zero actual tuple/list mismatches. The lock compared across that serialization boundary without first entering the JSON domain. The prospective mechanics verifier now JSON-normalizes the recomputation and then applies recursive exact- typed comparison, preserving rejection of Boolean/integer aliases. A direct model-free recomputation now authenticates the frozen TOKENIZER_EOS_ONLY_INTERFACE_QUALIFIED decision and winner. No mechanics data, model, or GPU call occurred. Because this changes reviewed mechanics code, a fresh exact-commit implementation review and replacement receipt are required before retrying the lock. The model-free suite passes 140/140.

Round seven verified that real decision repair, rejecting 20/20 typed aliases and four semantic mutations, and completed a read-only prospective lock build. It nevertheless held on the next exact-type boundary: durable sampling-plan validation used ordinary Python equality, accepting five n=1 to true aliases and 25 Boolean-to-integer aliases across the five arms. Sampling-plan validation now uses recursive exact JSON equality. The real-plan regression mutates both directions for all 30 fields and requires every case to fail. No mechanics payload, model, or GPU call occurred; another exact-commit review and replacement receipt remain required.

Round eight returned PASS_IMPLEMENTATION for exact pushed-green commit 3e7b650a90ff1d65fe371552354895756efcf728. It passed 140/140 tests; rejected all 58 real-plan type/schema cases and all five semantic seed mutations; reauthenticated the real calibration decision with the correct 32 key conversions; and completed read-only lock build/validation over 29/29 current critical files. Full-prefix, production durability, hidden binding, preflight, resource, calibration, routing, launcher, and static checks also passed. All mechanics/protected read inventories and model/GPU counters remained zero. The replacement hash-bound receipt must be committed, pushed, and green before the lock-only stage is retried.

After that replacement receipt was published green, the next lock-only attempt failed closed while hashing reviewed critical evidence: the pre-import path audit allowed all 22 runtime files and 11 support files but omitted the seven reviewed mechanics test files in the 29-file critical inventory. The lock was not written, and no mechanics payload, model, or GPU call occurred. A separate exact seven-file critical-test allowlist now permits only those review inputs; it does not widen runtime imports, prepared/mechanics data, or hidden access. The bootstrap test proves that this tuple is exactly the critical-minus-runtime set and is actually consumed by the path audit. Because bootstrap-reviewed code changed, another exact-commit review and replacement receipt are required. The model-free suite passes 141/141.

Round nine returned PASS_IMPLEMENTATION for exact pushed-green commit c0075a019fd0f202c3b0e6cf0be5528e08c61649. It passed 141/141 tests; proved the exact seven-file critical-test inventory; hashed all 29 critical files through the active audit while denying neighboring files; preserved the 22/11 runtime/support inventories; and passed every prior typed, semantic, full-prefix, durability, hidden-binding, calibration, launcher, and resource regression. The prospective lock build/validation was read-only, and mechanics payload reads, model/GPU calls, and protected-read inventories remained zero. The replacement hash-bound receipt must be committed, pushed, and green before the lock-only stage is retried.

The replacement receipt was committed, rebased, pushed, and green. An initial lock retry then failed closed because the shared-main rebase advanced HEAD to a commit whose Validate Repository workflow was still running; it wrote no lock and made no mechanics/model/GPU call. After both workflows for that exact HEAD passed, lock-only publication succeeded. The canonical implementation lock has SHA-256 d42e2db4b589e470f42d963b19e01a8b880fa7858a40b10966150c775c3d925b, binds all 29 critical and 22 runtime files, and records zero pre-lock generation requests, sampled outputs, and protected reads. Live mechanics remains unauthorized until this lock is itself committed, pushed, and green.

The lock was committed and both exact-commit workflows passed before live mechanics. The selected interface then transported perfectly: 24/24 exact echoes, 24/24 parses, and zero answer-cap contacts, balanced 12/12 in each arity. All five durable transactions completed and preserved 4,056 sampled outputs. Visible analysis nevertheless failed before selection because it recomputed the already-recorded transport decision through the initial authorization path, which correctly requires every later invocation to be absent; at replay time direct and all suffix invocations were already complete. No visible selection, resource decision, hidden score, hidden read, or benchmark read occurred. This is terminal instrument failure, not evidence for or against residual capability. The generated result is preserved, and a fresh-identity successor must separate initial transport authorization from post-chain replay authentication under a new review and lock.

Interpretation

The fresh result isolates a real termination-boundary effect: with identical paired prompts/seeds and authenticated shared prefixes, stopping on tokenizer EOS yields 48/48 strict structured answers while waiting for HF model EOS yields 0/48 because tokenizer EOS/newline remains pre-commit content. This is an interface qualification, not a capability gain. Thinking is actively worse at this interface (38/48 and 30/48, including 16 freeform cap contacts), so the no-think PROGRAM: winner was the only branch that advanced from calibration. That branch is now frozen after terminal instrument failure; only the fresh-identity successor may continue it.

Any conditional mechanics pass is scoped to a 24-task contamination-free large-effect pilot; it is not a confirmatory or general deployability claim.

Knowledgebase Update

  • Program evidence, backlog, scorecards, and shared synthesis now record the committed-green calibration and transport evidence, terminal mechanics instrument failure, and fresh-successor requirement.
  • Claim-ledger allocation is deferred: the repository's knowledgebase protocol forbids adding or promoting claims until the outstanding adversarial re-grade checklist is processed. No claim ID is consumed, avoiding a collision while preserving the result at program level.

Artifacts

  • src/protocol.py
  • scripts/run.py
  • configs/default.yaml
  • runs/smoke/summary.json
  • reports/preregistration.md
  • reports/design_review.md
  • reports/mechanics_failure_review.md
  • reports/artifact_manifest.yaml
  • runs/calibration/implementation_lock.json
  • runs/calibration/live_preflight.json
  • runs/calibration/decision.json
  • src/mechanics_runtime.py
  • src/mechanics_stage.py
  • src/mechanics_lock.py
  • scripts/run_mechanics.py
  • scripts/mechanics_launcher
  • runs/mechanics/failure.json

Report

Rendered from reports/report.md

Summary

Fresh sealed calibration passed: the tokenizer-EOS no-think interface was 48/48 exact and parseable in both prefix cells, while every matched HF-model- EOS control was 0/48. All 192 boundary pairs authenticated. The frozen decision is TOKENIZER_EOS_ONLY_INTERFACE_QUALIFIED, with tokenizer_eos_no_think_program_slot selected for the conditional mechanics stage. This establishes an interface effect, not yet a capability gain.

Research Program Fit

This is the required fresh successor to the terminal HF-EOS answer-seam factorial. It changes one causal boundary while preserving strict grammar and the full residual-mechanics firewall.

Method

The prospective design crosses answer boundary, thought policy, and answer prefix on fresh known-answer rows. First-stop uniqueness and all pre-commit bytes are strict. Mechanics is conditional on an independently qualified tokenizer-EOS interface.

Results

The committed-green implementation receipt and lock authorized 48 shared- thought requests and 384 paired answer requests. The transaction chain contains 432 sampled outputs across five invocations; all receipts, runner metadata, token costs, paired prefixes/seeds, thought-source rows, and terminal events authenticated.

CellExactParseCapsArity 2 exactArity 3 exactQualifies
tokenizer EOS · no-think · PROGRAM:48/4848/48024/2424/24yes
tokenizer EOS · no-think · freeform48/4848/48024/2424/24yes
tokenizer EOS · think512 · PROGRAM:38/4838/48018/2420/24no
tokenizer EOS · think512 · freeform30/4830/481615/2415/24no
HF EOS · no-think · PROGRAM:0/480/4800/240/24no
HF EOS · no-think · freeform0/480/4800/240/24no
HF EOS · think512 · PROGRAM:0/480/4800/240/24no
HF EOS · think512 · freeform0/480/48160/240/24no

The winner is the first qualifying cell in the preregistered priority: tokenizer_eos_no_think_program_slot. Its matched control is hf_model_eos_no_think_program_slot. Calibration decision SHA-256 is 2195c3d15ad9ece124ad99244baf637076963e53fc2532f1131ea01226d11ee5. No mechanics, hidden label, qualification, confirmation, or benchmark content was read.

Conditional mechanics

After a separate nine-round implementation review and committed-green winner-bound lock, live mechanics completed all five durable transactions and preserved 4,056 sampled outputs. The transport gate independently confirmed the selected interface: 24/24 exact echoes, 24/24 parses, zero cap contacts, and 12/12 successes in each arity.

Visible selection then stopped with terminal instrument failure. Its replay authentication recomputed the stored transport decision through the initial authorization path, whose temporal invariant correctly requires all later invocations to be absent. By then direct and all three suffix invocations were durably complete. Thus no visible selection or resource decision was written, hidden scoring was never authorized, and hidden and benchmark reads remained zero. The residual-capability comparison is unadjudicated.

Controls

Matched HF EOS, all-192-pair fail-closed authentication, one persisted thought per task, malformed stop traces, strict arity-parametric token content, fresh identities, same backend, and fully frozen conditional matched-compute mechanics are preregistered.

Oracle Versus Deployable Evidence

The 48 known-answer rows and the 24-row transport gate are interface instruments, so their perfect scores demonstrate a deployable strict answer boundary on this fresh substrate but not new task capability. Mechanics candidates were sampled, but the failed visible selector left hidden labels and oracle ceilings sealed. Only a fresh successor with a valid winner-bound residual-mechanics result can support a capability claim.

Interpretation

The fresh matched result supports the causal interface hypothesis. The paired arms generated the same answer-token prefix through the earliest stop, but the HF boundary retained tokenizer EOS plus newline as answer content and therefore failed strict grammar/exactness on all 48 rows. Tokenizer EOS commits exactly the intended alias sequence on all 48 rows in both no-think prefixes.

Thinking does not magnify this interface. The structured think512 cell fell to 38/48 and freeform to 30/48 with 16 cap contacts. This reinforces a practical lesson already visible elsewhere in the corpus: extra reasoning can damage a short exact-output channel even when it sometimes helps semantic reasoning.

Third-pass review additionally required one canonical proposal type. Every two-operation suffix is now bound to its semantic candidate first operation, and all suffix/direct proposal processing uses the resulting full three- operation tuple.

Fourth-pass review separated authenticated terminal geometry from answer correctness: unique-final early stops and exact-cap length events are scored, while malformed stop/length/reason geometry is an authentication failure.

Fifth-pass review replaced tokenizer-dependent selector identity with a semantic base-24/two-byte program ID and corrected the dual-qualification proof to account for the at-most-two exact-cap overlaps.

Sixth-pass global review aligned the intake and README summaries with that cap-bounded proof. At that pre-execution checkpoint no scientific or implementation result existed; the later calibration and mechanics records supersede that historical state.

Seven implementation review rounds (five countable) ended in exact-commit PASS_IMPLEMENTATION; the machine receipt, implementation lock, and their required CI workflows all authenticated before model execution.

Fresh model-free construction now passes. It produced 72 predecessor-disjoint tasks, all frozen calibration/transport/mechanics request inventories, balanced arity/stratum/alias receipts, and zero request/seed/prompt/derived-seed collisions. Mechanics gold is sealed as tracked AES-256-GCM ciphertext with an ignored local key. Those were pre-execution facts. The authorized live run later preserved 4,056 sampled mechanics outputs but failed before visible selection, as reported above.

Next Experiments

Create a fresh-identity successor rather than altering this result-bearing experiment. Preserve the initial generation-time invariant that transport must be the sole completed invocation; add a distinct replay path that authenticates the exact stored transport decision against the transport bundle while the separate full-chain verifier requires all later invocations complete. Mutate every temporal state in tests, obtain a new adversarial review and lock, and use new task IDs, record IDs, token identities, seeds, ciphertext, and key. The capability test must still beat every structured control and taskwise sampled- and logical-token-matched direct baseline.

Artifact Manifest

All calibration and failed mechanics artifacts are tracked. The only external artifact is the ignored AES-256 key; it was not used because visible selection never authorized hidden scoring.

Experiment log 2

Show the running log (2 entries, 2026-07-14)

2026-07-14

  • Created only after the predecessor published terminal NO_VALID_RESIDUAL_ANSWER_SEAM and passed adversarial interpretation review.
  • Registered the predecessor as the closest near-duplicate and changed only the answer-stage stopping hypothesis on fresh future identities.
  • Added a model-free first-stop/strict-precommit smoke with HF-EOS, early, interior/repeated, missing, and extra-precommit controls. No model was loaded or called; live execution remains held pending adversarial design review.
  • Independent adversarial review returned HOLD_DESIGN: boundary pairing was not fail-closed, thinking was not shared across all four continuation cells, grammar/config arity disagreed, and conditional mechanics was adaptable.
  • Prospectively froze all 192 causal pairs, one persisted thought per task, arity-parametric token grammar, new namespaces/seeds, full 24-task mechanics geometry, resource matching, inference, terminal outcomes, and lock order.
  • Extended the zero-call smoke to accept tokenizer-first, HF-first, and shared- cap paired traces while rejecting divergence and false length claims. Live calls remain held pending independent rereview and implementation review.
  • Exact-commit rereview returned a narrower HOLD_DESIGN. Prospectively froze the impossible-dual-qualification invariant, parse/cap semantics, per-arity alias and live-operation balance, disjoint transport namespace, arm-blind selector/all-eight-row success, direct-pool exhaustion terminal, and 24-task pilot claim scope. Model calls remain zero.
  • Third-pass review verified those repairs but held on suffix/direct proposal identity. Prospectively froze suffix binding to the semantic candidate first operation and canonical full-tuple identity for every downstream selector, execution, hash, hidden/oracle score, and support count. Model calls remain zero.
  • Fourth-pass review verified canonical proposal identity but held on terminal classification. Prospectively separated authenticated unique-final stops, authenticated exact-cap length traces with all tokens scored, and malformed geometry authentication failures. Model calls remain zero.
  • Fifth-pass review verified terminal classification but held on canonical hash serialization and an invalid disjointness proof. Prospectively froze an injective base-24/two-byte program ID for all 13,824 semantic triples and the correct cap-bounded-overlap impossibility proof. Model calls remain zero.
  • Sixth-pass focused review passed those repairs, while the global scan held on stale abbreviated proofs in the intake and README. Aligned both with the <=2/<=1 cap-overlap bounds and 86>48/43>24 arithmetic. Model calls remain zero.
  • Seventh-pass global review returned PASS_DESIGN for exact pushed/green commit abd2ffcd8bf4aa35c7f79e175a99cf6614c2ae27; both workflows succeeded and the immutable archive passed 35/35 tests. Construction is authorized, but live calls remain sealed pending implementation review and lock.
  • Ran deterministic model-free construction under seed 2026140700: 48 calibration plus 24 mechanics tasks passed exact-depth, uniqueness, per-arity 8/8/4/4, alias-position, predecessor-fingerprint, request, prompt, transport- namespace, and 5,904-derived-seed collision gates. Sealed 24 mechanics-gold rows as AES-256-GCM ciphertext under an ignored local key. No model was loaded or called and hidden plaintext was not written.
  • The committed-green calibration qualified only tokenizer_eos_no_think_program_slot: 48/48 exact and parse with zero cap contacts, while its matched HF-model-EOS control was 0/48. This is an interface result, not a capability result.
  • Implemented a prospective winner-bound mechanics candidate behind a second exact-commit lock. It uses the immutable calibration runner's registered tokenizer-EOS primitive, authenticates exact prompts/seeds/tokens/costs and terminal geometry, and preserves the frozen selector's canonical two-byte semantic program-ID tie-break independently of candidate row order.
  • Added a static parent/inode-provenance launcher, pre-import runtime hashing, public/mechanics/hidden path-audit boundaries, exact review and CI receipts, and model-free hostile controls for internal EOS, Boolean integer aliases, selector order, canonical pre-import receipts, lock ancestry, and hidden-read authorization. The full local experiment suite passes 118/118. Mechanics requests, outputs, protected arrays, ciphertext reads, key reads, hidden plaintext reads, and model calls remain zero pending a fresh independent adversarial implementation review.
  • The first conditional-mechanics exact-SHA review returned HOLD_IMPLEMENTATION for pushed-green commit fd06b505. It identified an incomplete path-audit support allowlist, a nonexistent calibration-verifier keyword, persisted tuple/list mismatches in generation and visible receipts, partial type/schema/logprob checks, a missing durable direct-pool-exhaustion terminal, and absent explicit overshoot/direct-row-ID receipt fields. The review ran 118/118 tests, rebuilt the static launcher byte-identically, and made zero protected reads, model requests, sampled outputs, or GPU calls.
  • Repaired all seven findings prospectively with exact support inventories, a scoped adapter around the immutable calibration verifier, JSON-domain durable comparisons, exact typed engine/prompt/seed/token/terminal/cost schemas, omission of unrequested likelihood diagnostics, the registered resource- exhaustion receipt, and self-contained resource matching. New hostile tests cover the real cross-module signature, durable round trips, Boolean aliases, extra keys, logprob injection, row-order-independent semantic IDs, and pool exhaustion. The model-free suite passes 124/124; mechanics calls and all protected reads remain zero pending a fresh exact-commit rereview.
  • The round-two exact-SHA review of pushed-green commit 3d2f0512 confirmed those seven repairs but returned HOLD_IMPLEMENTATION after 10/10 hostile durable-preflight mutations replayed successfully and Boolean schema values aliased integer version 1 at visible authorization and generic transaction boundaries. It ran 124/124 tests and made zero protected reads, model requests, sampled-output reads, or GPU calls.
  • Replaced partial preflight replay checks with a complete exact-typed expected value derived from every live runner/runtime field plus authenticated recorded CI, and made visible reanalysis exact-typed. The shared transaction module is calibration-locked, so an additive mechanics-only transaction layer now exact-authenticates STARTED, generated bundle, GENERATED, COMPLETE, chain, and registration receipts without changing the calibration anchor. Ten preflight mutations and durable Boolean/integer aliases are explicit regression tests. The model-free suite passes 130/130; mechanics calls and all protected reads remain zero pending a third exact-commit rereview.
  • The round-three review of pushed-green commit 9b527cbf confirmed the round-two closures but returned HOLD_IMPLEMENTATION: actual tuple-valued sampling would deadlock a fresh transaction after its first call, hidden scoring reread the visible receipt after authorization, and the additive primitive shallow-checked a predecessor before a fresh successor call. It ran 130/130 tests and made zero protected reads, model requests, sampled- output reads, or GPU calls.
  • JSON-normalized the complete fresh bundle before validation/write, carried the exact authorized visible object directly into hidden scoring with no second path read, and authenticated/rechecked the complete predecessor chain inside the transaction primitive. Regression tests use the actual frozen mechanics sampling plan, inspect hidden-analysis dataflow, and corrupt a predecessor before a fresh successor. The model-free suite passes 134/134; mechanics calls and protected reads remain zero pending a fourth exact-SHA review.
  • Round-four probes confirmed the production tuple path, no-reread hidden path, and eight static predecessor-corruption cases, but exposed concurrent mutation after the pre-call predecessor check and a double resolution of symbolic HEAD during hidden authorization. These were model-free synthetic callbacks; mechanics model/GPU calls and protected reads remain zero.
  • Added predecessor rechecks immediately after generation and before every successful promotion/recovery return, and resolved one commit once for both the visible blob comparison and authorization receipt. Generation-time and COMPLETE-publication mutation regressions fail closed. The full model-free suite passes 136/136 pending a fifth exact-SHA review.
  • Round five passed all prior regression families but returned HOLD_IMPLEMENTATION because the recheck hashed only predecessor COMPLETE; concurrent STARTED, bundle, or GENERATED changes were deferred to later full- chain rejection. The review ran 136/136 tests with zero model/GPU calls and empty protected-read inventories.
  • Replaced the terminal-file recheck with full exact prefix reauthentication and comparison against the original authenticated receipt. Added generation, publication, and recovery regressions for each non-terminal predecessor artifact. The suite passes 139/139 pending a sixth exact-SHA review.
  • Round six returned PASS_IMPLEMENTATION for exact pushed-green commit df096d330f09847ce844af6255b349b4f707f464: 139/139 tests passed, both launchers rebuilt byte-identically, every full-prefix boundary family passed, and the production tuple, hidden binding, typed gate, resource receipt, immutable calibration, and routing checks all passed. Model/GPU calls and all protected-read inventories remained zero. The canonical hash-bound review receipt is still a separate release gate before the mechanics lock.
  • After the round-six report and receipt were committed, pushed, and green, the lock-only launcher failed closed before writing a lock because the recomputed calibration decision retained 32 integer dictionary keys where canonical JSON requires string object keys; there were zero tuple/list differences. No mechanics payload, model, or GPU call occurred.
  • JSON-normalized the recomputed calibration decision before recursive exact- typed comparison, retaining Boolean/integer rejection. The actual frozen decision now reauthenticates model-free. This reviewed-code change invalidates the earlier release for lock purposes; a fresh exact-SHA rereview and updated canonical receipt are required before another lock attempt. The suite passes 140/140.
  • Round seven confirmed the real decision path, 20/20 typed aliases, four semantic mutations, and a complete read-only prospective lock build, then returned HOLD_IMPLEMENTATION: ordinary equality in durable sampling-plan validation accepted 30/30 integer/Boolean aliases across the five real arms.
  • Replaced sampling-plan equality with recursive exact JSON equality and added bidirectional real-plan regressions for all 30 aliases. Mechanics data, model/GPU calls, and protected reads remain zero pending another exact-SHA review and replacement receipt.
  • Round eight returned PASS_IMPLEMENTATION for exact pushed-green commit 3e7b650a90ff1d65fe371552354895756efcf728: 140/140 tests, all 58 real-plan type/schema mutations, five semantic seed mutations, the actual 32-key calibration normalization, read-only lock build/validation, and every prior regression family passed. Model/GPU calls and protected reads remained zero. The replacement canonical receipt is still a separate release gate.
  • After the replacement receipt was committed, pushed, and green, lock-only publication failed closed while hashing test_mechanics_bootstrap.py: the path audit omitted all seven reviewed test files from the 29-file critical inventory. No lock, mechanics payload read, model request, or GPU call occurred.
  • Added a separate exact seven-file pre-import critical-test allowlist and a bootstrap proof that it equals critical-minus-runtime and is consumed by the path audit. Runtime and data-access inventories remain unchanged. A fresh exact-SHA review and replacement receipt are required before retrying lock; the model-free suite passes 141/141.
  • Round nine returned PASS_IMPLEMENTATION for exact pushed-green commit c0075a019fd0f202c3b0e6cf0be5528e08c61649: 141/141 tests passed, the active audit hashed all 29 critical files while denying two neighbors, and the exact seven-file critical-test inventory left the 22/11 runtime/support inventories unchanged. All prior regression families passed; mechanics payload reads, model/GPU calls, and protected-read inventories remained zero. The replacement canonical receipt is the final release gate before lock retry.
  • Published the replacement receipt at shared-main commit 608d8e13551ee49187cadffdea79733340226c2b; both exact-commit workflows passed. The first lock retry after rebasing onto newer shared HEAD 7cebcf16981e2945aa0d71a6d78700fe82e2bef5 failed closed while that HEAD's validation workflow was still running. It wrote no lock and made no mechanics/model/GPU call.
  • After both workflows for 7cebcf16981e2945aa0d71a6d78700fe82e2bef5 passed, lock-only publication succeeded. implementation_lock.json is canonical with SHA-256 d42e2db4b589e470f42d963b19e01a8b880fa7858a40b10966150c775c3d925b, binds 29 critical and 22 runtime files, and records zero pre-lock generation requests, sampled outputs, and protected reads. Live mechanics remains held until this lock is committed, pushed, and green.
  • Committed and pushed the lock at 77c50a75fe82ca93d993f19a1897a6b3048c5824; both exact-commit workflows passed before live mechanics began.
  • Live mechanics completed all five durable transactions and preserved 4,056 outputs. Transport passed 24/24 exact echo and parse with zero caps, split 12/12 across both arities. Visible analysis then raised authentication requires later invocation to be absent: direct: replay of the stored transport decision incorrectly reused the initial authorization path after later invocations existed. No visible selection, resource decision, hidden score, hidden read, or benchmark read occurred.
  • Recorded terminal TERMINAL_INSTRUMENT_FAILURE in runs/mechanics/failure.json. The capability question is unadjudicated. Do not repair or rerun this result-bearing experiment; use fresh task/record identities and seeds in a successor whose reviewed transaction API separates initial transport authorization from post-chain replay authentication.

Scaffold

Created as a new experiment scaffold.

Data files 2

Result tables and metrics copied from the experiment folder — preview inline or open the raw file.

Reproduce

Smoke test

python3 -B experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/run.py --smoke

Full run

experiments/qwen35_4b_tokenizer_eos_answer_commit_factorial/scripts/calibration_launcher --stage run

Run steps are documented inside the experiment folder (README and scripts).

Browse the experiment folder on GitHub ↗