Research log Small Model Experimentation
GitHub

Count-Don't-Walk Enumeration

Finished2026-07-16Agentic Breadth InstallationGitHub ↗
Mechanism answer: menders moves for the first time (0.1 vs all controls 0.0) — but not by the taught mechanism

The one idea you need

The previous experiment taught the model a searcher's discipline for fix-the-broken-procedure puzzles: try every legal one-step change in a fixed, documented order, skipping the ones already tried. The discipline genuinely took — untrained models score exactly zero at it — but the model expressed it by narrating the whole march through the list, and on deep puzzles that narration ran out of room: 20 of its 21 unreadable exam answers were cut off mid-march while marching CORRECTLY. The insight this successor acts on: if the tried list already holds k entries in the fixed order, the right next candidate is simply entry k+1 — you can COUNT instead of WALK. Every puzzle now prints how many candidates each step offers (a small table of number ranges), and the model is taught a five-line arithmetic recipe: count the tried entries, add one, find which step's range holds that number, convert to the list slot, answer. The cost of the answer no longer grows with how deep the search is.

The question

If the model is taught to compute the next candidate's position with arithmetic (five short lines, always the same shape) instead of narrating a walk through the list, does the already-installed search discipline finally express itself within the answer budget — and clear the 50% bar the previous run missed?

What we found

Two results that point in opposite directions, both real. The taught compact phrasing did NOT take: at the local exam the trained model still thought all the way to its 1,024-token ceiling on most puzzles (25 of 40 answers cut off, barely fewer than its untreated siblings), and it produced the correct next candidate only 7 times in 40 — nowhere near the pre-set 50% bar. The model keeps its own long-winded style no matter how short the taught examples are. AND YET: at the sealed benchmark, the trained model solved a fix-the-procedure episode that its three comparison models — the raw base, its parent, and a sibling trained on the same replay data without the new lessons — all scored exactly zero on. Under the rules written down before the run, a candidate scoring above zero with every control at zero is the pre-declared positive outcome, and this is the first time it has happened for this family in the whole program. The honest caveats: it is one episode on one exam seed, a control model once hit the same score on a different seed in the previous experiment, and whatever the lessons installed that did the converting, it visibly is not the five-line arithmetic they taught.

Why it matters

The predecessor's autopsy said the failure was expression cost, not the skill: the model searched correctly and ran out of room saying so. If a constant-cost expression clears the pre-set 50% bar, it confirms the skill was there all along and only the phrasing was broken — and it gives the one benchmark family still stuck at zero its best remaining shot. If it still falls short, the compact-expression explanation dies and the enumeration route is genuinely closed, not just badly phrased.

Menders (sealed)0.1 vs 0 / 0 / 0first candidate-over-all-controls move; n=1 episode
Aggregate (sealed)0.3312tops replay 0.3298, parent 0.2950, base 0.0753
Compact phrasingrefutedmedian think at the 1,024 cap; 25/40 truncations
Fidelity bar7/40 = 0.175pre-set bar was 0.50; replay control drew 5/40
On this page
  1. Results at a glance
  2. Overview
  3. Report
    1. Summary
    2. Research Program Fit
    3. Method
    4. Results
    5. Controls
    6. Oracle Versus Deployable Evidence
    7. Next Experiments
    8. Artifact Manifest
  4. Experiment log
  5. Data files
  6. Reproduce
  7. Related

Results at a glance 1

Sealed event 78163: menders moves for the first time with all controls at zero MECHANISM_ANSWER per the preregistered precedence rule: count_walk drew menders 0.1 with base, parent, AND the replay-trained control all at exactly 0.0 — the first candidate-vs-all-controls menders movement in program history — and topped the aggregate outright. Twist: the cell's intended mechanism was refuted at its own local gate (think tokens still at the 1,024 cap, median 1024, 25/40 truncations; canonical-next fidelity 7/40 = 0.175 << the frozen 0.50 precondition; the replay control drew 5/40 off the rendered-ranges prompt delta). Teaching verified ≤105-token constant-cost think targets did NOT install short thinking at 160 rows — verbose enumeration is the model's own preference — yet the dose converted anyway (candidate-specific vs replay: menders +0.1, mirage +0.4, stockade +0.21, all single-seed). Honest scope: one episode in one sealed seed, and the reference cell's untreated replay control once drew 0.1 on a different seed; the funded successor is an eval-only multi-seed confirmation on the same committed composites before any claim.

score · arm →

00.10.20.30.4basebase00.0753zero_root_parentzero_root_parent00.295replay_ctl7replay_ctl700.33count_walk (trained)count_walk (trained)0.10.331
Data table
armmendersaggregate
base00.0753
zero_root_parent00.295
replay_ctl700.33
count_walk (trained)0.10.331

Numbers from experiments/qwen35_4b_count_dont_walk_enumeration/runs/benchmark/medium_tb1024_seed78163_pilot/summary.json

In the author’s words from the Report · “Summary”

MECHANISM_ANSWER at the sealed event — with the cell's own intended mechanism refuted on the way. count_walk drew menders 0.1 with ALL THREE controls (base, zero-root parent, replay) at exactly 0.0 at sealed seed 78163, firing the preregistered positive-precedence branch: the first candidate-vs-all-controls menders movement in program history. The candidate also topped the aggregate outright (0.3312 vs replay 0.3298, parent 0.2950, base 0.0753) with a 9-win/1-tie/0-loss goal gate against base. … Read the full result →

Overview

Scope at closure: n=1 episode on ONE sealed seed, the converting mechanism is NOT the taught five-line computation, NO claim was entered in the claim ledger, and the funded eval-only multi-seed confirmation cell was never run.

Research Program

  • Program: agentic_breadth_installation
  • Program question: can DESIGNED synthetic curricula install universal, transferable agentic skills into the one 4B — provably, on fully documented lineage?
  • Prior anchors: lifecycle 22 (qwen35_4b_zero_root_lineage_rebuild — the zero-root composite, tree 414f5829…), lifecycle 25 (qwen35_4b_clean_gym_mix_dose — ONE KIND PER DOSE AT FULL CONCENTRATION), and lifecycle 26 (qwen35_4b_enumerative_repair_protocol — the reference cell: the enumeration discipline INSTALLED, 9/40 canonical-next vs both controls at 0/40, but failed on its own terms at 22.5% fidelity).

Question

Lifecycle 27 — the evidence-backed successor to the enumerative-repair cell, changing ONLY the expression pedagogy. The reference cell's committed truncation forensics (experiments/qwen35_4b_enumerative_repair_protocol/analysis/truncation_forensics.md) showed its 22.5% fidelity UNDERCOUNTS the installed discipline: 20 of 21 unparseable gate rows were 1,024-token cap truncations caught mid-CORRECT walk — the model faithfully executed the frozen canonical walk and ran out of budget, a token cost that grows with the tried-list depth k. Does teaching the SAME discipline with a compact, constant-cost expression — COUNT, DON'T WALK — lift the canonical-next fidelity past the frozen 0.50 precondition and (conditionally) convert to the menders family?

Hypothesis

The discipline is already installable (proven by the reference cell's 9/40 vs 0/40). The bottleneck is expression cost. The tried list has k entries in canonical order, so the target is entry k+1 of the frozen order; with the per-step candidate ranges RENDERED IN THE PROMPT, locating entry k+1 is pure index arithmetic (range lookup + slot offset), constant in k. Training a fixed-shape five-line computation under a frozen think length budget (<= 120 real tokens; measured max 105) should express the installed discipline inside any bounded budget where the walker truncated.

Setup

  • Model: Qwen/Qwen3.5-4B (revision 851bf6e8…), always.
  • Parent and adapter base: the zero-root composite (large_artifacts/qwen35_4b_zero_root_lineage_rebuild/merged/zero_root_hygiene_explore, tree 414f5829…, weights 6e9aad25…), authenticated against lifecycle 22's committed merge receipt (e906caea…; byte-identical provenance copy in data/lineage/provenance/merge.json).

  • Treatment: data/sft_count_walk.jsonl — 160 rows, ONE KIND u_count_walk, at construction seed 77,191 (sha 21e6f5cb…), 20 rows per formalism across all eight machine formalisms REUSED from the menders dose-scale cell via a byte-identical machinery copy (scripts/gen_feedloop_curriculum.py — imported, never forked). The task shape is byte-equivalent to the reference cell's (partial enumeration episode; frozen canonical-order statement, byte-identical rule text; verified tried prefix, k over 0/1/3/6/10; unique both-trials fix; exact-match STEP <k>: <corrected step>). The ONLY designed deltas:

    1. THINK TARGETS teach COUNT-DON'T-WALK: a fixed-shape compact computation,

    identical five-line structure in every row — count the tried entries → k; the target is change number k+1; locate k+1 in the rendered ranges (explicit offset subtraction); resolve the offset to the action-list slot (skipping the written action); emit STEP <n>: <action>. Constant token cost in k, under a frozen THINK LENGTH BUDGET (five-line shape + caps in the generator, per row; the REAL tokenizer bound <= 120 tokens fail-closed in measure_source_tokens.py; measured max 105 / mean 95.8 real tokens). 2. THE ORDER STATEMENT gains the rendered per-step candidate counts ("step 1 offers 17 changes (numbers 1-17); step 2 offers 17 (numbers 18-34); …"), byte-identical rule text otherwise; per-step counts computed generically and the rendered ranges verified against the generator's own exhaustive enumeration exactly. 3. NEW GATE READING (recorded, non-gating): expression_cost — per-arm think-token-length distribution on the axis rows + truncation count, the expression-cost reading this lineage owes.

  • Surfaces: ALL inherited (no fresh-surface claim); freshness is at the ROW level — zero canonical-user-message overlap against 83 pinned predecessor corpora, streams, and gates, INCLUDING the reference cell's corpus/streams/gates (same formalisms, same episode shape — row-level freshness is the bar).
  • Arms: replay_ctl7 (control, trains FIRST) then count_walk (candidate); fresh rank-32/alpha-64 adapters, NO warm start, training seed 85 (84 is taken — next-free substitution recorded), standard recipe (1 epoch over 1,520 rows, 190 optimizer updates, LR 1e-5, batch 1×8, max length 4,096, w_think/w_close 0.2).
  • Exposure: exact zero-delta three-axis MILP (forward / nonzero-target / absolute loss mass ×5) at the frozen geometry — 1,280-row shared stratified core + 240-row variable block (control: 240 replay; candidate: 160 treatment + 80 fillers), namespace seed 55,171. Matched exactly: 1,438,010 forward / 564,379 nonzero / 621,239 mass×5 per arm; zero skips; 1,280 aligned core rows.
  • Local gate (three arms: parent + both trained): 40-row axis holdout at seed 88,056 (all u_count_walk, 5 per formalism, one per k-value; same invariants as the treatment) + three 104-row retention screens at 88,057/88,058/88,059 under pooled_k3. All seeds verified grep-fresh in seed contexts (known-taken: 88,043/88,047/88,049 and everything <= 88,055). Promotion: axis total strictly > parent AND > replay_ctl7 (ties fail) — single kind, so NO per-kind split exists; pooled retention bands on screen sums (correct −15, caps +9, parsed −9) vs both controls.
  • Preregistered NON-GATING mechanism readings: (a) episode_success_simulation — holdout from-scratch MEAN 27.1 turns (median 20.5, max 78; 80.0% of episodes need MORE than 10 turns); treatment corpus mean 32.6 (median 22, max 125; 86.9% > 10) — against a family episode budget publicly known only as "bounded". (b) enumeration_fidelity — per axis row: LEGAL, UNTRIED, CANONICAL-NEXT booleans; it feeds the frozen zero-draw scoping below. (c) expression_cost (NEW) — per-arm think-token-length distribution + truncation count on the axis rows.
  • Conditional benchmark (only on promotion): ONE sealed medium tb1024 event at fresh seed 78,163, four arms in frozen order — base (26d8ee48…/b654e033…), zero_root_parent (414f5829…), replay_ctl7, count_walk. Trained-arm pins are six fail-closed TODO-PIN slots in scripts/run_benchmark.py, frozen by check_design's NORMALIZED-HASH pin. Pilot gate: candidate aggregate strictly > base AND > replay_ctl7 AND > zero_root_parent. Recorded either way: the goal gate, the per-family table, and THE MENDERS READING. FROZEN ORDERED CONSEQUENCES, positive first, no third state for the zero draw: (1) ANY candidate menders > 0 where the controls sit at 0 is the mechanism answer; (2) a menders 0 WITH the fidelity precondition met (promoted locally AND holdout canonical-next rate F >= 0.50 AND F strictly above both controls' rates) is TURN_BUDGET_SCOPED — the protocol-install mechanism is NOT refuted; what closes is the pure-enumeration route at the family's actual budget; (3) a menders 0 WITHOUT that precondition fails on its own terms.
  • Standalone: data/lineage/ carries the complete clean-chain package — the six zero-root stage datasets, lifecycle 22's stage + merge receipts as provenance documents, the trainer/merger copies, and a clean-chain manifest recording this cell's dose as STAGE 7 (count_walk, seed 85). NO blend root exists anywhere in this cell (fail-closed).
  • Hidden-label boundary: gate answers and per-row audits live only in data/local_tasks_seed*.jsonl; the model-facing local_input_seed*.jsonl files carry id/messages/meta only. The benchmark suite directory is never read; only the trusted aggregate gateway runs.

Run

Smoke (no GPU, no writes):

PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -B experiments/qwen35_4b_count_dont_walk_enumeration/scripts/run.py --smoke

Full (one stage per pushed checkpoint, each behind its review verdict):

.venv/bin/python -B experiments/qwen35_4b_count_dont_walk_enumeration/scripts/run.py --stage train-control
# then: train-candidate, merge-arms, local, benchmark

Standalone lineage verification (no GPU) / full clean-chain rebuild (GPU):

.venv/bin/python -B experiments/qwen35_4b_count_dont_walk_enumeration/scripts/rebuild_clean_chain.py --verify-inputs

Results

Pending: the model-free construction is frozen; GPU stages run behind their review verdicts.

Interpretation

Pending results. Separate deployable evidence from oracle/hidden evaluation.

Knowledgebase Update

  • Program evidence updated: pending results.
  • Program backlog updated: pending results.
  • Claim ledger updated: pending results.

Artifacts

  • src/ — frozen vLLM runner (byte-identical to the reference cell's).
  • scripts/ — staged harness, the count-walk generator + the byte-copied menders machinery + canonical retention generator, corpus builder with audits, exposure pipeline (with the real-tokenizer think-budget certification), gate with the expression-cost reading, benchmark runner with the menders reading, clean-chain rebuild script, vendored trainer/merger copies.
  • configs/ — frozen identity.
  • data/ — treatment corpus + manifest, replay copy, exposure streams + receipts, gate files, design receipts, clean-chain lineage package (data/lineage/).
  • runs/ — stage receipts (written by the staged GPU runs).
  • reports/artifact_manifest.yaml

Report

Rendered from reports/report.md

Summary

MECHANISM_ANSWER at the sealed event — with the cell's own intended mechanism refuted on the way. count_walk drew menders 0.1 with ALL THREE controls (base, zero-root parent, replay) at exactly 0.0 at sealed seed 78163, firing the preregistered positive-precedence branch: the first candidate-vs-all-controls menders movement in program history. The candidate also topped the aggregate outright (0.3312 vs replay 0.3298, parent 0.2950, base 0.0753) with a 9-win/1-tie/0-loss goal gate against base. But the treatment's central hypothesis — that verified ≤105-token constant-cost think targets would install SHORT thinking — was refuted at the local gate: the trained candidate still thinks to the 1,024-token cap (median 1,024; 25/40 truncations vs replay 27, parent 32), and canonical-next fidelity reached only 7/40 = 0.175 (replay drew 5/40 by itself — the rendered-ranges prompt delta lifted every arm). Honest scope: the menders reading is one episode within one sealed seed, and the reference cell's untreated replay control once drew 0.1 on a different seed; the preregistered rule makes this the mechanism answer for THIS event, and the program's confirmation doctrine requires a fresh multi-seed confirmation cell before any capability claim.

Research Program Fit

Menders was the last benchmark family never moved by any documented intervention; it blocked every confirmation sweep miss. The predecessor (enumerative repair) produced the starkest install contrast on record (9/40 vs 0/40) with zero conversion; its truncation forensics funded this cell.

Method

Byte-equivalent clone of the reference cell except three designed deltas: (1) fixed-shape five-line compact think targets (count → k+1 → range lookup → slot → answer), byte-checked per row against a pure re-derivation, all ≤105 real-tokenizer tokens; (2) the order statement additionally renders per-step candidate counts/ranges, verified against the exhaustive enumeration; (3) a non-gating expression_cost gate reading. Arms replay_ctl7 (trained first) and count_walk, fresh rank-32 adapters from the zero-root composite (seed 85), exact zero-delta MILP (1,438,010 forward / 564,379 nonzero / 621,239 mass×5 per arm), merged composites 044a4599…/d5fdc55c…, three-lens pre-GPU adversarial review (no MAJOR), full fail-closed pin ladder. Seeds: gate 88056, screens 88057-88059, sealed benchmark 78163.

Results

  • Local gate: count_walk PROMOTED — axis totals strictly beat BOTH controls; pooled_k3 retention within every band (deltas vs parent pooled mean +1.0, replay −0.33; screen SD 5.09).
  • Fidelity cascade (candidate/replay/parent, of 40): parseable 15/13/8; legal 13/11/8; untried 10/8/7; canonical-next 7/5/1. F = 0.175 << the frozen 0.50 precondition.
  • Expression cost (the reading this cell owed): think tokens median 1,024 (the cap) in ALL arms; truncations 25/27/32. Short thinking did NOT install at this dose.
  • Sealed event 78163 (medium, tb1024): aggregates 0.0753 base / 0.2950 parent / 0.3298 replay / 0.3312 count_walk. Menders 0.0 / 0.0 / 0.0 / 0.1 → frozen_interpretation = MECHANISM_ANSWER (the positive branch takes precedence regardless of the failed fidelity precondition, which scoped only the interpretation of a zero). Goal gate vs base: 9 strict wins, tie on rites, no losses.

Controls

Three controls at the event (raw base, zero-root parent, replay-trained sibling) — all at menders 0.0. The replay control's existence is what makes the reading a treatment effect rather than a generic-training effect; the parent tie on aggregate (replay 0.3298 vs candidate 0.3312, delta +0.0013) shows the aggregate lift is mostly replay compounding, while the menders episode is candidate-specific.

Oracle Versus Deployable Evidence

The oracle-side story inverted: the intended mechanism (compact expression) is refuted — the model keeps its verbose enumeration preference; whatever converted at menders is NOT visibly the taught five-line computation. Candidate deployable readings: menders 0.1, mirage 0.8 (vs replay 0.4), stockade 0.334 (vs replay 0.122) — all single-seed, unconfirmed.

Next Experiments

A fresh eval-only multi-seed confirmation cell on the SAME two committed composites (no retraining): does count_walk's menders > 0 with all controls at 0 replicate across fresh sealed seeds? Only a replicated contrast supports any claim; a non-replication closes the reading as seed noise, and the expression-cost law (short thinking does not install at 160 rows) stands either way.

Artifact Manifest

See artifact_manifest.yaml (corpus 21e6f5cb70…, design receipt d835fc5c38…, merged trees 044a4599…/d5fdc55c…, promotion receipt and sealed summary under runs/).

Experiment log 4

Show the running log (4 entries, 2026-07-16)

Scaffold

Created as a new experiment scaffold.

2026-07-16 — model-free construction frozen (lifecycle 27)

  • Cloned the entire enumerative-repair reference cell (lifecycle 26): scripts, tests, lineage package, frozen vLLM runner, the two-direction menders consequences with the 0.50 fidelity precondition, the six-slot normalized-pin hardened benchmark runner. Machine simulators, legality bounding, canonical order, K_CYCLE, and uniqueness invariants stay byte-equivalent (gen_feedloop_curriculum.py byte-identical to the menders source; train_think.py/merge_adapter.py/lineage trainers byte-identical to their proven sources).
  • The ONE designed delta (evidence: experiments/qwen35_4b_enumerative_repair_protocol/analysis/truncation_forensics.md): the expression pedagogy. Think targets are a fixed-shape five-line compact computation (count → k+1 → rendered-range lookup with explicit offset subtraction → action-list slot skipping the written action → emit), constant token cost in k, under a frozen think length budget (five-line shape + caps per row in the generator; REAL tokenizer bound <= 120 tokens fail-closed in measure_source_tokens.py — measured max 105, mean 95.8 over the frozen corpus). The order statement gains the rendered per-step candidate counts, verified against the exhaustive enumeration exactly. New non-gating gate reading: expression_cost (per-arm think-token-length distribution + truncation count on the axis rows).
  • Corpus frozen: data/sft_count_walk.jsonl sha 21e6f5cb…, 160 rows single-kind u_count_walk at construction seed 77,191; manifest 4343251285…; zero row overlap against 83 pinned predecessor sources including the reference cell's corpus/streams/gates.
  • Exposure frozen: exact zero-delta three-axis MILP at namespace seed 55,171 (HiGHS optimal, 0.4 s) — 1,438,010 forward / 564,379 nonzero / 621,239 mass×5 per arm, 1,280 aligned core rows, zero skips; independent validation receipt baaa454d….
  • Gate frozen at seeds 88,056 + 88,057/88,058/88,059 (sealed aggregate 78,163); gen_local_gate.py --check and check_design.py --check byte-stable across repeated runs; rebuild_clean_chain.py --verify-inputs green (stage 7 = count_walk, training seed 85).
  • Seeds verified grep-fresh in seed contexts; ONE next-free substitution recorded: training seed 84 is taken (task seed of qwen35_4b_hypothesize_verify_wall), so 85.
  • Note: during design freeze, the train_think_stage12.py sha pin in check_design.py was found altered by one hex character (…8805910c…) relative to the byte-identical vendored copy's true sha256 (…8805510c…, matching the lineage manifest, the provenance receipts, and lifecycle 22's committed original); check_design correctly failed closed, and the pin was corrected to the verified file hash.
  • Tests: 178 green (unittest discover), including the new think-budget/constant-shape suite, the rendered-range-vs-enumeration suite, and the expression-cost gate-reading suite. run.py --smoke green end to end. Boundary drills refuse (train/merge/local/benchmark stages abort on a dirty tree and on unfilled TODO-PINs).

2026-07-16 — Pre-GPU adversarial review (three lenses): NO MAJOR; amendments recorded

Three independent review lenses ran against the frozen design at ddde0e37 before any GPU stage: (1) preregistration/consequence-rule soundness, (2) treatment-content correctness — an independent per-row re-derivation of all 200 model-facing rows from prompt text alone, 0 errors, byte-exact — and contamination (0 banned-vocab hits, 0 overlap across 83 pinned sources, no renamed-structure leak), (3) pins/reproducibility — every pin live-verified non-vacuous, the normalized-hash mutation probes all held (slot canonicalization exactly 6 slots; injection/relocation/duplication all fail closed), seeds grep-fresh, stage ordering and merge refusals confirmed. Verdict: no MAJOR finding on any lens. Actions taken from the minors, pre-GPU:

  • REVIEW AMENDMENT (preregistration): added frozen consequence 4 — candidate menders > 0 with ANY control > 0 is DESCRIPTIVE ONLY, no mechanism claim; matches the code's existing frozen_interpretation = None branch; interpretation-only, no code change, no pin churn.
  • ERRATA (design receipt, seed 85): the receipt sentence "zero seed-context hits anywhere in the repo" for training seed 85 is overstated by its own evidentiary standard: "seed": 85 appears as a per-row data field in experiments/qwen35_4b_meta_induction/data/train_shift.jsonl (one row), the same class of hit the receipt cites as secondary takenness evidence for 84. Not a collision (per-row generation field in an unrelated cell vs a torch training seed; program precedent excludes per-row fields — the reference cell recorded-and-excluded the same class for its seed 83). The receipt is NOT rewritten; this errata is the record. Seed 85 stands.
  • ACCEPTED INHERITED LIMITATIONS (logged, not fixed here; carried byte-identically from the reference cell): (a) rebuild_clean_chain verify_inputs unconditionally requires the sibling zero-root cell's receipts to exist even though all information is in-cell — a scoping fix belongs in a future cell's template; (b) eval_local_vllm.py is deferred-pinned only (drift before the local stage is caught by clean-pushed-main + post-hoc sha, not a design-time pin); (c) authenticate_local_promotion verifies the promotion receipt by sha/pointer without recomputing canonical_next_counts — a hand-edited receipt could flip verdict 3 to 2 (both non-success verdicts; the raw local receipt is committed and sha-pinned so tampering is deterministically detectable post-hoc); (d) benchmark ledger: a crash between summary write and closed-append wedges (recovery: delete summary.json, --resume regenerates byte-identically without new gateway calls); no file lock against concurrent invocations (single-operator environment; double-consumption detectable as 2 ledger rows).
  • READOUT PRIORS (from the content lens, to carry into interpretation): the locate arithmetic is exercised only at canonical indices <= 11 and never targets steps 4-5 (train/gate symmetric; family episodes needing deeper indices are extrapolation); 98.75% of training rows teach proposing a candidate that does NOT repair (the intended propose-and-let-the-trial-judge discipline, but a strong prior against "answer = the fix").

2026-07-16 — Sealed event 78163: MECHANISM_ANSWER; cell closed

  • Local gate (88056-88059): count_walk PROMOTED (strict totals over both controls; retention in-band) but the cell's central bet was refuted at the same gate: think tokens still at the 1,024 cap (median) with 25/40 truncations, fidelity 7/40 = 0.175 << 0.50, and the replay control drew 5/40 by itself off the rendered-ranges prompt delta. Short thinking does not install at a 160-row dose; verbose enumeration is the model's own preference, not a walk-pedagogy artifact.
  • Sealed medium event at 78163: menders 0.1 with ALL controls at 0.0 -> frozen_interpretation = MECHANISM_ANSWER (positive precedence branch; the failed fidelity precondition scoped only zero readings). First candidate-vs-all-controls menders movement in program history. Aggregate: candidate 0.3312 tops all arms (replay 0.3298, parent 0.2950, base 0.0753); goal gate vs base 9 wins / 1 tie / 0 losses.
  • Honest scope, recorded at closure: n=1 episode in one sealed seed; the reference cell's untreated replay control drew menders 0.1 on a different seed; the mechanism that converted is NOT the taught compact expression (refuted above) — the dose installed something else (candidate-specific: menders +0.1, mirage +0.4, stockade +0.21 vs replay, all single-seed). Per the program's confirmation doctrine the funded successor is an EVAL-ONLY multi-seed confirmation on the same committed composites; no capability claim until it replicates.

Data files 1

Result tables and metrics copied from the experiment folder — preview inline or open the raw file.

Reproduce

Smoke test

PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -B experiments/qwen35_4b_count_dont_walk_enumeration/scripts/run.py --smoke

Full run

.venv/bin/python -B experiments/qwen35_4b_count_dont_walk_enumeration/scripts/run.py --stage train-control  # then train-candidate, merge-arms, local, benchmark

Run steps are documented inside the experiment folder (README and scripts).

Browse the experiment folder on GitHub ↗