Feedback-Loop + State-Chain Install
The one idea you need
The last measurement left the program's best clean model beating the untouched base on eight of ten benchmark families with zero losses — only two zero-zero ties remain. The public one-line descriptions of those two families share a trait every past training set lacked: they are multi-turn episodes, where the model must act, observe feedback, and revise. Every lesson this program ever trained was a single question with a single answer. This experiment teaches the missing loop itself — on six invented toy machines that look nothing like the benchmark — and then takes one shot at the full ten-family sweep.
The question
Does teaching the act-observe-revise loop and hidden-state tracking on invented machines flip the last two zero-score families without breaking the eight wins?
What we found
The dose split down the middle. The hidden-state-tracking half installed cleanly — on brand-new test instances the trained model tracked procedures better than both its parent and a matched control. The feedback-repair half failed completely: after eighty lessons on those exact toy machines, the model got zero of twenty fresh ones right, worse than models that never saw a lesson. That is the third teaching method to fail at the repair-style skill. Forgetting was judged for the first time under the new three-quiz averaging rule, which worked exactly as calibrated — and caught the trained model falling just outside the allowed margin against the control. No benchmark shot was taken; the sealed seed stays sealed.
Why it matters
This is the closest the program has been to its goal — one model, every family up. The bet is honest (~15-30%): the mechanism is new (episode protocol, not repair content), the instruments are freshly calibrated, and either outcome sharpens the map.
On this page
Results at a glance 1
How to read
Grouped bars: each model's score on twenty fresh feedback-repair tasks and twenty fresh state-tracking tasks.
Takeaway → State-tracking rose above both controls; feedback-repair fell to zero despite eighty training lessons on those same machines.
Data table
| gate instrument | hygiene_explore_parent | replay_ctl | feedloop_state |
|---|---|---|---|
| u_feedloop (20) | 1 | 1 | 0 |
| u_statechain (20) | 7 | 10 | 11 |
| retention pooled mean (104) | 62.3 | 65 | 59.3 |
Numbers from experiments/qwen35_4b_feedback_loop_state_chain_install/runs/local/seed88026_promotion.json
Technical framing
Episode-holdout correct (of 20) and pooled retention, seed 88026 gate — NOT_PROMOTED, split install: u_statechain installed (11/20, strict over both controls) while u_feedloop failed completely (0/20 on fresh instances of its own training formalisms — the third failed pedagogy at the menders-shaped skill); axis total tied replay 11-11 (ties fail); pooled retention fell 5.67 below the replay control, 0.67 outside the calibrated +/-5 band, on the pooled_k3 protocol's first live use (event delta SD 4.08 vs calibration 4.27). Sealed seed 78151 never opened.
In the author’s words from the Overview · “Results”
Both arms trained clean (1,520 rows each, zero skips, fresh rank-32 adapters); the 12-run local gate executed with full boundary authentication (table on the experiment page). Promotion failed on three of the frozen bars: axis total tied replay (11–11, ties fail); u_feedloop lost to both controls; pooled retention fell 5.67 below the replay control (band ±5) while passing the parent band (−3.0). The event's own measured noise (delta SD 4.08 across the three screens) matched the calibration study's 4.27 — the pooled_k3 instrument performed exactly as designed on its first use. Seed 78,151 was never opened and is permanently sealed per the frozen contract.
Overview
The two-tie install: teach the episode protocol (using rerun feedback; tracking narrated hidden state) that the public metadata names for menders and rites — the only two families separating the hygiene_explore parent from the recorded all-families goal gate at medium.
Research Program
- Program:
agentic_breadth_installation. - Program question: can synthetic curricula install general capability that lifts the held-out aggregate without a negative family?
- Prior anchors: the medium measurement (parent at 8/10 strict wins, zero losses, ties only at menders/rites); the tier forensics (goal-gate venue is medium); the calibration cell (pooled_k3 retention protocol, first use here); the trace-repair kill rule (this dose's mechanism argument is the episode PROTOCOL, not repair content).
Question
Does a single-dose episode-protocol curriculum — 80 act→observe→revise feedback-loop rows plus 80 narrated hidden-state chain rows on six invented formalisms — install the two missing multi-turn skills without breaking the parent's eight strict wins?
Setup
- Parent and adapter base: the
hygiene_explorecomposite (tree 9eb653d7…), fresh rank-32/alpha-64 adapters, no warm start. - Corpus:
data/sft_feedloop_state.jsonl(e6d45ed4…), 160 rows, seed 77,130, generator-verified invariants (≥2 LEGAL candidates after evidence round 1, exactly 1 after round 2, extended-grammar exclusion audit per the pre-freeze review amendment; ≥3 hidden-state updates with stateless/last-step distractors verified wrong), banned-vocab + 56-token fresh-surface audits clean. - Exposure: exact zero-delta MILP vs
replay_ctl(1,393,242 forward tokens, 584,414 targets, 640,286 mass×5 per arm). - Local gate: axis holdout 88,026 (20+20) with strict per-kind bars; retention pooled over screens 88,027/88,028/88,030 under pooled_k3 bands.
- Conditional benchmark: medium, tb1024, sealed seed 78,151, four models, hardened runner; goal gate recorded either way.
Run
PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -B experiments/qwen35_4b_feedback_loop_state_chain_install/scripts/run.py --smoke
# staged: --stage train-control | train-candidate | merge-arms | local | benchmarkResults
Both arms trained clean (1,520 rows each, zero skips, fresh rank-32 adapters); the 12-run local gate executed with full boundary authentication:
| arm | axis total (40) | u_feedloop (20) | u_statechain (20) | retention pooled (104) | caps pooled | parsed pooled |
|---|---|---|---|---|---|---|
| hygiene_explore_parent | 8 | 1 | 7 | 62.33 | 10.67 | 93.33 |
| replay_ctl | 11 | 1 | 10 | 65.00 | 11.00 | 93.33 |
| feedloop_state | 11 | 0 | 11 | 59.33 | 13.00 | 90.67 |
- Promotion failed on three of the frozen bars: axis total tied replay (11–11, ties fail); u_feedloop lost to both controls; pooled retention fell 5.67 below the replay control (band ±5) while passing the parent band (−3.0).
- The event's own measured noise (delta SD 4.08 across the three screens) matched the calibration study's 4.27 — the pooled_k3 instrument performed exactly as designed on its first use.
- Seed 78,151 was never opened and is permanently sealed per the frozen contract.
Interpretation
- The split is the finding. The state-chain protocol lesson INSTALLED (+4 over parent, +1 over the strong replay control, on fresh instances) — narrated hidden-state tracking is teachable at a 80-row dose, the rites-relevant skill. The feedback-loop lesson did NOT install even in-domain: 80 training rows on the same four formalisms yielded 0/20 on fresh instances — worse than untrained controls. Repair-with-feedback is now the THIRD failed pedagogy at the menders-shaped skill (after asserted single-turn repair and demonstrated bounded search, both killed by rule), and this one fails at the instrument that its own training surface defines.
- Replay continuation strengthened again: the control gained +2.67 pooled retention over the parent and reached 10/20 on statechain untrained — replay remains the strongest single broad move, and half of the candidate's apparent statechain edge rides the replay core.
- The retention cost (−3.0 vs parent) sits inside the revised 1–4-point tax law; the failure against the replay band is the dose competing against replay's own gain, which the calibrated bands are designed to catch.
Knowledgebase Update
- Program evidence updated: the split-install verdict, the third repair-pedagogy failure, and the pooled_k3 instrument validation recorded.
- Program backlog updated: the statechain-only successor (drop the dead feedloop rows) is the natural next branch; menders needs a mechanism argument no small designed dose has survived.
- Claim ledger updated: no new claim.
Artifacts
data/: frozen corpus + manifests, exposure streams + receipt, four gate input pairs + design receipts.reports/preregistration.md: the full frozen contract.
Report
Rendered from reports/report.md
Summary
The two-tie install closed NOT_PROMOTED with a split mechanism reading. The state-chain half installed: 11/20 on fresh holdout instances, strictly above both the parent (7) and the strong replay control (10) — narrated hidden-state tracking is teachable at an 80-row dose. The feedback-loop half failed completely: 0/20 on fresh instances of the very formalisms it trained 80 rows on, below both untrained controls (1/20) — repair-with-feedback is the third failed pedagogy at the menders-shaped skill. Promotion failed on the axis-total tie against replay (11–11), the u_feedloop loss, and a pooled retention deficit of 5.67 against the replay control (0.67 outside the calibrated ±5 band; the parent band passed at −3.0). The pooled_k3 instrument's first live use measured delta SD 4.08 versus the calibration's 4.27 — the new protocol performed as designed. Sealed seed 78,151 was never opened and is permanently sealed.
Research Program Fit
The medium measurement's funded successor: the parent sits at 8/10 strict family wins with zero losses, and both blocking families are multi-turn episode families per public metadata while every prior curriculum in the line is single-turn — the dose targets the protocol gap.
Method
See the preregistration: frozen corpus with generator-verified uniqueness/state-depth invariants and an extended-grammar exclusion audit, exact three-axis MILP exposure, control-first fresh-adapter training, strict per-kind axis bars plus pooled_k3 retention bands, hardened conditional benchmark runner.
Results
runs/local/seed88026_promotion.json: axis totals 8 / 11 / 11 (parent / replay / candidate; tie fails), per-kind u_feedloop 1 / 1 / 0 and u_statechain 7 / 10 / 11 (statechain strict win, feedloop loss), pooled retention 62.33 / 65.00 / 59.33 with caps 10.67 / 11.00 / 13.00 and parsed 93.33 / 93.33 / 90.67; replay gained +2.67 retention over the parent, repeating the replay-compounding law.
Controls
Active exposure-matched replay control; parent baseline; overlap receipts across all thirteen predecessor gates and 21 pinned corpora/streams; banned-vocabulary plus 56-token fresh-surface audits; every stage fail-closed behind its preregistered review verdict.
Oracle Versus Deployable Evidence
Executable truth grades everything local; the benchmark stage consumes only gateway aggregates and public family scores; benchmarks/ contents are never read.
Next Stage
Closed. The natural successor is a statechain-only dose (drop the dead feedloop rows) from this same parent under the same calibrated gates; menders remains open with no surviving small-dose pedagogy.
Artifact Manifest
Frozen corpus, streams, receipts, and four gate input pairs in-repo; the parent and base composites external with committed pins; trained adapters and merges will live outside the repo with hashes pinned in receipts.
Experiment log 3
Show the running log (3 entries, 2026-07-15)
2026-07-15 — Model-free design freeze
- Opened as the medium measurement's funded successor: the recorded goal gate is two tie-flips wide (menders and rites, both multi-turn episode families per the public suite metadata), and every prior curriculum in the line is single-turn — the dose teaches the episode protocol (act→observe→revise feedback loops; narrated hidden-state chains) on six invented formalisms with a fresh 56-token surface audit.
- Corpus frozen (160 rows, generator-verified uniqueness/state-depth invariants); exact zero-delta MILP exposure vs the replay control from the hygiene_explore parent; fresh rank-32 adapters; axis holdout at 88,026 + THREE retention screens (88,027/88,028/88,030 — 88,029 collided and was substituted by the next-free rule) for the first use of the calibrated pooled_k3 protocol; conditional medium event at sealed 78,151 behind the hardened seed-boundary runner.
- 75 unit tests green; smoke green end-to-end; no model event has run.
2026-07-15 — Adversarial review: bounded-grammar amendment pre-freeze
- Four-lens review (contract, corpus with hand-simulation, gate statistics, seed safety) confirmed one MAJOR and refuted nothing: the feedloop specs documented unbounded op parameters while the uniqueness audit enumerated only the finite grammar — 13/80 training and 2/20 holdout rows admitted out-of-grammar valid fixes (false-promotion bias on the strict kind gate; false uniqueness claims in 13 think targets).
- Fixed before any commit or model event: every parameterized operation's legal values are now documented in the rendered specs (all four formalisms, including the previously undocumented item pools), think targets reworded to quantify over legal steps, and generation hard-fails unless an extended-grammar audit (amounts to 12, full pools) shows every out-of-bound alternative excluded only by the rendered clause. Same rng draws; all 100 feedloop rows genuinely unique; counts test-pinned.
- Corpus/streams/receipts regenerated (corpus e6d45ed4…, MILP re-solved exact zero-delta, retention screens byte-unchanged); 76/76 tests green; smoke green.
2026-07-15 — Training, the pooled_k3 gate, and closure
- Both arms trained clean behind green checkpoints (control loss 0.4334, candidate 0.54); merges published and pinned; the 12-run local event executed with boundary re-authentication throughout.
Verdict NOT_PROMOTED on three frozen bars: axis total tied replay 11–11; u_feedloop 0/20 versus 1/20 for both controls (the candidate trained on 80 in-formalism rows and installed nothing); pooled retention 59.33 vs replay 65.00 — 0.67 outside the ±5 pooled band (parent band passed at −3.0). u_statechain installed: 11/20, strict over parent 7 and replay 10. Event delta SD 4.08 ≈ the calibration's 4.27 — the new instrument behaved exactly as calibrated on first use.
- Seed 78,151 permanently sealed. The mechanism map sharpens: state-chain protocol is teachable; repair-with-feedback is the third failed pedagogy at the menders-shaped skill; replay gained retention again.
Reproduce
Smoke test
PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -B experiments/qwen35_4b_feedback_loop_state_chain_install/scripts/run.py --smokeFull run
checkpointed scripts/run.py stages train-control / train-candidate / merge-arms / local / benchmarkRun steps are documented inside the experiment folder (README and scripts).