On-Policy Failure-Prefix Universal Curriculum

Results at a glance 4

Overview

This result-separated successor tests whether training corrective continuations from the model's own fresh procedural failure prefixes installs a reusable reasoning and commit policy better than another idealized trace curriculum or exact-token replay.

Research Program

• Program: agentic_breadth_installation
• Program question: can a contamination-free installed mechanism improve every held-out benchmark family rather than redistribute wins?
• Closest near-duplicate: qwen35_4b_universal_state_table_compiler_token_match.
• Additional anchors: qwen35_4b_gauntlet_breadth_round1, qwen35_4b_interactive_policy_curriculum, qwen35_4b_verifier_conditioned_recovery_bank, C53, C56, and C59.

Question

Do masked corrective continuations attached to the authenticated parent's actual failure prefixes teach bounded execution, induction, scoring, and exact commitment at the deployed interface while preserving its broad replay policy?

Hypothesis

The state-table predecessor was executable and truth-audited but off-policy: its ideal traces did not resemble the model's actual declaration confusion, repeated induction, score-count errors, or correct-state-without-commit prefixes. Fresh parent rollouts plus executable-oracle continuations should place supervision exactly at those states. The mechanism is false unless the sole candidate strictly beats both the unchanged parent and a same-parent exact-forward-token replay continuation on a fresh paired local gate, overall and on execute/induct/probe.

Setup

• Model: only Qwen/Qwen3.5-4B, pinned revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a.
• Proposed parent: authenticated close_xi adapter; the failed scaffold and state-table candidates are not inherited.
• Proposed data source: fresh experiment-owned procedural tasks, followed by authenticated parent rollouts and executable-oracle failure localization. No prior local-gate item may enter training.
• Proposed treatment: masked assistant-prefix correction at first observable failure states, including bounded commit, declaration-versus-operation parsing, induction loop termination, probe-score recomputation, repair propagation, and exact answer serialization.
• Mechanism-falsifying control: independent same-parent replay continuation matched on encoded forward tokens, optimizer steps, seed, backend, and aligned shared replay positions.
• Hidden-label boundary: benchmarks/ remains read-forbidden. Only an aggregate gateway may run after local promotion; all-family lift, higher-tier confirmation, and matched-compute sample-more remain required for a universal claim.
• Reserved seeds: construction 77113, parent rollout 66113, training 47, fresh local 88009, and conditional aggregate 78139.

Run

Model-free design smoke:

PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -B experiments/qwen35_4b_universal_on_policy_prefix_repair_token_match/scripts/run.py --smoke

The design, parent merge, parent rollout, failure inventory, and exact-token freeze are separate published checkpoints. Verify every model-free derived artifact with:

PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -B experiments/qwen35_4b_universal_on_policy_prefix_repair_token_match/scripts/run.py --smoke

The terminal local event is preserved and authenticated by the same smoke command. Do not rerun it or open an aggregate stage in this directory. Any successor needs a new result-separated experiment and fresh seeds.

Results

CPU feasibility passed. Construction seed 77,113 deterministically freezes 288 truth-audited tasks, 48 for each of six failure classes. The model-facing JSONL omits hidden oracle and answer fields. Tests cover exact prefix masking, failure-only selection, delayed-commit cutoff, declaration misuse, generation caps, and the merged-Qwen architecture gate. The authenticated close_xi adapter was explicitly merged into a full composite: 128/128 applied LoRA modules were nonzero, the merged weight hash is 4933f2dd...eb373, and the external merge-receipt hash is 1fbc84b3...5557. From pushed-green commit 21e1eb59, one frozen same-backend parent event produced all 288/288 greedy natural-thinking rollouts, 170,252 sampled tokens at 849.9 tokens/s. Rollout/metadata/log/receipt hashes are 8010632f...3b17f / 9fe81276...664 / ed0d4fc4...26b7 / c6b98b79...74fa. The original postvalidator rejected only an impossible post-open git_dirty=false assertion; an explicit no-generation recovery path authenticated every other frozen field and wrote the receipt without rerunning the model.

The frozen miner has now graded the experiment-owned substrate. It found 230 failed and 58 passing parent rows; all 230 failures had a reachable clean prefix. Available failures by bounded-induction/commit/declaration/probe/repair/state class were 46/48/35/24/36/41, so every fixed quota cleared without borrowing. It selected exactly ten per class. The 60-row repair source and complete inventory hashes are 30141538...d84b8 / 7230af52...dfe7. Selected prefixes contain 47,123 masked tokens total (33 minimum, 785.4 mean, 1,024 maximum); 42 selections cut at the generation cap, ten at the immediate-commit boundary, and eight at the answer boundary.

The separately frozen training streams now contain 320 rows and exactly 304,313 forward tokens apiece, with zero skips, 200 byte-identical position-aligned replay rows, and 40 optimizer steps. All repairs fit below the 4,096-token ceiling; the largest final row is 2,991 tokens. The candidate replaces 33,421 replay target tokens with masked context, leaving 111,983 nonzero-weight tokens and 25,049.4 absolute loss mass versus 145,404 and 31,311.2 for control. This is an explicit intervention caveat, not hidden behind the forward-token match. Token-receipt SHA-256 is eb08026f...e0cfc; the second review verdict is PASS_CONTROL_TRAINING.

From pushed-green commit a8529c04, the replay control then trained for exactly one epoch and 40 updates from the authenticated parent. It encoded 320/320 rows with zero skips, consumed the registered 304,313 forward tokens, and finished with training loss 0.4588 in 272.8 trainer seconds (292.4 wrapper seconds). The normalized log/receipt hashes are a49076ec...3501 / f78f2069...d6de; adapter config/weights are 0dfd9bda...120f / bb59d3bd...5154d. From separately pushed-green control checkpoint b690a4b3, the prefix-repair candidate independently restarted from the same authenticated parent and also completed 320/320 rows, zero skips, one epoch, and 40/40 updates over 304,313 forward tokens. Its final training loss was 1.288 in 282.4 trainer seconds (298.2 wrapper seconds). Candidate log/receipt hashes are e895c546...ca0 / 846d8107...7098; adapter config/weights are 91b7db57...37de / 85811191...0f14. Each 169,903,320-byte adapter has 256 tensors and 42,467,328 elements; every tensor is finite and nonzero. These are operational training results only. No capability measurement, local evaluation, or benchmark event exists.

Fresh local seed 88,009 is now frozen model-free after paired training. It contains 26 truth-audited tasks, two for each of 13 skills. Source/model-input/receipt hashes are 9682744e...acdee / ff407551...ce988 / 3982d5b8...6e85a; input rows expose neither answer nor oracle. Canonical messages have zero overlap with 658 training or parent-collection messages and 234 messages regenerated at prior reserved local seeds. A pre-outcome protocol amendment moves all three arms from the prospective Transformers path to the now-mandatory pinned vLLM runner. Each arm will use an explicit merged composite and identical greedy natural-thinking geometry. The absolute 24-parse/17-correct gate and strict total plus execute/induct/probe wins over both controls remain frozen. At that design checkpoint, review verdict PASS_CONTROL_MERGE authorized only the replay-control merge; no trained-arm merge, local model call, capability score, or benchmark event yet existed.

From pushed-green local-design commit 6dc0e677, the replay-control adapter was then explicitly merged into the Qwen3.5 composite. All 128 applied LoRA modules were nonzero. Tracked run-receipt/log hashes are bc78f332...d550 / 7ab404b8...8995; external merge-receipt/full-weight hashes are aa763255...45a3 / 7ab4c419...6e2e. The 9,078,620,536-byte shard passed the exact merged-Qwen architecture and frozen local engine-request gate. This is deployment lineage only. At that checkpoint no candidate merge, local model call, capability score, or benchmark event existed.

After that checkpoint was rebased, pushed, and passed Validate Repository run 29355088731 plus Publish Research Site run 29355089298, the candidate was merged from clean SHA 619f1e53. It likewise applied 128/128 nonzero LoRA modules and saved one 9,078,620,536-byte shard. Tracked run-receipt/log hashes are 3deff026...438d / 58c7c9ec...d9f6; external merge-receipt/full-weight hashes are baa2027e...6d5a / 376e2082...b528. Independent lineage, merged-Qwen architecture, and exact frozen engine-request validation passed. Both trained-arm deployments now exist, but no local model call, capability score, or benchmark event yet existed.

From pushed-green candidate-deployment commit a12e4758, one paired vLLM event ran on fresh seed 88,009. Parent/replay/candidate scored 16/18/15 correct, parsed 24/23/23, and contacted the generation cap 2/3/3 times. Their execute+induct+probe subtotals were 2/1/0 of six: candidate was 0/2 on all three target skills. It failed the accuracy, parse, cap, execute, induct, and probe gates and all four strict control-relative checks. Paired against replay, candidate won one task and lost four; no per-skill count improved, while order, probe, and trace each lost one. Local/promotion receipt hashes are b4b333ca...b8c8 / 1e048e75...f5c. All nine raw output/metadata/log hashes independently recomputed, benchmark_data_read=false, promotion is empty, and aggregate seed 78,139 remains sealed.

Interpretation

The on-policy source solved data availability, not installation. Teacher-forcing long realized failure prefixes did not teach the model to avoid or repair analogous states when it had to generate the prefix itself. Selection was dominated by cap boundaries, and the candidate sacrificed 33,421 supervised target tokens relative to replay; the result therefore rejects this complete matched-forward-compute recipe, not every possible on-policy correction objective. Retire long masked failure-prefix continuation as the next lever. A successor should move supervision before the failure—at short, prospectively detectable decision boundaries—and must preserve full target exposure or use an exact target-token control.

Knowledgebase Update

• Program evidence and shared synthesis: record the terminal local negative and the pre-failure/target-exposure design constraint it adds.
• Program backlog: retires this mechanism and keeps aggregate seed 78,139 sealed.
• Claim ledger: unchanged; no broad capability claim was exposed.

Artifacts

• idea_intake.md
• configs/default.yaml
• scripts/run.py
• scripts/gen_rollout_tasks.py
• scripts/mine_prefix_repairs.py
• scripts/measure_source_tokens.py
• scripts/materialize_streams.py
• scripts/validate_streams.py
• scripts/train_trial.py
• scripts/gen_local_gate.py
• scripts/check_local.py
• scripts/merge_trained_arm.py
• scripts/eval_local_vllm.py
• data/design_receipt.json
• data/rollout_task_manifest.json
• data/prefix_failure_inventory.json
• data/prefix_repair_source.jsonl
• data/source_token_lengths.json
• data/stream_manifest.json
• data/stream_token_receipt.json
• data/replay_after_close.jsonl
• data/prefix_repair_after_close.jsonl
• data/local_tasks_seed88009.jsonl
• data/local_input_seed88009.jsonl
• data/local_design_receipt.json
• runs/parent_rollout/seed66113.receipt.json
• runs/training/replay_after_close.log
• runs/training/replay_after_close.json
• runs/training/prefix_repair_after_close.log
• runs/training/prefix_repair_after_close.json
• runs/merges/replay_after_close.log
• runs/merges/replay_after_close.json
• runs/merges/prefix_repair_after_close.log
• runs/merges/prefix_repair_after_close.json
• runs/local/seed88009.json
• runs/local/seed88009_promotion.json
• runs/local/seed88009_close_xi_parent.jsonl
• runs/local/seed88009_close_xi_parent.meta.json
• runs/local/seed88009_close_xi_parent.log
• runs/local/seed88009_replay_after_close.jsonl
• runs/local/seed88009_replay_after_close.meta.json
• runs/local/seed88009_replay_after_close.log
• runs/local/seed88009_prefix_repair_after_close.jsonl
• runs/local/seed88009_prefix_repair_after_close.meta.json
• runs/local/seed88009_prefix_repair_after_close.log
• analysis/prefix_failure_inventory.md
• reports/design_review.md
• reports/compute_review.md
• reports/local_design_review.md
• reports/preregistration.md
• reports/report.md
• reports/artifact_manifest.yaml

Report

Rendered from reports/report.md

Summary

Model-free design, explicit parent deployment, authenticated rollout collection, failure-only mining, and the second exact-compute review are complete. Every fixed class quota passed, and two frozen 320-row streams match at exactly 304,313 forward tokens with zero skips. Both independently initialized arms have now trained and authenticated. The fresh same-backend local gate rejected prefix repair 15/26 versus replay 18/26 and parent 16/26; no benchmark event ran and aggregate remains sealed.

Research Program Fit

The experiment belongs to agentic_breadth_installation. It changes the intervention state from idealized truth traces to on-policy deployment prefixes while retaining the universal line's exact-token replay and strict promotion contract.

Method

Construction seed 77,113 produces 288 truth-audited tasks balanced across six failure classes. The authenticated close_xi adapter is explicitly merged because runtime vLLM LoRA is a verified silent no-op. One greedy natural-thinking vLLM event collected 288 parent outputs at seed 66,113 and cap 1,024. The frozen model-free miner selected ten reachable failures per class and masks every generated parent-prefix token from loss. Deterministic stream construction combines 200 position-aligned shared replay rows with either 60 repairs plus 60 replay fillers or 120 disjoint replay-control rows. The actual training encoder and pinned tokenizer measure every final row.

Results

CPU feasibility, deterministic generation, and adversarial design review passed. Source/model-input hashes are 32589348...1172 / 7a643e96...a5485c; design receipt hash is 98c6a168...5638. The parent composite merge then applied 128/128 nonzero LoRA modules. Its single weight shard is 4933f2dd...eb373 and its external merge receipt is 1fbc84b3...5557. The frozen parent event then completed all 288/288 rollouts with 170,252 sampled tokens at 849.923 tokens/s. Rollout/metadata/log hashes are 8010632f...3b17f / 9fe81276...664 / ed0d4fc4...26b7 and the authenticated receipt is c6b98b79...74fa. The initial postvalidator rejected only its own post-open dirty-tree condition; explicit recovery reran no generation and bound the completed event to commit 21e1eb59. At that rollout checkpoint, no failure grading or downstream event had run.

The separately published rollout then opened model-free grading. Of 288 rows, 230 failed at least one registered condition and 58 passed; all 230 failures had a clean reachable prefix. Available failures by bounded-induction/commit/declaration/probe/ repair/state class were 46/48/35/24/36/41, clearing every quota of ten. The 60-row repair source is 30141538...d84b8; full inventory is 7230af52...dfe7. Selected prefixes contain 47,123 masked tokens, with min/mean/max 33/785.383/1,024. Forty-two cut at the generation-cap boundary, ten at the first token beyond the commit budget, and eight at the answer boundary.

The final control/candidate hashes are 541805df...be6 / 9a43f3be...03f1, and the exact token receipt is eb08026f...e0cfc. Both arms contain 320 rows, 304,313 forward tokens, zero skips, 200 aligned common rows, and 40 updates. The longest row is 2,991 of 4,096 tokens. Forward compute is equal, but target composition is not: candidate minus control is +33,421 masked-context, −33,949 think-target, zero close-target, +528 answer-target, and −33,421 total target tokens. Candidate/control nonzero-weight tokens are 111,983/145,404; absolute loss masses are 25,049.4/31,311.2. The second review records this ambiguity and authorizes only the control. No model load, adapter training, capability measurement, or benchmark event ran during the freeze.

After compute-freeze commit a8529c04 passed both workflows, the replay control trained for one epoch and exactly 40 updates. It encoded 320/320 rows with zero skips, finished at loss 0.4588, and took 272.8 trainer seconds. Log/receipt hashes are a49076ec...3501 / f78f2069...d6de; adapter config/weights hashes are 0dfd9bda...120f / bb59d3bd...5154d. The 169,903,320-byte adapter contains 256 finite, nonzero tensors and 42,467,328 elements. This authenticates the control artifact but is not capability evidence.

After that control checkpoint passed both workflows as commit b690a4b3, the prefix-repair candidate independently restarted from the same parent. It likewise encoded 320/320 rows with zero skips and completed exactly 40 updates over one epoch. Final loss was 1.288; trainer/wrapper times were 282.4/298.2 seconds. Candidate log/receipt hashes are e895c546...ca0 / 846d8107...7098; adapter config/weights hashes are 91b7db57...37de / 85811191...0f14. Its 169,903,320-byte adapter also contains 256 finite, nonzero tensors and 42,467,328 elements. This completes the paired operational training stage, not a capability comparison.

After paired training, local seed 88,009 froze 26 truth-audited tasks, two per registered universal skill. Source/model-facing/receipt hashes are 9682744e...acdee / ff407551...ce988 / 3982d5b8...6e85a. Model input contains only ids, messages, and public metadata. It has zero canonical-message overlap with 658 training/collection messages and 234 regenerated messages at prior reserved local seeds. No model was called.

Before any local outcome, the active repository inference contract required a symmetric amendment from the prospective Transformers process to the pinned vLLM runner. Parent, replay, and candidate will all deploy as explicit composites with identical natural-thinking, greedy, seed, token cap, batch geometry, and runner bytes. The frozen absolute and strict control-relative promotion rules are unchanged. Local review verdict PASS_CONTROL_MERGE authorizes only the separately checkpointed replay-control merge.

After local-design commit 6dc0e677 passed both workflows, that replay-control merge applied 128/128 nonzero LoRA modules. Tracked receipt/log hashes are bc78f332...d550 / 7ab404b8...8995; external merge receipt and 9,078,620,536-byte weight shard hash to aa763255...45a3 / 7ab4c419...6e2e. The saved composite passes the exact Qwen3.5 architecture and frozen local engine-request gate. This is an authenticated deployment artifact, not capability evidence.

After that checkpoint passed Validate Repository run 29355088731 and Publish Research Site run 29355089298, candidate merge from clean SHA 619f1e53 applied 128/128 nonzero modules. Tracked receipt/log hashes are 3deff026...438d / 58c7c9ec...d9f6; external receipt and 9,078,620,536-byte weight shard hash to baa2027e...6d5a / 376e2082...b528. Independent lineage, architecture, and frozen engine-request validation passed. This too is deployment evidence only.

The single authorized local event then ran from pushed-green commit a12e4758. Parent/replay/candidate scored 16/18/15 correct, 24/23/23 parsed, and 2/3/3 cap contacts on the same 26 fresh rows. Execute+induct+probe subtotals were 2/1/0 of six; candidate scored zero on every target kind. It failed six absolute checks and all four strict relative checks. Against replay it flipped one miss to a win but four wins to misses; no skill count improved, while order, probe, and trace each fell by one. Local/promotion receipt hashes are b4b333ca...b8c8 / 1e048e75...f5c. Independent recomputation verified the promotion decision and all nine raw hashes. No benchmark data was read and aggregate seed 78,139 remains sealed.

Controls

Baseline is authenticated close_xi. The mechanism-falsifying control is an independent same-parent replay continuation matched on exact encoded forward tokens, optimizer steps, seed, and aligned shared replay. It must train and publish first. Both arms have now trained independently from the parent, and candidate preflight authenticated the committed control receipt, log, and external adapter before model load. The local deployment order is published parent composite, replay-control merge, candidate merge, then one three-arm vLLM local stage. Every transition completed and was preserved separately; the failed local gate terminates the experiment.

Oracle Versus Deployable Evidence

Executable truth is permitted only to construct tasks, grade parent failures, and build corrections or the fresh local gate. Hidden oracle fields are excluded from both rollout and local model input; commit tasks deliberately expose verified work as their public task substrate. Local seed 88,009 was materialized only after training, and its messages are disjoint from training and prior reserved local seeds. benchmarks/ remains read-forbidden and the aggregate gateway stays sealed.

Interpretation

The parent supplies enough failures in every registered class and exact forward compute is matched, so data availability and deployment do not explain the negative. Long realized failure-prefix teacher forcing did not transfer to generating or repairing analogous prefixes on fresh tasks. The selected set is dominated by capped states, while masking those contexts leaves 33,421 fewer supervised target tokens than replay. The result rejects the complete recipe but does not separate prefix conditioning from target-composition effects. “First failure” also remains the first machine-observable boundary, which can occur too late to teach the decision that caused it.

Next Experiments

Do not rerun seed 88,009 or open aggregate seed 78,139. A result-separated successor should intervene at short pre-failure decision boundaries and match supervised target exposure, with fresh local and conditional aggregate seeds.

Artifact Manifest

Parent identity, frozen task hashes, replay hashes, the staged external merged checkpoint, parent-rollout hashes, stream hashes, and both trained adapters are recorded in artifact_manifest.yaml; local task/input/protocol hashes and both merged-arm deployments are registered. The terminal local receipt, promotion receipt, and every raw arm artifact are tracked; no benchmark result exists.

Experiment log 12

Reproduce

Smoke test

PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -B experiments/qwen35_4b_universal_on_policy_prefix_repair_token_match/scripts/run.py --smoke

Full run

none: terminal local negative; verify the preserved event with smoke_command and use a new experiment for follow-up work

Run steps are documented inside the experiment folder (README and scripts).

Browse the experiment folder on GitHub ↗

Related

Related experiments

Nearest by shared name, tags, and programs.

• 2026-07-13 Mid-Density Token-Matched Universal Curriculum
• 2026-07-14 Search-Scaffold Universal Curriculum
• 2026-07-13 Low-Density Token-Matched Universal Curriculum
• 2026-07-13 → 14 Close-Weighted Universal Commit Seam
• 2026-07-14 Policy-Supported Successful-Sibling Universal Curriculum
• 2026-07-14 Natural-Language State-Table Universal Curriculum