Replay-Anchored Universal Curriculum Continuation
The one idea you need
This experiment kept the model practicing broad tasks while adding a smaller set of abstract lessons. A replay-only comparison received the same number of updates, revealing whether any gain came from the designed lessons or simply from continuing broad practice.
The question
Can gentle joint practice add designed general skills without erasing the broad behavior the model already learned?
What we found
No at this dose. The designed mix passed its fresh synthetic screen but scored about 42% overall, below both the 44% mature policy and the 49% replay-only comparison. It also fell below base on one family. Replay-only was the useful surprise: it improved eight of ten families, tied the other two, and regressed none.
Why it matters
Broad replay was not merely protecting old behavior; it was still adding capability. Future designed curricula must beat another replay continuation, not just the older starting policy. The next search should begin from this stronger replay-refreshed model, use far fewer designed rows, and match token exposure as well as update count.
On this page
Results at a glance 2
How to read
For each family, compare the replay-only and designed-mix changes from base. Bars below zero are regressions; replay-only has none while the designed mix has one.
Takeaway → The designed mix trails replay-only by about six aggregate points; replay saw 17% more token exposure, so the next test must token-match.
Data table
| public benchmark family | replay refresh minus base | warm union minus base |
|---|---|---|
| chronicle | 0.5 | 0.625 |
| lockpick | 0.25 | 0 |
| menders | 0.125 | 0 |
| mirage | 0.75 | 0.5 |
| rites | 0 | -0.125 |
| siftstack | 0.625 | 0.875 |
| sirens | 0 | 0 |
| stockade | 0.476 | 0.488 |
| toolsmith | 0.25 | 0.125 |
| warren | 0.125 | 0 |
Numbers from experiments/qwen35_4b_universal_replay_anchor/runs/benchmark/quick_tb1024_seed78133_pilot1/summary.json
Technical framing
Replay-only improves eight families; designed arm trails — Aggregate-only quick@1024, seed 78133, same merged qwen_vllm backend. Replay-only is nonnegative on all ten families and positive on eight; the designed arm regresses rites and loses 0.0613 aggregate. Replay received 17.3% more forward tokens, so mechanism attribution remains open.
How to read
Each bar totals 1,520 rows. Both streams share 1,120 replay rows; the candidate adds 400 designed lessons while the comparison adds 400 more replay examples.
Takeaway → The matched construction isolates designed lessons from the benefit of simply continuing broad practice.
Data table
| frozen dose component | warm_union candidate | replay_refresh control |
|---|---|---|
| shared replay | 1120 | 1120 |
| designed curriculum | 400 | 0 |
| additional replay | 0 | 400 |
Numbers from experiments/qwen35_4b_universal_replay_anchor/data/dose_manifest.json
Technical framing
Matched-step arms isolate 400 designed rows against extra replay — Both warm-start blend, contain 1,520 rows, and run 190 optimizer steps. The replay control receives 17.3% more forward-token compute, so a candidate win is conservative.
In the author’s words from the Overview · “Results”
warm_union consumed all 1,520 rows with zero skips, completed 190 steps in 1,320.4 seconds, and ended at finite loss 0.7727. Adapter weights SHA-256: 26837fade89e75ffd6cc5922b0dec4a1361e93d98228c9137ca59240e1a18f18. On frozen local seed 88,003 it passed every promotion gate: accuracy 0.7308, parse rate 0.9615, one cap contact, and zero feasible-route abstentions. Induction and state carry were each 0/2, so this is an installability pass rather than broad-transfer evidence. The pass authorized the matched replay_refresh control. It consumed all 1,520 rows with zero skips, completed 190 steps in 1,342.0 seconds, and ended at finite loss 0.4365. Its adapter SHA-256 is c296c774d20403c7de9c810bfa825dbbe22bd0683c37692a929bd2bb13e3d36a. … Read the full result →
Overview
Outcome: replay anchoring preserved local installability but the designed candidate lost to both controls; replay-only refresh became the strongest measured anchor.
Research program
- Program:
agentic_breadth_installation - Parent:
qwen35_4b_universal_curriculum - Prior anchors: C14, C53, C54, and C56
Question
Can a low-rate continuation from the strong C53 blend policy install the parent's truth-audited general procedures without displacing broad behavior when every update window is anchored by broad replay?
The parent sequential arm made a useful but narrow update: fresh synthetic accuracy rose from 0.5000 to 0.6923, yet the aggregate benchmark score fell 0.1385 behind blend and three public families regressed below base. This experiment changes only the integration geometry. It does not add benchmark-shaped data or inspect benchmark items.
The parent's preregistered from-base replay union subsequently reached 0.6923 local accuracy but failed its prospective parse-rate (0.8462) and cap-contact (4/26) gates; its benchmark stayed sealed. That strengthens the rationale for retaining the mature policy during integration without changing this experiment's already-frozen design.
Frozen design
Both trained arms warm-start the immutable C53 blend adapter and use Qwen/Qwen3.5-4B at revision 851bf6e..., learning rate 1e-5, LoRA rank 32 / alpha 64, batch 1 x accumulation 8, max length 4,096, w_think=0.2, seed 42, and one epoch (190 optimizer steps).
warm_union: 400 designed rows plus 1,120 broad replay rows.replay_refresh: the identical 1,120 replay rows plus 400 additional replay rows.blend: immutable strong control.base: pinned reserialized base control.
The exact nested doses are deterministically derived from copied, checksum-pinned parent artifacts. Both contain 1,520 rows with zero tokenizer skips. The candidate has 1,231,404 forward tokens; the replay-only control has 1,444,589, making the compute asymmetry conservative for the candidate.
See idea_intake.md, design_review.md, and preregistration.md for the full rationale and gates.
Run
The non-GPU smoke path checks deterministic bytes, source and dose hashes, exact row counts, zero skips, the sole permitted model identity, and Python syntax:
.venv/bin/python experiments/qwen35_4b_universal_replay_anchor/scripts/run.py --smokeExpensive stages are explicit and fail closed rather than overwriting prior artifacts:
.venv/bin/python experiments/qwen35_4b_universal_replay_anchor/scripts/run.py --stage train-candidate
.venv/bin/python experiments/qwen35_4b_universal_replay_anchor/scripts/run.py --stage local
.venv/bin/python experiments/qwen35_4b_universal_replay_anchor/scripts/run.py --stage train-control
.venv/bin/python experiments/qwen35_4b_universal_replay_anchor/scripts/run.py --stage merge
.venv/bin/python experiments/qwen35_4b_universal_replay_anchor/scripts/run.py --stage benchmarkThe local screen is frozen at seed 88,003. A passing candidate receives one aggregate- only Menagerie quick@1,024 event at seed 78,133 through the trusted vLLM gateway. The pilot requires strict improvement over base on all ten public families, aggregate at least as high as blend, and aggregate higher than the matched-step replay control.
Results
warm_union consumed all 1,520 rows with zero skips, completed 190 steps in 1,320.4 seconds, and ended at finite loss 0.7727. Adapter weights SHA-256: 26837fade89e75ffd6cc5922b0dec4a1361e93d98228c9137ca59240e1a18f18.
On frozen local seed 88,003 it passed every promotion gate: accuracy 0.7308, parse rate 0.9615, one cap contact, and zero feasible-route abstentions. Induction and state carry were each 0/2, so this is an installability pass rather than broad-transfer evidence. The pass authorized the matched replay_refresh control. It consumed all 1,520 rows with zero skips, completed 190 steps in 1,342.0 seconds, and ended at finite loss 0.4365. Its adapter SHA-256 is c296c774d20403c7de9c810bfa825dbbe22bd0683c37692a929bd2bb13e3d36a. Both adapters were explicitly merged: the candidate merged weight SHA-256 is 29baf3ad182e900d01186058d795004b984a5f436958d8cca5e0ebcf199422f6, and the replay-control merged weight SHA-256 is 22c61cebd6091d0b8380e2d7318b4d4db99ef24eb30b942157e44f37d26cbc9e. On the frozen quick@1,024 seed 78,133 event, warm_union scored 0.4238 aggregate: 0.2488 above base, but 0.0172 below blend and 0.0613 below replay_refresh. It regressed rites by 0.125 below base, tied four other families, and therefore failed every universality promotion rule except positive aggregate.
The mechanism control was the important positive lead. replay_refresh scored 0.4851, beating base by 0.3101 and blend by 0.0441. It had no negative family, improved eight of ten, and tied base on rites and sirens. This is not a universal-feature result because the two ties violate the strict all-family requirement; it is the strongest next anchor and shows that continued broad replay itself had not saturated. Because the replay arm received 17.3% more forward tokens despite matched optimizer steps, the full candidate-control gap is not a clean content attribution; the successor must token-match the arms.
Artifacts
data/dose_manifest.json: deterministic source lineage and dose hashes.data/dose_token_receipt.json: exact token exposure and zero-skip proof.scripts/materialize_doses.py: nested stratified dose construction.scripts/train_trial.py,eval_curriculum.py,merge_trial.py, andrun_benchmark.py: fail-closed staged harnesses copied into this result-bearing successor.reports/artifact_manifest.yaml: external adapter and merged-model locations.
Report
Rendered from reports/report.md
Summary
Complete negative for the designed curriculum. Replay anchoring preserved a clean local install, but the candidate lost to both the strong starting policy and the replay-only mechanism control on the paired aggregate event. The replay-only control became a stronger next anchor with no family regression and eight strict family gains.
Research program fit
This is the result-separated integration follow-up to qwen35_4b_universal_curriculum. The parent established that 800 truth-audited designed lessons are installable but, when continued sequentially from blend at 5e-5, specialize and displace broad behavior. This experiment tests replay anchoring at a fivefold lower learning rate. Its from-base replay-union control also failed local parse and cap gates, leaving mature-policy retention as the unresolved geometry.
Method
Both arms warm-start the immutable C53 blend adapter and use the pinned Qwen/Qwen3.5-4B. Each receives 1,520 rows, one epoch, and 190 effective-batch-8 optimizer steps. The candidate substitutes 400 designed rows for 400 replay rows while sharing the other 1,120 replay rows byte-for-byte with the mechanism control.
Exact source and derived hashes are in data/dose_manifest.json. At max length 4,096, all 3,040 arm-rows encode without a skip. Candidate exposure is 1,231,404 forward tokens; replay-control exposure is 1,444,589.
Results
warm_union completed 1,520/1,520 rows, zero skips, 190 steps, finite loss 0.7727, and an authenticated nonzero adapter. On seed 88,003 it achieved 0.7308 exact accuracy, 0.9615 parse rate, 1/26 cap contacts, and zero feasible-route abstentions, passing all four local gates. Per-kind residuals include induction 0/2 and state 0/2; local success does not establish universality.
replay_refresh completed 1,520/1,520 rows, zero skips, 190 steps, and finite loss 0.4365. Its adapter SHA-256 is c296c774...d36a, versus 26837fad...8f18 for the candidate. Both were explicitly merged. The candidate merged weight SHA-256 is 29baf3ad...22f6; the replay-control merged weight SHA-256 is 22c61ceb...bc9e. On frozen quick@1,024 seed 78,133, aggregate scores were base 0.1750, blend 0.4410, replay_refresh 0.4851, and warm_union 0.4238. The candidate was +0.2488 versus base, -0.0172 versus blend, and -0.0613 versus replay refresh. Its rites score fell 0.125 below base; only five families strictly improved. It therefore failed the no-negative-family, every-family-positive, strong-control, mechanism-control, and overall pilot gates.
replay_refresh beat base by 0.3101 and blend by 0.0441. All ten family deltas versus base were nonnegative and eight were strictly positive; rites and sirens tied base. This is not the registered all-family outcome, but it is a material control result and the correct anchor for a result-separated successor.
Controls
- Immutable base and C53
blendstrong controls. - Optimizer-step-matched replay-only mechanism control with more token compute.
- Fresh local seed 88,003 and, conditionally, aggregate-only quick@1,024 seed 78,133.
- Explicit adapter merges and one
qwen_vllmbackend for every benchmark arm. - No benchmark item, transcript, verifier detail, result detail, or raw stream crosses the trusted gateway.
Interpretation
The designed signal remained locally learnable at low rate, but the arm containing 400 designed rows transferred worse than replay alone. The candidate does not establish a universal feature. Because replay refresh had 17.3% more forward-token exposure despite matched optimizer steps, this comparison rejects the candidate but does not isolate designed content as the cause of the full gap. The replay control still shows that the mature policy was improvable by continued diverse practice; broad replay is an active capability baseline, not a neutral retention ingredient.
Next experiments
Use replay_refresh as the immutable anchor in a new experiment. Target the two tied families through abstract, contamination-free procedures only after a fresh local qualification, while requiring retention of all eight strict gains. Match both optimizer steps and forward-token exposure against replay continuation, and retain matched-compute sampling in confirmation on a fresh aggregate seed. Do not retune this result-bearing directory or reuse seed 78,133.
Artifact manifest
See artifact_manifest.yaml; planned entries are replaced with authenticated checksums immediately after training and merge.
Experiment log 6
Show the running log (6 entries, 2026-07-13)
Scaffold
Created as a result-separated successor after the parent designed-only arm displaced three benchmark families and lost 0.1385 aggregate to blend.
2026-07-13 — design freeze and smoke
- Copied the parent's 800-row truth-audited designed corpus and the C53 2,240-row broad replay corpus into the experiment; source SHA-256 values match their parents.
- Materialized exact nested arms: candidate = 400 designed + 1,120 shared replay; control = the same 1,120 replay + 400 additional replay.
- Both arms have 1,520 rows and 190 optimizer steps. Exact tokenization at max length 4,096 produced zero skips. Candidate dose is 1,231,404 forward tokens and control is 1,444,589.
- Frozen local seed 88,003 and aggregate-only quick@1,024 seed 78,133 before training.
scripts/run.py --smokepassed, and all three dose construction tests passed.
2026-07-13 — parent factorial closed; candidate launch
- The parent's from-base 800-designed + 2,240-replay arm completed 3,040/3,040 rows with zero skips, then failed its frozen local parse (0.846 < 0.90) and cap-contact (4 > 2) gates at seed 88,002. Benchmark seed 78,132 remained unconsumed.
- Started the frozen
warm_unioncandidate only after the parent negative was durable.replay_refreshremains gated on candidate local success to avoid unnecessary spend.
2026-07-13 — candidate local pass
warm_unioncompleted all 1,520 rows and 190 optimizer steps with zero skips, finite loss 0.7727, and adapter SHA-25626837fad...8f18.- Frozen seed 88,003 passed: accuracy 0.7308, parse 0.9615, cap contacts 1/26, route abstentions 0. Induction and state remained 0/2 and are preserved as residuals.
- Candidate training receipt SHA-256 is
450e367c...cbd4; local gate receipt SHA-256 is7ae1d6ae...fe11. - The pass authorized and launched the frozen
replay_refreshmechanism control.
2026-07-13 — mechanism control and explicit merges
replay_refreshcompleted all 1,520 rows and 190 optimizer steps with zero skips, finite loss 0.4365, and adapter SHA-256c296c774...d36a. Its checked-in training receipt SHA-256 isf2a92713...cba0.- Explicitly merged the candidate and control into full checkpoints. Candidate merged weight SHA-256 is
29baf3ad...22f6; replay-control merged weight SHA-256 is22c61ceb...bc9e. - External merge-receipt SHA-256 values are
35894a31...63cf(candidate) andf32d7fc6...af1c(control). The frozen paired aggregate event is now authorized.
2026-07-13 — aggregate pilot; candidate negative, replay anchor advances
- Consumed the sole frozen quick@1,024 seed 78,133 event through the trusted aggregate-only vLLM gateway. All four arms used explicit merged checkpoints.
- Aggregate scores: base 0.1750,
blend0.4410,replay_refresh0.4851, andwarm_union0.4238. warm_unionwas +0.2488 versus base but -0.0172 versusblendand -0.0613 versus replay refresh. It regressedritesby 0.125 below base, so the frozen all-family, strong-control, and mechanism-control gates all failed.replay_refreshwas +0.3101 versus base and +0.0441 versusblend, with eight strictly positive families and no negative family. It tied base onritesandsirens, so it is a stronger anchor rather than a universal-feature winner.- Replay refresh had 17.3% more forward-token exposure despite matched optimizer steps. The candidate is rejected, but the whole gap cannot be assigned to content; a successor must match both steps and token exposure.
- Closed the experiment as a preserved negative. Any adaptive targeting of the two tied axes must use a new experiment, new local seed, and new benchmark seed.
Data files 1
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
Reproduce
Smoke test
.venv/bin/python experiments/qwen35_4b_universal_replay_anchor/scripts/run.py --smokeFull run
.venv/bin/python experiments/qwen35_4b_universal_replay_anchor/scripts/run.py --stage allRun steps are documented inside the experiment folder (README and scripts).