Qwen3.5-4B Same-Prefix Advantage Routing
The one idea you need
Picture a lead actor who keeps fumbling one exact scene. Two understudies each do several takes of that same scene. You swap one in only if it clearly beats both the actor and the rival, and only after a fresh, separate batch of takes confirms it.
The question
When a small model stalls partway through a problem, can you tell which of two specialist helper models would actually finish it better from that exact spot?
What we found
Only when it replicates, and here one helper didn't. The test actually ran: the deep specialist's local edge held up on a fresh, separate batch, but the quick specialist looked strong on one batch of stuck points and then went slightly negative on a second, so the check refused to trust it and the whole plan was called off. The insight is the reframe: a leaderboard tells you which teacher wins on average, not which one helps at a specific stuck moment, and here the stronger teacher's overall edge barely clears the bar. So each teacher must prove itself where the student fails, on fresh samples, or it sits out.
Why it matters
When merging specialist models into one small model, don't pick your teacher by leaderboard rank. Score each candidate on the exact states your model fails, copy its behavior only where it verifiably wins on separate samples, and skip the rest.
On this page
Results at a glance 2
How to read
Two equal-height bars. The left bar counts trial continuations used to pick a teacher; the right counts a separate untouched set used to verify its edge. Each holds 4,608 continuations across two problem blocks. Equal size is the point.
Takeaway → The pool that picks a teacher and the pool that confirms it are exactly the same size and fully separate, so a teacher can never be chosen and graded by the same evidence.
Data table
| branch role | continuation branches across two blocks |
|---|---|
| teacher selection | 4608 |
| independent teacher audit | 4608 |
Numbers from experiments/qwen35_4b_same_prefix_advantage_routing/configs/default.yaml and reports/preregistration.md
Technical framing
Preregistered split-branch evidence budget — Pre-run design chart: 192 states per block × two blocks × three policies × four branches. Selection outcomes choose the teacher; only the equally sized disjoint audit split estimates advantage.
How to read
Three bars showing where each round's 80 lesson-units come from: 30 from quick-teacher states, 30 from deep-teacher states, and 20 from replaying the original blend. Taller bars carry more of the training; the two teachers are matched.
Takeaway → Three-quarters of each round teaches new behavior split evenly between the two teachers, while a quarter re-anchors the original skills as a guard against forgetting.
Data table
| update source | training units |
|---|---|
| quick-routed | 30 |
| deep-routed | 30 |
| frozen-soup anchor | 20 |
Numbers from experiments/qwen35_4b_same_prefix_advantage_routing/configs/default.yaml and runs/smoke/summary.json
Technical framing
Preregistered consume-once units per MOPD round — Pre-run design chart: 20 optimizer updates × four microsteps = 80 units. Capability occupies 75%, split equally between teachers; the frozen starting soup anchors the remaining 25%.
In the author’s words from the Report · “Interpretation Contract”
The first terminal branch is route nonexistence for the required two-teacher composition claim. Locality, integration, controls, confirmation, and benchmarks are unreached—not negative and not inferred.
Overview
Status
Terminal negative at the preregistered split-branch route gate. Deep had replicated positive continuation advantage, but quick was worse than the soup student in block 1 (-0.0253 macro) after being strongly positive in block 0. MOPD, locality, controls, confirmation, and benchmarks were therefore not run.
Research Program
- Primary program:
agentic_breadth_installation. - Supporting programs:
posttraining_and_adaptation,evidence_conditioned_selection,benchmark_generalization,process_control_and_tool_use, andreliability_and_safety. - Program question: can training-only verification identify locally useful same-origin teachers and install their complementary behavior in one 4B checkpoint without the collapse seen in indiscriminate dense distillation?
- Closest anchors:
qwen35_4b_pareto_policy_integration,qwen35_4b_gauntlet_frontier, andqwen35_4b_opsd_pressure_locality_audit.
Question
On exact states visited by the strongest existing one-checkpoint student, can independent verifier branches identify both C54 source policies as genuinely better continuation teachers, and—only if that advantage replicates—can positive-advantage-routed MOPD produce one checkpoint that beats the student, both teachers, visible routing, matched controls, and sample-more?
Hypothesis
Coarse quick/deep labels failed because endpoint benchmark rank is not a local teacher label. The useful unit is a student-visible prefix. At each residual state, independent continuations from the quick teacher, deep teacher, and current student estimate continuation value. A teacher is used only when its selection-split mean is strictly above both alternatives; tied or nonpositive states abstain. If that rule retains positive advantage on fresh audit branches and two independent state blocks, corrected top-k reverse KL can transfer the locally better policy while a frozen-student anchor limits collateral drift.
Setup
- Model: only
Qwen/Qwen3.5-4Bat revision851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. - Teachers: independently regenerated C54
quick_blendanddeep_apexcomposites from the same pinned base—the pair used by C54's now directly measured tier router. The corrected large-n C54 estimate puts the apex medium gain at+0.321 ± 0.017 SE, straddling rather than decisively clearing+0.32; this run assumes no local teacher rank and measures it from scratch. - Student: an independently regenerated 40% quick / 60% deep convex LoRA-delta soup, the strongest existing joint point rather than either endpoint.
- Substrate: the self-contained 14-family procedural gym copied from the prior experiment; the 12 trained families produce updates and
brineworkplusspindleremain transfer-only. - State distribution: fresh failed student atoms at an exact mid-thought token prefix and failed interactive episodes immediately before the first invalid action (or the final action if every action was syntactically valid).
- Hidden-label boundary: scores select training states and teachers but are never rendered into a model prompt. Benchmark content is never read or imported; its CLI remains unreachable until every procedural gate passes.
- Primary metric: equal-weight quick/deep paired joint score on two sealed procedural blocks, with one-sided stratified-bootstrap lower bounds.
The Split-Branch Gate
For every frozen state, each of the two teachers and the student receives four route-selection continuations and four disjoint audit continuations. The route uses selection outcomes only. It chooses a teacher iff that teacher's mean is strictly greater than both the current student's and the alternate teacher's; otherwise it abstains. No effect-size margin is imposed.
The rule qualifies only if both named teachers have adequate preregistered support and, separately for each teacher, selected-teacher minus student and selected-teacher minus alternate-teacher are positive in both state blocks and have pooled one-sided 95% lower bounds above zero on the untouched audit branches. This is the experiment's teacher-existence test. MOPD is forbidden if it fails.
It failed exactly that test. Both teachers had ample support and the combined router was positive, but quick's selected states did not retain positive audit advantage over the current soup student in the independent block. The gate was not weakened to a pooled-only rule.
Post-result diagnostics localize the failure to conditional selection rather than weak absolute scoring. Selection/audit correlations for each policy were 0.79--0.86, yet only 6/26 quick routes in block 1 remained the strict audit winner. Its apparent selection margin over the student was +0.319, so adding a +0.10 or +0.25 threshold would still have produced a negative audit mean. Independent route halves agreed on the same quick states with Jaccard only 0.267 and 0.122 across blocks. See the exploratory diagnostic report.
Conditional Training and Controls
If qualified, four rounds refresh exact current-student states and apply the same frozen route. Seventy-five percent of consume-once units use corrected teacher-top-50 reverse KL from equally represented quick/deep routed states; 25% use the frozen soup as a retention anchor. A five-update exact-logit pilot must first keep centered non-target drift at or below 0.10 logits, entropy loss within 10%, and corrected top-k loss within the registered ceiling.
Matched controls are shuffled routing, the old visible quick/deep coarse route, off-policy best-teacher-continuation SFT, fixed-deep-teacher MOPD, the no-update soup, and explicit parameter soups. The deployable visible router selects quick for quick atoms and deep for deep atoms/episodes. The terminal sample-more arm is execution-filtered best-of-8 from the soup under the identical vLLM runner.
Run
CPU/scientific smoke:
python3 experiments/qwen35_4b_same_prefix_advantage_routing/scripts/run.py --smokeReached GPU stages are resumable and fail closed:
python3 experiments/qwen35_4b_same_prefix_advantage_routing/scripts/run.py --stage model-smoke
python3 experiments/qwen35_4b_same_prefix_advantage_routing/scripts/run.py --stage build-student
python3 experiments/qwen35_4b_same_prefix_advantage_routing/scripts/run.py --stage route-qualify
python3 experiments/qwen35_4b_same_prefix_advantage_routing/scripts/run.py --stage locality
python3 experiments/qwen35_4b_same_prefix_advantage_routing/scripts/run.py --stage integrate --seed 42
python3 experiments/qwen35_4b_same_prefix_advantage_routing/scripts/run.py --stage controls
python3 experiments/qwen35_4b_same_prefix_advantage_routing/scripts/run.py --stage confirmThere is deliberately no benchmark stage until a procedural confirmation receipt explicitly authorizes it.
Decision Rule
The final artifact is one merged 4B checkpoint with no deployment-time teacher or verifier. It passes only if its joint lower bound is above zero versus each teacher, the soup, the visible router, and every matched one-checkpoint control; both quick and deep means beat the better source endpoint in both blocks; three preregistered training seeds point in the same direction; retention and transfer regress by no more than 0.02; and greedy performance beats the soup's execution-filtered best-of-8. Tiny but replicated gains count. Large unstable gains do not.
Artifacts
idea_intake.md: novelty and anti-duplication decision.configs/default.yaml: frozen states, branches, seeds, gates, and controls.reports/preregistration.md: estimands and terminal decision rule.reports/design_review.md: adversarial review before model output.reports/literature_review.md: primary-paper basis for the design.reports/route_diagnostics.md: post-gate mechanism diagnosis and next tests.runs/preregistration_receipt.json: immutable design hashes and commit.analysis/: gate and final machine-readable receipts.reports/artifact_manifest.yaml: external checkpoints and regeneration.
If the procedural receipt opens the benchmark, the upstream C54 power correction is binding: report three quick events and at least eight medium events, all paired and without exclusion. A three-event medium mean cannot support the terminal claim.
Report
Rendered from reports/report.md
Status
Terminal negative at the preregistered route-qualification gate. Both teachers had adequate support, deep replicated, and the combined router was positive, but quick's audit advantage over the soup student reversed sign in block 1. No MOPD or later stage is authorized.
Research Program Fit
This is the clean state-level successor requested by the agentic breadth and posttraining programs after qwen35_4b_pareto_policy_integration disproved the assumed quick/deep local route without reaching MOPD.
Method
The frozen method compares both same-origin teachers and the strongest 40/60 soup student on exact failed student states. Four branches choose a strictly better teacher or abstain; four disjoint branches estimate selected-teacher advantage over both student and alternate. Both teachers must independently replicate before any dense update. Conditional training uses corrected top-k MOPD, a frozen-soup anchor, exact-logit locality, matched routing/off-policy controls, two confirmatory blocks, three seeds, visible routing, and best-of-8.
Evidence
The route study used 384 fresh soup states and 9,216 teacher/student continuations, plus soup-only acquisition rollouts: 12,726,694 sampled tokens total. It routed 111 states and abstained on 273. Support passed for quick (29/26 by block) and deep (22/34).
Deep's audit advantage over the student was +0.1216 and +0.0655 by block, with pooled one-sided 95% LCB +0.0657; its alternate-teacher contrast also passed. Quick beat deep and had pooled student LCB +0.0677, but its student-relative block means were +0.2009 and -0.0253. That negative replicate is terminal under the frozen rule. The combined router passed, but composition requires both named teachers to be independently useful.
The key result is methodological: pooling would have declared quick useful and authorized MOPD, while the independent-block sign gate exposed a nonreplicable local teacher. The correct response is to preserve the negative, not lower the bar or train on the favorable block.
Post-Gate Mechanism Diagnosis
Absolute continuation scores were not generally noisy: selection-to-audit correlations were 0.79--0.86 for all three policies. Conditioning on the largest of three four-branch estimates was the unstable step. Quick block 1 had an apparent selection advantage of +0.319 over the student but an audit state mean of -0.019; only 6/26 selected states remained strict quick winners on audit. Independent halves recovered the same quick route on 12/29 and 6/26 states across blocks.
A fixed positive margin does not solve that winner's curse. Retaining only quick block-1 states with observed selection margins of at least 0.10 or 0.25 left audit means of -0.0259 and -0.0089. The six-state 0.50 tail was positive but had only 1/6 strict audit winners and is not inferential.
The route was predominantly atom-level (101/288 atoms versus 10/96 episodes). Four fully reported posthoc cross-block grouping rules also failed to produce a credible replacement: the exact-cell rule was flat against the student in one direction, family and kind-level each lost a contrast, and family-kind's positive reverse result selected deep only. Details and the full machine-readable sensitivity are in route_diagnostics.md and analysis/route_diagnostics.json.
The independently regenerated 40/60 soup has weight hash 04610723…; all 128 mapped adapter deltas were nonzero. On eight fixed same-prompt canaries, every adapted arm changed from base, quick and deep differed 8/8, and soup differed from quick 8/8 and deep 7/8. This rules out an installation/no-op failure but is deliberately not scored as task evidence.
Oracle Versus Deployable Boundary
The verifier and three-policy branch comparison are training-only acquisition instruments. The primary deployable artifact must be one merged Qwen3.5-4B checkpoint. The visible two-checkpoint router and verifier-best sample-more are explicit baselines, not hidden components of the learned arm.
Interpretation Contract
The first terminal branch is route nonexistence for the required two-teacher composition claim. Locality, integration, controls, confirmation, and benchmarks are unreached—not negative and not inferred.
Best Next Test
Deep's replicated conditional advantage is the strongest surviving path. A new experiment should first requalify that frozen deep route on fresh states and test whether deep-only, verifier-backed MOPD can improve the existing soup without erasing its quick behavior. That is the shortest clean test of the still-untested update kernel.
Two-teacher composition should not reuse four-branch statewise argmax. It needs cross-fitted direct estimates of each teacher - student advantage, sequential branch allocation for uncertain states, a frozen predictor, and a third untouched qualification block. If quick cannot independently replicate under that design, it should be retired as a complementary teacher rather than rescued with an observed-margin threshold. Any eventual checkpoint must still beat both sources, the soup, visible routing, matched controls, and sample-more.
Artifact Manifest
Large source, soup, adapter, and merged checkpoints are external under large_artifacts/qwen35_4b_same_prefix_advantage_routing/ as specified in artifact_manifest.yaml. Small configs, receipts, raw score tables, analyses, and narrative remain in git.
Experiment log 8
Show the running log (8 entries, 2026-07-12)
2026-07-12 — intake and design
- Created a new experiment rather than extending the terminal Pareto qualification directory.
- Re-read the repository scorecards, C50-C54 evidence, model/vLLM playbooks, lifecycle rules, and the predecessor's harness, raw qualification receipt, preregistration, design review, and primary-paper map.
- Rechecked the 2026 primary literature. The design adopts MOPD's same-origin corrected top-k loss, SRPO's “do not densely distill already-correct samples” lesson, and the OPD failure literature's distribution/locality diagnostics.
- Froze the scientific correction: teacher advantage means verified continuation return on the identical student state, not a teacher's aggregate rank or hinted token log-probability. Four samples route and four disjoint samples audit; no arbitrary positive margin exists.
- Chose the independently regenerated 40% quick / 60% deep soup as the student so a pass must extend the strongest existing joint checkpoint, not merely recover from one weaker endpoint.
- No task-model output existed while the intake, config, preregistration, design review, literature review, and implementation plan were written.
2026-07-12 — upstream power correction incorporated before lock
- A 29-commit shared-main advance landed before this experiment created its immutable receipt. C54 pooled nine apex medium events and corrected the earlier favorable n=3
+0.345draw to+0.321 ± 0.017 SE; the tier router reaches the medium ceiling but does not decisively clear the old bar. - The teacher checkpoints and local scientific question are unchanged: no quick/deep rank is assumed and both must win on independent same-prefix audit branches. The visible tier router is now an even more direct baseline.
- The frozen downstream power rule now requires at least eight medium benchmark events. No task-model output had run, so this is a legitimate pre-lock correction rather than an outcome-dependent amendment.
2026-07-12 — immutable design lock
- Design commit:
668274493e0c150ed62fcca534ef99c3c06967ef. runs/preregistration_receipt.jsonbinds SHA-256 digests of the config, intake, preregistration, design review, and literature review. Every model stage verifies both those digests and commit ancestry before loading Qwen.
2026-07-12 — conditional implementation accepted before model output
- Implemented fresh training-state generation, selection-only online routing, equal 30/30 quick/deep capability quotas, 20 successful-soup anchors, exact target-top-50 caching, corrected sparse reverse KL, and four-round consume-once training.
- Implemented the five-update batch-of-one locality audit, three fixed primary seeds, shuffled/coarse/fixed-deep dense controls, best-continuation SFT, independent 25/50/75 parameter soups, the visible router, and verifier-best soup best-of-8 confirmation arm.
- Rejected two inherited implementation shortcuts before outcomes: shuffled routing now preserves atom/episode teacher quotas, and the short locality run uses an exact proportional 8/7/5 quick/deep/anchor mix rather than a random prefix of the full-round shuffle.
- Replaced the predecessor locality/confirmation analyzers because their old schemas and statistics did not implement this preregistration. The locality gate now uses the registered mean of per-row median centered non-target drift, and confirmation applies equal quick/deep paired macro deltas across both sealed blocks.
- All 48 unit/invariant tests, the 14-family gym selftest, AST checks, source artifact checks, and count-reachability smoke passed. No task-model output had been generated by this experiment at this point.
2026-07-12 — pinned model preflight
- The vLLM runner loaded only revision
851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a, resolved full decode CUDA graphs at every registered smoke size, matched four exact semantic outputs, and recorded exact prompt/token metadata. - The independent Transformers 5.13.0 + PEFT 0.19.1 + bitsandbytes 0.49.2 path exposed finite
(1, 1, 248320)logits on the NVIDIA L40 and confirmed both required Qwen3.5 fast-path extensions. This is engineering evidence, not task evidence. - The first soup construction stopped before writing weights because the two PEFT configs serialized the same target-module set in different list orders. The merge guard now normalizes that unordered field while retaining the stronger exact tensor-key/layout check; a regression test binds the fix.
2026-07-12 — source and soup installation gate
- Independently regenerated the registered 40% quick / 60% deep LoRA-delta soup. All 128 mapped modules had nonzero deltas; FP32 products used TF32 disabled; the explicit composite weight hash is
04610723f3f46d0a094ae0e5bc1a491bb6ad9e0fb6c8a84417dfe5e527f15b50. - Under identical greedy prompts and the frozen 16k vLLM geometry, quick, deep, and soup each changed all 8 base completions. Quick differed from deep on 8/8, soup from quick on 8/8, and soup from deep on 7/8.
- Every source/soup merge was nonzero and every local path/receipt hash matched. The canary authorized split-branch route qualification. These are installation/no-op checks, not capability measurements.
2026-07-12 — terminal split-branch route result
- Generated two independent 192-state blocks, each with 144 exact mid-thought atom states and 48 exactly replayed episode states. Across soup acquisition and all quick/deep/student branches, the run consumed 12,726,694 sampled tokens with no engine, replay, count, or provenance failure.
- The strict selection rule routed 111/384 states (28.9%): quick had support 29/26 across blocks and deep 22/34, above the required 16 everywhere.
- Deep passed. Its audit advantage over student was
+0.1216and+0.0655by block (pooled one-sided 95% LCB+0.0657); versus alternate it was+0.1599and+0.1210(LCB+0.1170). - Quick did not pass. It beat the alternate in both blocks and had a positive pooled bound, but selected-teacher-minus-student changed from
+0.2009in block 0 to-0.0253in block 1. The independent-block sign requirement caught the instability that a pooled-only rule would hide. - The combined router passed both contrasts, including
+0.2034and+0.0195versus student, but this cannot substitute for separately identifying both teachers. The experiment therefore stopped before teacher-logit caching, locality, MOPD, controls, confirmation, or benchmarks.
2026-07-12 — post-gate diagnosis and strategic close
- Added a descriptive analysis over the already generated branches; it does not alter the preregistered decision or authorize a later stage.
- Absolute policy values were reproducible across branch halves (Pearson
0.79--0.86, MAE0.093--0.117), but statewise three-way winner labels were not. Only 6/26 block-1 quick routes remained quick audit winners; the same-state quick-route Jaccard across halves was0.122. - Quick block 1 showed selection optimism of
+0.339versus the student. Its apparent selection margin was+0.319, while the audit state mean was-0.019. Posthoc+0.10and+0.25selection margins retained 24 and 22 states and remained negative, so reinstating an arbitrary effect-size floor is not a repair. - Unconditional teacher-minus-soup audit means changed sign across blocks for both teachers. Deep's passed result is genuinely conditional; neither source endpoint can simply replace the soup.
- All four posthoc cross-block grouping sensitivities were preserved. None constitutes a qualified two-teacher router: exact cells were flat in one direction, family and kind-level each failed a contrast, and family-kind had no reverse-direction quick support.
- Strategic close: test the validated deep route and MOPD kernel in a fresh deep-only experiment; require a cross-fitted direct-advantage predictor and a third untouched block before trying two-teacher composition again.
Data files 1
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
runs/smoke/summary.json2.7 kB
Reproduce
Smoke test
python3 experiments/qwen35_4b_same_prefix_advantage_routing/scripts/run.py --smokeFull run
Run model-smoke, build-student, route-qualify, then only the stages authorized by each machine-readable gate receipt.Run steps are documented inside the experiment folder (README and scripts).