Qwen3.5-4B OPSD Pressure Locality Audit
The one idea you need
Picture an open-book coding exam where the same reference page is handed to the grader checking each keystroke. The page lifts the grader's confidence only where the code copies its exact words, and stays silent at the one keystroke that separates a right answer from a plausible wrong one.
The question
Before training a small model to copy its own best code, does handing the grader a helpful reference sharpen its judgment right where correct and near-miss wrong code split apart?
What we found
No. At the exact spots where correct code diverges from code that passes surface tests but is secretly wrong, the reference hint adds essentially nothing — scoring no better than a scrambled, meaningless hint. The hint only boosts the grader where the code echoes its own wording, or where it leaks the full answer outright. Neither teaches real correctness, so the planned training run was cancelled before it burned any compute.
Why it matters
Before spending compute to train a model on its own best answers, verify the grader's edge lands on the real right-versus-wrong decisions, not surface word overlap. This cheap check, run before training, catches a hint that would teach mimicry.
On this page
Results at a glance 3
How to read
Bars show how much each hint shifts the grader toward the correct branch beyond giving no hint at all; the zero line means no added help, and taller is better. Left to right: the real reference hint at decision points, a scrambled fake hint, an answer-leaking hint, and forks where the hint's words appeared in the code.
Takeaway → The real-hint bar sits at zero, level with the scrambled fake; only the word-overlap bar towers up, so the hint helps by matching surface text, not by teaching correctness.
Data table
| hint condition / fork stratum | delta over student |
|---|---|
| weak hint, task-specific forks | -0.008 |
| shuffled hint, task-specific forks | -0.028 |
| full reference, task-specific forks | 0.213 |
| weak hint, hint-overlap forks | 1.36 |
Numbers from reports/report_summary.json (also final_report.md fork summary table)
Technical framing
Does the hint add correct-branch preference beyond the student? (delta over no-hint student, nats/token) — Gate FAIL: weak hints add ~zero signal on task-specific forks (like shuffled control); their big gain is only on hint-overlap forks.
How to read
Bars show how strongly each grader leans toward the correct branch over the wrong one at the decision points; taller means a more confident preference. Left to right: no hint at all, the real reference hint, a scrambled hint, and an answer-leaking hint.
Takeaway → All four bars stand at nearly the same height, meaning the grader already favored the correct branch on its own, so the reference hint changes almost nothing.
Data table
| scoring condition | mean preference for correct branch |
|---|---|
| no-hint student | 4.45 |
| weak retrieved hint | 4.44 |
| shuffled hint | 4.42 |
| full reference (leakage ceiling) | 4.66 |
Numbers from reports/report_summary.json (also final_report.md gate table)
Technical framing
Absolute correct-branch preference on task-specific forks — The student already prefers the correct branch by 4.45 nats/token; only the answer-leaking full-reference hint raises it.
How to read
Bars show how much extra weight the hinted grader adds to individual pieces of code; taller means more upweighting. Left to right: the real hint on correct pieces, the real hint on wrong pieces, a scrambled hint on correct pieces, and an answer-leaking hint on correct pieces.
Takeaway → The real-hint-on-correct-pieces bar is tallest, hinting at broad help, but that optimistic view collapses at the exact forks that actually separate right code from wrong.
Data table
| hint condition / token bucket | mean positive teacher-student gap |
|---|---|
| weak hint, correct tokens | 0.745 |
| weak hint, wrong tokens | 0.068 |
| shuffled hint, correct tokens | 0.05 |
| full reference, correct tokens | 0.544 |
Numbers from final_report.md token pressure buckets table
Technical framing
Rollout-level token pressure on discriminating tokens — Broad token view looks optimistic (weak hints upweight correct tokens), but the fork gate shows this is not localized at the correctness bits.
In the author’s words from the Report · “Interpretation”
This audit kills the immediate positive-only OPSD training run under the weak retrieved-hint setup. The hinted teacher is not useless: it strongly moves probability on hint-overlap forks and broad correct-rollout discriminating tokens. But it does not add incremental task-specific branch knowledge beyond what the base student already assigns. That is the near-fatal failure mode for OPSD here: dense credit exists, but it is not localized at the hidden-correct bits that distinguish correct code from visible-pass hidden-wrong near-misses. Full-reference hints do add task-specific signal, but that is a leakage ceiling. … Read the full result →
Overview
Standalone no-training audit for positive-only on-policy self-distillation.
The audit asks whether a privileged hinted teacher provides dense token-level signal at the exact places where hidden-correct code diverges from visible-pass hidden-wrong near-misses. It is a gate before any OPSD/OPD training.
Primary Gate
For matched hidden-correct and visible-pass hidden-wrong candidates on the same task, find same-prefix code forks. At each fork, score:
log p_teacher(correct_branch | shared_prefix, weak_hint) - log p_teacher(wrong_branch | shared_prefix, weak_hint)
The weak hint is the retrieved verified algorithm associated with the hidden-correct adaptation. Full reference-code hints are included only as a leakage ceiling, not as success evidence.
Supporting Analysis
The package also reports token-level student/teacher gaps for full correct and wrong rollouts:
gap_t = log p_teacher(token_t | prefix, hint) - log p_student(token_t | prefix)
Token buckets split shared boilerplate, discriminating correct chunks, discriminating wrong chunks, parse/format tokens, and other tokens. Discriminating forks are further stratified by whether their branch text overlaps with the retrieved hint.
Decision Rule
Run training only if weak-hint teacher preference is positive on task-specific forks, above shuffled-hint control, and not merely a full-reference leakage effect. If weak hints only upweight shared or hint-overlap tokens, this audit kills the OPSD training run.
Report
Rendered from reports/final_report.md
Motivation
This no-training audit tests whether positive-only on-policy self-distillation has the right token-localized signal before any adapter training is attempted. The target case is hidden-correct code versus visible-pass hidden-wrong near-misses for the same task.
The primary gate is same-prefix counterfactual branch preference: at executable code forks, does a weak hinted teacher prefer the hidden-correct branch over the hidden-wrong branch, and does that hint add preference beyond the no-hint student and shuffled-hint control?
Data
- Matched correct/wrong pairs: 14
- Tasks represented: [35, 44, 87]
- Executable code forks scored: 50
- Task-specific forks: 14
- Hint-overlap forks: 36
- Estimated scoring cost: 136060 forward tokens across 512 scored sequences.
Gate Result
Gate: FAIL
weak retrieved hint does not add task-specific correct-branch preference beyond student and shuffled control.
| statistic | value |
|---|---|
| weak task-specific absolute preference | 4.441 |
| weak task-specific delta over student | -0.008 |
| weak task-specific fraction prefers correct | 1.000 |
| shuffled task-specific absolute preference | 4.421 |
| shuffled task-specific delta over student | -0.028 |
| full-reference task-specific absolute preference | 4.662 |
| full-reference task-specific delta over student | 0.213 |
The important distinction is absolute preference versus incremental signal. The no-hint student already prefers the correct task-specific branches by 4.449 nats/token on average. The weak retrieved hint scores those branches at 4.441, which is a slight decrease (-0.008) rather than an added signal.
Fork Summary
| context / stratum | n | mean preference | mean student preference | delta over student | frac delta positive |
|---|---|---|---|---|---|
| weak / task-specific | 14 | 4.441 | 4.449 | -0.008 | 0.357 |
| weak / hint-overlap | 36 | 4.745 | 3.388 | 1.357 | 0.667 |
| shuffled / task-specific | 14 | 4.421 | 4.449 | -0.028 | 0.429 |
| full-reference / task-specific | 14 | 4.662 | 4.449 | 0.213 | 0.786 |
The weak hint does add large signal on hint-overlap forks: 1.357. That is exactly the retrieval-surface effect the audit was designed to catch. The effect does not transfer to task-specific forks.
Token Pressure Buckets
| bucket | mean positive gap | positive rate | n |
|---|---|---|---|
| weak correct discriminating | 0.745 | 0.749 | 491 |
| weak wrong discriminating | 0.068 | 0.376 | 596 |
| weak correct parse/format | 0.220 | 0.883 | 111 |
| weak wrong parse/format | 0.226 | 0.670 | 112 |
| shuffled correct discriminating | 0.050 | 0.470 | 491 |
| full-reference correct discriminating | 0.544 | 0.656 | 491 |
The rollout-level bucket view is more optimistic than the fork gate: weak hints give positive pressure to correct discriminating tokens overall. But the fork gate shows the crucial caveat: at task-specific same-prefix branches, the weak hint does not improve the student's preference. This means the broad token pressure is likely dominated by trajectory or retrieval-surface effects, not the local correctness bit needed for training.
Example Forks
Worst weak-hint task-specific deltas:
| task | correct branch | wrong branch | weak delta over student | weak preference | student preference |
|---|---|---|---|---|---|
| 44 | r'^\w+' | 'ab{3}?' | -0.180 | 5.753 | 5.933 |
| 44 | r'^\w+' | '\Bz\B' | -0.171 | 4.240 | 4.411 |
| 87 | ) | : if key in merged: merged[ | -0.145 | 3.491 | 3.636 |
Best weak-hint task-specific deltas:
| task | correct branch | wrong branch | weak delta over student | weak preference | student preference |
|---|---|---|---|---|---|
| 35 | n + | x): count = 0 i = | 0.142 | 2.606 | 2.465 |
| 87 | , | ) merged.update( | 0.225 | 5.618 | 5.394 |
| 87 | , | merged.update( | 0.491 | 7.891 | 7.400 |
Interpretation
This audit kills the immediate positive-only OPSD training run under the weak retrieved-hint setup.
The hinted teacher is not useless: it strongly moves probability on hint-overlap forks and broad correct-rollout discriminating tokens. But it does not add incremental task-specific branch knowledge beyond what the base student already assigns. That is the near-fatal failure mode for OPSD here: dense credit exists, but it is not localized at the hidden-correct bits that distinguish correct code from visible-pass hidden-wrong near-misses.
Full-reference hints do add task-specific signal, but that is a leakage ceiling. It does not justify deployable OPSD because the hint contains the answer and resembles gold/reference distillation rather than weak privileged guidance.
Decision
Do not proceed to Stage-2 OPSD training on this weak-hint formulation.
The next experiment should either:
- create stronger deployable evidence before distillation, such as independent retrieval-consensus or mined counterexample observations, then rerun this locality audit; or
- change the teacher hint so it contains task-specific discriminating evidence without leaking the reference solution.
Until the static locality gate passes, training would likely amplify retrieved surface form and shared structure rather than teach the missing correctness bits.
Artifacts
data/matched_pairs.jsonldata/fork_pressure_scores.jsonldata/token_pressure_scores.jsonlreports/pair_summary.jsonreports/pressure_summary.jsonreports/report_summary.jsonreports/figures/
Experiment log 1
Show the running log (1 entry, 2026-06-26)
2026-06-26
- Created standalone no-training OPSD pressure-locality audit package.
- Copied generated retrieval-adaptation candidate pools, retrieval plan, verified library, and generic evaluator/model utilities into the package.
- Localized the experiment identity to
qwen35_4b_opsd_pressure_locality_audit. - Pre-registered primary gate as same-prefix counterfactual teacher preference at code forks.
- Built 14 hidden-correct versus visible-pass hidden-wrong matched pairs across tasks 35, 44, and 87.
- Extracted 50 executable code forks: 14 task-specific and 36 hint-overlap.
- Ran one-pair model-scoring smoke test, then full scoring over 512 sequences.
- Corrected the gate to include delta over the no-hint student; absolute teacher preference alone was confounded because the no-hint student already strongly preferred the correct task-specific branch.
Final gate result: fail.
- Weak retrieved hint task-specific absolute preference: 4.441.
- No-hint student task-specific preference: 4.449.
- Weak retrieved hint delta over student: -0.008.
- Shuffled hint delta over student: -0.028.
- Full-reference leakage ceiling delta over student: +0.213.
- Weak retrieved hint-overlap delta over student: +1.357.
- Generated final report, machine-readable summary, and four figures under
reports/.
Figures 4
Data files 3
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
Reproduce
Runnable scripts exist in the experiment folder, but the exact invocation was not written down.