Qwen3.5-4B Verifier-Guided Self-Improvement Report
The one idea you need
Picture a coder who studies only their own past solutions that worked. They grow fluent at familiar patterns but stop exploring alternatives, so their several fresh attempts start to look alike. On a new problem where the usual move fails, they have no backup.
The question
If a small coding model retrains only on the solutions it wrote that actually passed the tests, does it get better at solving brand-new problems it has not seen?
What we found
No. After a four-billion-parameter model retrained on its own test-passing code, its success on unseen problems barely moved, going from about 65% to about 65% (a hair lower). Retraining made its several attempts look more alike, shrinking variety. The simplest lever won instead: just generating more attempts at solving time reached about 69%, beating every retraining approach.
Why it matters
For small coding models, spend compute on generating more diverse attempts and filtering them by actually running the code, not on fine-tuning the model on its own successes, which quietly collapses the variety that makes multiple tries pay off.
On this page
Results at a glance 5
How to read
Each bar is one approach; height is the share of unseen problems where at least one of several attempts passes the hidden tests, so taller is better. Bars are the frozen model, retrain-on-verified-wins, retrain-on-unfiltered-code, retrain-on-reference-answers, and simply sampling more attempts.
Takeaway → The sample-more bar is the tallest, standing above every retraining bar, so extra attempts beat teaching the model on its own work.
Data table
| arm | coverage (pass@K) |
|---|---|
| base (frozen) | 65.3% |
| verified self-train | 64.7% |
| unverified self-train | 62% |
| oracle SFT | 63.3% |
| sample more (matched compute) | 68.7% |
Numbers from reports/summary_coverage.csv
Technical framing
Held-out MBPP coverage: self-training arms vs just sampling more — One round of verified self-training did not beat the frozen model on held-out tasks; extra inference sampling did (+3.3 pp).
How to read
Two bars per group, frozen model versus retrained model, across three sets: practiced problems, unseen problems, and a different problem set; taller is better. Height is the share solved by at least one attempt.
Takeaway → The retrained bar edges up only on practiced problems and sits slightly lower on both unseen sets, so the small gain does not carry over.
Data table
| evaluation split | frozen base | verified self-trained (round 1) |
|---|---|---|
| MBPP train (80) | 70% | 71.2% |
| MBPP held-out (150) | 65.3% | 64.7% |
| HumanEval transfer (150) | 75.3% | 74.7% |
Numbers from reports/summary_coverage.csv
Technical framing
Verified self-training gains on train tasks but not on held-out or transfer — The +1.3 pp gain on the 80 training tasks does not generalize: held-out and transfer coverage each drop 0.7 pp.
How to read
Each bar is one approach; height is the average number of candidate solutions the model produces per unseen problem, and more candidates give more chances to land a working one. Rightmost is the sample-more approach.
Takeaway → The two self-training bars sit below the frozen model's, so retraining on its own code shrinks the pool of attempts, while sampling more lifts it far higher.
Data table
| arm | mean candidates per task |
|---|---|
| base (frozen) | 3.71 |
| verified self-train | 3.4 |
| unverified self-train | 3.01 |
| oracle SFT | 3.83 |
| sample more (matched compute) | 7.01 |
Numbers from reports/summary_coverage.csv
Technical framing
Self-training shrinks the candidate pool on held-out tasks — Self-trained models emit fewer candidates per task, narrowing the pool that coverage is harvested from.
How to read
Two bars per group, frozen versus retrained model, with one group per rule for picking the single answer to submit; height is final accuracy on unseen problems, so taller is better. Rules range from first-that-runs to a best-possible pick.
Takeaway → In every group the frozen model's bar is at least as tall, so switching the answer-picking rule cannot rescue the retrained model.
Data table
| commit-selection policy | frozen base | verified self-trained |
|---|---|---|
| first visible pass | 63.3% | 61.3% |
| public-signature majority | 63.3% | 61.3% |
| base-model verifier | 64.7% | 63.3% |
| oracle upper bound | 65.3% | 64.7% |
Numbers from report table (reports/qwen35_4b_verifier_guided_self_improvement_report.md, Commit Selection)
Technical framing
Committed accuracy at budget 5: selection policy matters less than the pool — Across every selection policy the self-trained model commits worse answers than the frozen base; the pool, not selection, is binding.
How to read
Two bars per group, frozen versus retrained model, comparing a small 20-problem preview against the full 150-problem test; height is the share solved, so taller is better.
Takeaway → The retrained bar leads on the small preview but falls slightly behind on the full test, showing the early promise was just noise.
Data table
| Held-out evaluation size | Base model | Verified self-train |
|---|---|---|
| Smoke eval (20 tasks) | 65% | 70% |
| Full eval (150 tasks) | 65.3% | 64.7% |
Numbers from experiments/qwen35_4b_verifier_guided_self_improvement/reports/summary_coverage.csv
Technical framing
A 20-task smoke test flipped sign at full scale — The 20-task pilot showed a +5 pp gain for verified self-training that vanished into a slight regression on the full 150-task held-out set.
In the author’s words from the Overview · “Summary”
The 20-task smoke signal was positive, but the 150-task held-out run regressed slightly.
Overview
This top-level README was generated during repository normalization because the imported experiment did not include one.
- Source track:
track-z - Primary report: reports/qwen35_4b_verifier_guided_self_improvement_report.md
- Metadata: metadata.yaml
How To Read
Start with the primary report, then inspect data/, reports/, analysis/, src/, and scripts/ as available. This folder remains self-contained; do not move its run data into shared directories.
Summary
The main result is negative for the central question. Verified self-training did not raise held-out generation coverage under this local LoRA/data budget. The 20-task smoke signal was positive, but the 150-task held-out run regressed slightly.
Report
Rendered from reports/qwen35_4b_verifier_guided_self_improvement_report.md
Date: 2026-06-25
Executive Read
The main result is negative for the central question. Verified self-training did not raise held-out generation coverage under this local LoRA/data budget. The 20-task smoke signal was positive, but the 150-task held-out run regressed slightly.
- MBPP held-out: 65.3% -> 64.7% (-0.7 pp).
- HumanEval transfer: 75.3% -> 74.7% (-0.7 pp).
- MBPP train: 70.0% -> 71.2% (+1.3 pp).
Rounds 2 and 3 were intentionally stopped at the pre-registered gate because held-out coverage did not move in the right direction after round 1.
The controls sharpen the read:
- Unverified self-training is worse than verified self-training on MBPP held-out, so the execution filter is load-bearing.
- Oracle/reference SFT on the same 80 train tasks also does not beat base on MBPP held-out, so the failure is not only noisy self-generated labels.
- More inference sampling beats the training arms on MBPP held-out: 65.3% -> 68.7%.
Coverage
| Split | Arm | n | Coverage | Visible cov. | Candidates | Distinct |
|---|---|---|---|---|---|---|
| train | base | 80 | 70.0% | 70.0% | 3.74 | 96.2% |
| train | selftrain_verified_r1 | 80 | 71.2% | 71.2% | 2.83 | 98.4% |
| heldout | base | 150 | 65.3% | 65.3% | 3.71 | 97.3% |
| heldout | selftrain_verified_r1 | 150 | 64.7% | 64.7% | 3.40 | 97.9% |
| heldout | selftrain_unverified | 150 | 62.0% | 62.0% | 3.01 | 98.3% |
| heldout | oracle_sft | 150 | 63.3% | 63.3% | 3.83 | 99.8% |
| heldout | sample_more_matched_compute | 150 | 68.7% | 68.7% | 7.01 | 97.5% |
| transfer | base | 150 | 75.3% | 75.3% | 3.08 | 91.7% |
| transfer | selftrain_verified_r1 | 150 | 74.7% | 74.7% | 2.77 | 93.9% |
Primary metric: coverage is pass@K for the sampled pool, meaning at least one candidate passes hidden tests. Hidden tests were not used for self-training selection.
Commit Selection
| Arm | Policy | Budget | Selected | Coverage captured |
|---|---|---|---|---|
| base | first_visible | 5 | 63.3% | 96.9% |
| base | public_signature_majority | 5 | 63.3% | 96.9% |
| base | base_verifier | 5 | 64.7% | 99.0% |
| base | oracle_coverage | 5 | 65.3% | 100.0% |
| selftrain_verified_r1 | first_visible | 5 | 61.3% | 94.8% |
| selftrain_verified_r1 | public_signature_majority | 5 | 61.3% | 94.8% |
| selftrain_verified_r1 | base_verifier | 5 | 63.3% | 97.9% |
| selftrain_verified_r1 | oracle_coverage | 5 | 64.7% | 100.0% |
| selftrain_unverified | first_visible | 5 | 60.0% | 96.8% |
| selftrain_unverified | public_signature_majority | 5 | 60.0% | 96.8% |
| selftrain_unverified | base_verifier | 5 | 62.0% | 100.0% |
| selftrain_unverified | oracle_coverage | 5 | 62.0% | 100.0% |
| oracle_sft | first_visible | 5 | 60.7% | 95.8% |
| oracle_sft | public_signature_majority | 5 | 60.7% | 95.8% |
| oracle_sft | base_verifier | 5 | 61.3% | 96.8% |
| oracle_sft | oracle_coverage | 5 | 63.3% | 100.0% |
| sample_more_matched_compute | first_visible | 10 | 66.7% | 97.1% |
| sample_more_matched_compute | public_signature_majority | 10 | 66.7% | 97.1% |
| sample_more_matched_compute | base_verifier | 10 | 66.7% | 97.1% |
| sample_more_matched_compute | oracle_coverage | 10 | 68.7% | 100.0% |
Selection remains secondary here because coverage is the binding quantity. oracle_coverage is the diagnostic upper bound: if a hidden-correct candidate exists in the pool, it commits one.
Figures
Interpretation
This run does not support the hypothesis that one round of verified rejection-sampling SFT expands Qwen3.5-4B's coding frontier on held-out tasks. It mostly narrows the pool: candidate count, visible-passers, and transfer coverage all decrease slightly after verified SFT. The best current deployable lever in this package is not small-SFT self-improvement; it is preserving or increasing sample diversity and then using execution/selection to harvest coverage.
A stronger future positive would need to change at least one of these constraints: substantially more train tasks, stronger multi-round data accumulation without diversity collapse, curriculumed repair data for tasks with zero initial coverage, or a generator objective that explicitly preserves pass@K diversity rather than only imitating passing samples.
Artifacts
- Experiment package:
/workspace/experiments/qwen35_4b_verifier_guided_self_improvement - Large adapters/checkpoints:
/workspace/large_artifacts/qwen35_4b_verifier_guided_self_improvement - Coverage CSV:
reports/summary_coverage.csv - Commit CSV:
reports/summary_commit.csv
Experiment log 7
Show the running log (7 entries)
Date: 2026-06-25
Experiment directory: /workspace/experiments/qwen35_4b_verifier_guided_self_improvement
Large artifacts directory: /workspace/large_artifacts/qwen35_4b_verifier_guided_self_improvement
Objective
Test whether execution-verified self-training raises generation coverage on held-out coding tasks. The measurement is coverage x selection-capture: coverage asks whether any sampled candidate passes hidden tests, while selection-capture asks whether a deployable selector commits a hidden-correct candidate when one exists.
Protocol
The loop is:
- Sample candidates from Qwen3.5-4B.
- Execute candidates against visible tests.
- Fine-tune the generator on visible-test-passing candidates.
- Re-sample with the updated generator.
- Evaluate coverage, deployable pass@1, selection capture, and diversity on train, held-out, and transfer splits.
Hidden tests are used only for evaluation and oracle diagnostic ceilings, not for selecting self-training examples.
Initial Plan
- Smoke gate: run a small sample on 10 train tasks, 20 held-out tasks, and 20 transfer tasks to verify execution, parsing, adapter loading, and SFT.
- Main gate: run MBPP train self-improvement and evaluate on at least 150 held-out MBPP test tasks plus 150 HumanEval transfer tasks if the smoke results and compute budget are viable.
- Controls: unverified self-training, oracle/reference SFT, and more-sampling matched-compute baseline.
Running Notes
- Created standalone experiment directory and separate large artifact directory.
- Added package-local Qwen loading and Python execution utilities.
- Added
sample_round.pyfor round-specific sampling with optional generator LoRA adapter. - Added
build_generator_sft.pyfor verified, unverified, and oracle/reference SFT datasets. - Added
train_generator_sft.pyfor QLoRA generator fine-tuning.
Smoke Gate
Smoke settings:
- MBPP train: 10 tasks.
- MBPP held-out: 20 tasks.
- Direct samples/task: 3.
- Repair attempts/task: 1.
- Max new tokens: 180.
- Verified-SFT steps: 30.
Smoke train sampling completed:
- Mean candidates/task: 3.2.
- Hidden coverage: 50.0%.
- Visible coverage: 50.0%.
- Mean visible-passers/task: 1.7.
The visible-test filter produced 12 smoke SFT examples from 10 tasks; 10 of those also passed hidden tests.
Smoke verified-SFT adapter trained successfully and could be loaded for generation.
Smoke held-out comparison on 20 MBPP test tasks:
| Arm | Coverage | Visible coverage | Mean candidates | Mean visible-passers |
|---|---|---|---|---|
| base | 65.0% | 65.0% | 2.95 | 1.80 |
| selftrain_verified_r1_smoke | 70.0% | 70.0% | 2.65 | 1.55 |
Decision: proceed to a larger run. To keep the run tractable while preserving the load-bearing held-out size, the main config is tightened to 80 MBPP train tasks, 150 MBPP held-out tasks, 150 HumanEval transfer tasks, 4 direct samples/task, 1 repair attempt/task, and 80 generator-SFT steps.
Main Run Ledger
Main base sampling on 80 MBPP train tasks completed:
- Mean candidates/task: 3.74.
- Hidden coverage: 70.0%.
- Visible coverage: 70.0%.
- Mean visible-passers/task: 2.08.
SFT dataset construction from the base train pool:
| Dataset | Examples | Hidden-positive examples | Visible-positive examples |
|---|---|---|---|
| verified | 115 | 105 | 115 |
| unverified | 158 | 95 | 103 |
| oracle/reference | 80 | 80 | 80 |
The selftrain_verified_r1 adapter was trained for 80 steps on the verified examples.
Main held-out and transfer sampling results:
| Split | Arm | Coverage | Visible coverage | Mean candidates | Mean visible-passers |
|---|---|---|---|---|---|
| MBPP train | base | 70.0% | 70.0% | 3.74 | 2.08 |
| MBPP train | selftrain_verified_r1 | 71.2% | 71.2% | 2.83 | 1.76 |
| MBPP held-out | base | 65.3% | 65.3% | 3.71 | 1.98 |
| MBPP held-out | selftrain_verified_r1 | 64.7% | 64.7% | 3.40 | 1.71 |
| HumanEval transfer | base | 75.3% | 75.3% | 3.08 | 2.54 |
| HumanEval transfer | selftrain_verified_r1 | 74.7% | 74.7% | 2.77 | 2.35 |
Decision after round 1: do not run rounds 2 and 3. The pre-registered gate required held-out coverage to move in the right direction; the 150-task held-out readout regressed slightly despite a small train increase. The smoke +5 point signal did not replicate at larger n.
The unverified self-training control was trained for 80 steps on the unfiltered parsed/safe sample set. Held-out MBPP sampling completed:
- Hidden coverage: 62.0%.
- Visible coverage: 62.0%.
- Mean candidates/task: 3.01.
- Mean visible-passers/task: 1.63.
The oracle/reference SFT control was trained for 80 steps on reference solutions for the same 80 MBPP train tasks. Held-out MBPP sampling completed:
- Hidden coverage: 63.3%.
- Visible coverage: 63.3%.
- Mean candidates/task: 3.83.
- Mean visible-passers/task: 2.00.
Interim interpretation: verification filtering is load-bearing relative to unverified self-training, but under this data and LoRA budget it does not beat the base generator on held-out MBPP or HumanEval transfer. Oracle/reference SFT also does not beat base on held-out MBPP, which points to a broader small-SFT/generalization limit rather than only noisy self-generated labels.
The matched-compute sample-more baseline used the base generator with 8 direct samples/task and 2 repair attempts/task on the same 150 MBPP held-out tasks. It completed in 59:56 wall time.
- Hidden coverage: 68.7%.
- Visible coverage: 68.7%.
- Mean candidates/task: 7.01.
- Mean visible-passers/task: 3.60.
This baseline beat base coverage (65.3%) and all fine-tuned generator arms, but did so at roughly double the sampled candidate count and a much higher execution/runtime cost. This is the strongest practical result: preserving and increasing sampling diversity is a better use of the available compute than the tested 80-step SFT loops.
Final Artifacts and Audit
Final report generated:
reports/qwen35_4b_verifier_guided_self_improvement_report.mdreports/summary_coverage.csvreports/summary_commit.csvreports/figures/coverage_by_arm.pngreports/figures/diversity_and_pool_size.pngreports/figures/commit_accuracy_heldout.pngreports/figures/training_losses.png
Audit results:
- All package scripts compile with
python -m py_compile src/*.py scripts/*.py. - The experiment package contains no files larger than 10 MB.
- The experiment package is 18 MB.
- Large LoRA/tokenizer artifacts are isolated under
/workspace/large_artifacts/qwen35_4b_verifier_guided_self_improvement(401 MB). - The report, log, config, source, and scripts are standalone and do not refer to earlier experiments.
Figures 4
Data files 2
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
Reproduce
Runnable scripts exist in the experiment folder, but the exact invocation was not written down.