Research log Small Model Experimentation
GitHub

Qwen3.5-4B Adaptive Evidence Budget Policy

Finished2026-06-24imported · line ZActive Evidence AcquisitionBenchmark GeneralizationEvidence-Conditioned SelectionInterpretability and Diagnostics+4 moreGitHub ↗
The model learns when it has seen enough

The one idea you need

Think of a doctor diagnosing an illness by ordering tests one at a time. A tool already picks the single most informative next test. A separate trained judge does only one thing: name the diagnosis now, or order one more test.

The question

Can a small model learn when it has gathered enough evidence to commit to an answer, instead of always running every available check?

What we found

Yes. Trained just for this single decision, the model matched the accuracy of running all ten checks — solving about 92 of every 100 tasks — while using only about five checks on average, near a perfect-hindsight stopper's roughly four. Untrained, the same model quit almost immediately and solved just 5%. Crude fixed numeric-cutoff rules also collapsed, to between 5 and 7%.

Why it matters

For agent loops that gather evidence step by step, skip the hardwired step count and the crude confidence threshold. A cheap small model tuned only to say stop-or-continue can roughly halve the checks at no accuracy cost.

Same accuracy, far fewer checks10 → about 5 checksaverage checks the trained model needs to hit the top accuracy of running all ten
Same model, before vs after training5% → 92.5%share of tasks solved; untrained it quits instantly, trained it knows when to keep going
How close to a perfect stopper4.8 vs 4.2 checkschecks used by trained model vs a perfect-hindsight ideal, at identical accuracy
Cheaper fixed budgets fall short45% → 74% → 92.5%accuracy when forced to always stop at 3, 6, then 10 checks
On this page
  1. Results at a glance
  2. Overview
  3. Report
    1. Objective
    2. Data
    3. Key Result
    4. Overall
    5. By Template
    6. Interpretation
    7. Figures
    8. Reproduction
  4. Experiment log
  5. Figures
  6. Data files
  7. Reproduce
  8. Related

Results at a glance 3

Accuracy versus how many checks were spent

How to read

The horizontal axis is average checks used; the vertical axis is the share of tasks solved. Up and to the left is better. A line traces simple fixed cutoffs of three, six, and ten checks; two dots mark the trained model and a perfect-hindsight stopper.

40%60%80%100%46810fixed budget sweep (3/6/10)fixed budget swee…SFT stop/moreoracle stop

Takeaway → The trained model's dot sits above the fixed-cutoff line, reaching top accuracy at fewer checks, and nearly touches the perfect-hindsight dot.

Data table
Average probes usedfixed budget sweep (3/6/10)SFT stop/moreoracle stop
345%——
4.22——92.5%
4.81—92.5%—
674.4%——
1092.5%——

Numbers from experiments/qwen35_4b_adaptive_evidence_budget_policy/reports/summary_overall.csv

Technical framing

Accuracy vs probes spent: SFT stop/more matches full budget at half the cost — The SFT controller hits fixed-budget-10 accuracy (92.5%) with 4.81 probes, near the oracle's 4.22 — above the fixed-budget Pareto curve.

Share of tasks solved by each stopping rule

How to read

Each bar is a different stopping rule; bar height is the share of tasks solved, so taller is better. The untrained model and the crude cutoff rules sit low on the left; the trained model, the perfect-hindsight stopper, and the full ten-check budget stand tall on the right.

0%25%50%75%100%base stop/morebase stop/more5%threshold <=1000threshold <=10005%threshold <=100threshold <=1006.9%fixed budget 3fixed budget 345%fixed budget 6fixed budget 674.4%fixed budget 10fixed budget 1092.5%oracle stoporacle stop92.5%SFT stop/moreSFT stop/more92.5%

Takeaway → The untrained model's tiny bar leaps to a tall one after training, matching both the perfect-hindsight stopper and the full ten-check budget.

Data table
conditionaccuracy
base stop/more5%
threshold <=10005%
threshold <=1006.9%
fixed budget 345%
fixed budget 674.4%
fixed budget 1092.5%
oracle stop92.5%
SFT stop/more92.5%

Numbers from experiments/qwen35_4b_adaptive_evidence_budget_policy/reports/summary_overall.csv

Technical framing

Hidden-all accuracy by stopping policy — The base model stops almost immediately (5% accuracy); SFT lifts the same model to 92.5%, tying oracle stopping and the full 10-probe budget.

Checks spent on easy versus hard tasks

How to read

Two task types are shown, an easy one and a hard one; bar height is average checks used, so lower means less effort. Bars compare the trained model, the perfect-hindsight stopper, and a rule that always runs all ten checks.

02.557.510affine-modaffine-mod2.341.8210compare-gatecompare-gate7.286.6210

Takeaway → On easy tasks the trained model's bar is short; on hard tasks it is tall, closely tracking the perfect-hindsight stopper instead of the flat ten.

Data table
conditionSFT stop/moreoracle stopfixed budget 10
affine-mod2.341.8210
compare-gate7.286.6210

Numbers from experiments/qwen35_4b_adaptive_evidence_budget_policy/reports/summary_by_template.csv

Technical framing

Adaptive spend: probes used per task template — The SFT policy spends few probes on easy affine-mod tasks and more on hard compare-gate tasks, tracking the oracle while matching its accuracy.

In the author’s words from the Report · “Interpretation”

This test separates the value of the adaptive inference loop from the value of learning the stop rule. If the SFT policy lies on or above the fixed-budget Pareto curve, posttraining learned useful budget control. If fixed budgets dominate it, the practical lever is simply allowing more executable observations and using a transparent budget rule.

Overview

This standalone experiment tests whether Qwen3.5-4B can be posttrained as a STOP/MORE controller for an executable verifier.

The verifier chooses the next probe by target-independent expected split. The model does not choose probes and does not name operators. Its only deployable decision is whether to stop and commit the current verifier-selected program or request one more executable observation, up to a maximum budget of ten probes.

Large artifacts are intentionally outside this directory:

Main outputs:

  • reports/qwen35_4b_adaptive_evidence_budget_policy_report.md
  • reports/figures/
  • reports/*.csv
  • reports/eval/*.json
  • run_logs/
  • logs/experiment_log.md

Reproduction:

python scripts/build_dataset.py --train-per-cell 40 --eval-per-cell 20 --query-pool-cases 96 --max-budget 10
python scripts/eval_budget_policy.py --policy fixed --name fixed_budget3 --fixed-budget 3 --max-budget 10
python scripts/eval_budget_policy.py --policy fixed --name fixed_budget6 --fixed-budget 6 --max-budget 10
python scripts/eval_budget_policy.py --policy fixed --name fixed_budget10 --fixed-budget 10 --max-budget 10
python scripts/eval_budget_policy.py --policy threshold --name threshold_100 --threshold 100 --max-budget 10
python scripts/eval_budget_policy.py --policy threshold --name threshold_1000 --threshold 1000 --max-budget 10
python scripts/eval_budget_policy.py --policy oracle_stop --name oracle_stop --max-budget 10
python scripts/eval_budget_policy.py --policy base --name base_budget_policy --max-budget 10
python scripts/train_budget_sft.py --max-steps 220 --batch-size 2 --grad-accum 2
python scripts/eval_budget_policy.py --policy adapter --name sft_budget_policy --adapter-dir /workspace/large_artifacts/qwen35_4b_adaptive_evidence_budget_policy/models/budget_sft_lora --max-budget 10
python scripts/make_report.py

Report

Rendered from reports/qwen35_4b_adaptive_evidence_budget_policy_report.md

Objective

This standalone experiment tests whether Qwen3.5-4B can be posttrained as a STOP/MORE controller for an executable verifier. The verifier supplies the best deployable next probe by target-independent expected split; the model decides whether to commit now or spend another probe, up to a maximum budget of ten.

Data

  • Train records: 240.
  • Eval records: 160.
  • Train STOP/MORE states: 2640.
  • Eval STOP/MORE states: 1760.

Key Result

  • SFT STOP/MORE reached 92.5% accuracy using 4.81 probes on average.
  • Fixed budget 3/6/10 reached 45.0% / 74.4% / 92.5%.
  • Oracle stopping reached 92.5% using 4.22 probes on average.

Overall

PolicyHidden-all accuracyAvg probesCandidates leftHidden-equivalent left
Fixed budget 345.0%3.001040.512.9
Fixed budget 674.4%6.00544.78.8
Fixed budget 1092.5%10.00353.83.6
Threshold <=1006.9%1.86397.64.3
Threshold <=10005.0%0.94600.74.8
Oracle stop92.5%4.22356.23.8
Base Qwen stop/more5.0%0.043819.4129.6
SFT Qwen stop/more92.5%4.81353.83.6

By Template

Cells show accuracy / average probes.

PolicyAffine-modCompare-gate
Fixed budget 387.5% / 3.002.5% / 3.00
Fixed budget 698.8% / 6.0050.0% / 6.00
Fixed budget 1098.8% / 10.0086.2% / 10.00
Threshold <=10012.5% / 0.071.2% / 3.65
Threshold <=100010.0% / 0.000.0% / 1.88
Oracle stop98.8% / 1.8286.2% / 6.62
Base Qwen stop/more10.0% / 0.000.0% / 0.09
SFT Qwen stop/more98.8% / 2.3486.2% / 7.28

Interpretation

This test separates the value of the adaptive inference loop from the value of learning the stop rule. If the SFT policy lies on or above the fixed-budget Pareto curve, posttraining learned useful budget control. If fixed budgets dominate it, the practical lever is simply allowing more executable observations and using a transparent budget rule.

Figures

Reproduction

python scripts/build_dataset.py --train-per-cell 40 --eval-per-cell 20 --query-pool-cases 96 --max-budget 10
python scripts/eval_budget_policy.py --policy fixed --name fixed_budget3 --fixed-budget 3 --max-budget 10
python scripts/eval_budget_policy.py --policy fixed --name fixed_budget6 --fixed-budget 6 --max-budget 10
python scripts/eval_budget_policy.py --policy fixed --name fixed_budget10 --fixed-budget 10 --max-budget 10
python scripts/eval_budget_policy.py --policy threshold --name threshold_100 --threshold 100 --max-budget 10
python scripts/eval_budget_policy.py --policy threshold --name threshold_1000 --threshold 1000 --max-budget 10
python scripts/eval_budget_policy.py --policy oracle_stop --name oracle_stop --max-budget 10
python scripts/eval_budget_policy.py --policy base --name base_budget_policy --max-budget 10
python scripts/train_budget_sft.py --max-steps 220 --batch-size 2 --grad-accum 2
python scripts/eval_budget_policy.py --policy adapter --name sft_budget_policy --adapter-dir /workspace/large_artifacts/qwen35_4b_adaptive_evidence_budget_policy/models/budget_sft_lora --max-budget 10
python scripts/make_report.py

Experiment log 5

Show the running log (5 entries)

Setup

Mechanism

  • The verifier chooses the next probe by target-independent expected split over the full probe pool.
  • The model sees observed executions, current survivor count, current deterministic selected program, and the expected split statistics for the best next probe.

  • The model outputs:

    • A: STOP and commit the current selected program.
    • B: MORE and request one additional executable probe.
  • Training labels are oracle-supervised from hidden checks: STOP once the current selected program is hidden-correct, otherwise MORE until the maximum budget.

Planned Arms

  • fixed_budget3
  • fixed_budget6
  • fixed_budget10
  • threshold_100
  • threshold_1000
  • oracle_stop
  • base_budget_policy
  • sft_budget_policy

Success Criterion

The SFT policy should land on or above the fixed-budget accuracy/probe Pareto curve. If fixed budgets dominate it, the useful mechanism is the inference loop with more probes, not learned stop control.

Results

  • Built 240 train records, 160 eval records, 2,640 train STOP/MORE states, and 1,760 eval STOP/MORE states.
  • Trained Qwen3.5-4B QLoRA STOP/MORE adapter for 220 optimizer steps.

  • Overall hidden-all accuracy / average probes:

    • fixed_budget3: 45.0% / 3.00
    • fixed_budget6: 74.4% / 6.00
    • fixed_budget10: 92.5% / 10.00
    • threshold_100: 6.9% / 1.86
    • threshold_1000: 5.0% / 0.94
    • oracle_stop: 92.5% / 4.22
    • base_budget_policy: 5.0% / 0.04
    • sft_budget_policy: 92.5% / 4.81

  • By template, sft_budget_policy reached:

    • pair_affine_mod: 98.8% / 2.34 probes
    • pair_compare_gate: 86.2% / 7.28 probes
  • Interpretation: Qwen posttraining learned a useful budget controller. It matched the accuracy of always spending ten probes while using less than half the probes on average, and it landed close to the hidden-oracle stopping efficiency.

Figures 4

accuracy by template
accuracy by template · reports/figures/
accuracy vs probes
accuracy vs probes · reports/figures/
budget sft loss
budget sft loss · reports/figures/
probes by template
probes by template · reports/figures/

Data files 3

Result tables and metrics copied from the experiment folder — preview inline or open the raw file.

Reproduce

Run steps are documented inside the experiment folder (README and scripts).

Browse the experiment folder on GitHub ↗