Qwen3.5-4B: The Compute-Optimal Confidence Policy (select + abstain + escalate)
The one idea you need
Picture the model answering a coding problem several times. A cheap read of its own confidence (one logit: 'is this right?') picks the best try and flags the ones it is unsure about. A tempting extra step is to let the model think LONGER on the flagged-hard problems instead of drawing more tries. This experiment tests all three moves at matched compute — and powers up the shakiest one until the answer is trustworthy.
The question
Can the model's own confidence be turned into one deployable policy — pick the best answer, abstain when unsure, and spend extra compute wisely — with no test-execution or grader?
What we found
Partly. Picking the answer with the highest single-token P(True) confidence is the best grader-free selector at every compute level, and abstaining below a confidence threshold trades coverage for accuracy cleanly — both robust. But the appealing third move — spending compute on more THINKING for the flagged-hard tasks rather than more samples — showed no benefit once powered up to 400 tasks: an early small-sample win did not replicate (every 95% interval now includes zero). Thinking longer helps a little across the board, just not selectively on the hard tail — because these coding tasks nearly saturate the model's thinking budget already.
Why it matters
It gives a shippable, verifier-free recipe (select + abstain from one logit) AND a methodological lesson: a pre-registered, under-powered subset win was flagged as shaky and then REVERSED on the larger sample — power up before you claim. No grader, no bigger model, just the fixed 4B's own logits.
Results at a glance 3
How to read
Bars are MBPP accuracy at nine candidates for each selection rule. Single-token P(True)-select leads the verifier-free rules; the execution line and oracle need a grader.
Data table
| selection rule | MBPP accuracy at 9 candidates |
|---|---|
| greedy | 0.701 |
| majority-vote | 0.742 |
| mean-logprob | 0.725 |
| P(True)-select | 0.762 |
| execution-line | 0.84 |
| oracle | 0.848 |
Numbers from experiments/qwen35_4b_confidence_policy/runs/frontier.json
Technical framing
P(True) confidence-select is the best verifier-free selector (MBPP, k=9) — greedy/majority/mean-logprob/P(True) are verifier-free; execution-line and oracle require a grader.
How to read
Each group is the hardest X% of tasks by confidence. Escalating them to a bigger think budget (left) barely differs from spending the same compute on more samples (right) — every 95% interval on the difference includes zero.
Data table
| abstained (low-confidence) task subset | escalate to budget 2048 | more samples @ 256 (matched compute) |
|---|---|---|
| hardest 20% | 0.289 | 0.268 |
| hardest 30% | 0.306 | 0.302 |
| hardest 40% | 0.357 | 0.34 |
| hardest 50% | 0.406 | 0.4 |
Numbers from experiments/qwen35_4b_confidence_policy/runs/escalation.json
Technical framing
Escalate vs breadth on the abstained tail: null at n=400 (matched compute) — Powered up to n=80-200 per subset; every esc-minus-breadth 95% bootstrap CI spans 0.
conf-select minus majority · task difficulty (per-task pass rate) →
Data table
| task difficulty (per-task pass rate) | P(True)-select minus majority-vote accuracy (k=6) |
|---|---|
| hard (pass 0.08) | 0.045 |
| medium (pass 0.54) | 0.038 |
| easy (pass 0.97) | -0.007 |
Numbers from experiments/qwen35_4b_confidence_policy/runs/difficulty_curve.json
In the author’s words from the Overview · “Result”
This reaches full-pool MBPP accuracy (0.762) at ~4.25 average samples vs 9 for uniform sampling — a ~2× compute saving, strictly beating uniform at 7/9 operating points (and 7/9 on HumanEval too). One logit drives selection, abstention, and allocation. (scripts/adaptive_compute.py) Why it works — the difficulty curve. Pooling MBPP+HumanEval (312 tasks) and binning by per-task pass rate, the confidence-select edge over majority-vote concentrates on hard tasks: hard (pass 0.08) +0.045, medium (0.54) +0.038, easy (0.97) −0.007. Selection pays exactly where the model is uncertain — the same tasks abstention flags — so a confidence-gated allocator is self-consistent. … Read the full result →
Overview
The confidence arc established three levers in isolation: confidence-select beats majority vote (C41), the single-token P(True) judge beats sequence mean-logprob (C46), and answer-token probability predicts correctness for abstention (C40). None were ever fused into one deployable decision policy, and none were plotted against sample-more at equal compute — the capstone C41/C46 explicitly owed.
This experiment builds that policy on the fixed Qwen3.5-4B and asks: for a low-confidence (abstained) task, is compute better spent on more breadth (more samples) or more serial depth (a higher think budget on the same model)?
The registered capstone is complete. The deferred agentic-domain arbitration is an unexecuted follow-up direction, not unfinished work in this result-bearing experiment.
What it does
Part 1 — select + abstain frontier (post-hoc, zero new inference). scripts/frontier.py reuses the cached 244-task MBPP candidate pool from qwen35_4b_code_confidence (9 candidates/task, each with full_pass ground truth, p_true, mean_logprob, behavior_signature). It sweeps the compute level k and compares selection policies at matched compute, plus the conf+abstain risk-coverage curve.
Part 2 — the escalation arm (new think-mode generation). ../qwen35_4b_code_confidence/scripts/gen_budget.py generates think-mode MBPP candidate pools at two think budgets (256, tight; 2048, generous) over the same 120 tasks, each candidate scored full_pass (hidden-assert execution), p_true (no-think C10 judge), with n_think recorded for token-matched compute accounting. scripts/escalation.py then compares, at matched token-compute and with verifier-free confidence-select, whether escalating the abstained tasks to budget 2048 beats spending the same compute on more breadth at budget 256.
Result
The capstone — compute-optimal = confidence-gated adaptive ALLOCATION. The deployable answer to "compute-optimal confidence policy" is to spend samples only where they help: sample greedily once, read its single-token P(True); if high, commit (1 sample); if low, sample K and conf-select; abstain below a floor. This reaches full-pool MBPP accuracy (0.762) at ~4.25 average samples vs 9 for uniform sampling — a ~2× compute saving, strictly beating uniform at 7/9 operating points (and 7/9 on HumanEval too). One logit drives selection, abstention, and allocation. (scripts/adaptive_compute.py)
Why it works — the difficulty curve. Pooling MBPP+HumanEval (312 tasks) and binning by per-task pass rate, the confidence-select edge over majority-vote concentrates on hard tasks: hard (pass 0.08) +0.045, medium (0.54) +0.038, easy (0.97) −0.007. Selection pays exactly where the model is uncertain — the same tasks abstention flags — so a confidence-gated allocator is self-consistent. (scripts/difficulty_curve.py)
The verifier-free pieces that hold:
- Confidence-select — argmax single-token P(True) is the best verifier-free selector on moderate-difficulty MBPP (k=9: 0.762 > majority 0.742 > logprob 0.725; per-cand AUROC 0.77), but only difficulty-dependently: it ties majority on easy HumanEval (base pass 0.91, both 0.941).
- Abstention — max-P(True) gives a clean risk-coverage curve (68% coverage → 0.866; solvability AUROC 0.72).
The null (honest correction). The tempting escalate step — spend the extra compute on more think budget (depth) for the flagged-hard tasks — was a small-sample win (n=24: 0.458 vs 0.304) that did NOT replicate at n=400 (esc−breadth +0.004…+0.022, every 95% bootstrap CI spanning 0). MBPP nearly saturates the think budget (median ~86 think tokens at both budget 256 and 2048); the budget-bound tail is capability-hard, not merely compute-starved. So the compute lever is breadth allocation, not depth escalation.
Codified as C57. Part 2 was regenerated on vLLM (~10× faster than the initial HF pass) at n=400 with bootstrap CIs.
Layout (all analyses CPU-only, post-hoc)
scripts/adaptive_compute.py— the capstone: confidence-gated adaptive allocation vs uniform sampling (--src mbpp|humaneval).scripts/difficulty_curve.py— conf-select vs majority by task difficulty.scripts/frontier.py— selection frontier + abstention risk-coverage (--src mbpp|humaneval).scripts/escalation.py— escalation(depth)-vs-breadth with bootstrap CIs.../qwen35_4b_code_confidence/scripts/vllm_gen_budget.py— fast vLLM think-mode multi-budget generation (GPU);gen_budget.pyis the older HF one.runs/*.json— aggregate results (adaptive_compute, difficulty_curve, frontier, escalation) and the two vLLM candidate pools (per-candidate fields only, no menagerie contents).
Experiment log 8
Show the running log (8 entries, 2026-07-13 → 14)
2026-07-13 — CORRECTION: escalation null at n=400 (backend switch HF->vLLM + power-up)
Per C57's own next-test #1 (the n=24-60 subset was flagged under-powered), powered up the escalation arm and it REVERSED. Two changes: (a) switched generation from the HF backend to vLLM (~10x faster; user prompt "why HF when vLLM is right there"); (b) fixed a vLLM judge bug — after "Answer: " the 4B emits the SPACE-PREFIXED " A"/" B" (ids 357/417), not the bare "A"/"B" (32/33), so reading 32/33 gave a degenerate p_true=0.5. The vLLM p_true is then a strong signal (per-cand AUROC 0.756-0.769; solvability AUROC 0.715-0.727, matching the HF pool).
Regenerated the full pools on vLLM at n=400 x k=6, budgets 256 vs 2048.
- SURVIVES: pure-2048 modestly > pure-256 on the accuracy-vs-tokens frontier (high-k 0.593 vs 0.581).
- REVERSED: selective escalation of the abstained tail vs matched-compute breadth is NULL: esc-minus-breadth +0.022/+0.004/+0.017/+0.006 at abstain 20/30/40/50%, every 95% bootstrap CI spanning 0 (n=80/120/160/200). The first-pass HF n=24-60 win (hardest-20% 0.458 vs 0.304) did NOT replicate.
- LIKELY MECHANISM: MBPP nearly SATURATES the think budget (the 4B self-limits to ~108-172 think tokens even at budget 2048), so the serial-compute lever is too weak to differentiate. Whether escalation helps on a genuinely budget-BOUND family is the owed test.
- LESSON: power up abstained-subset effects before claiming; a pre-registered under-powered win was correctly flagged and then reversed.
Robust deployable core: SELECT (argmax P(True)) + ABSTAIN (max-P(True) threshold), both verifier-free from one logit. C57 corrected.
2026-07-13 — GENERALIZATION: the conf-select advantage is difficulty-dependent (HumanEval ties majority)
Ran the identical Part-1 frontier on the cached HumanEval pool (68 tasks x 9 cands, same schema; frontier.py --src humaneval). HumanEval is EASY for the 4B (base pass 0.91, 67/68 solvable). There the confidence-SELECT advantage over majority-vote VANISHES: k=9 conf 0.941 == majority 0.941 (majority slightly ahead at k=7/8). On MBPP (base pass ~0.53) conf-select clearly beat majority (0.762 vs 0.742). So single-token P(True) selection beats self-consistency only when the task is hard enough that selection matters; when the model is already ~0.9 accurate, majority-vote catches up. Abstention still works (HumanEval risk-coverage clean, but only 1 unsolvable task limits the solvability AUROC). P(True) > mean-logprob holds on both. C57's SELECT claim is thus bounded to moderate-difficulty tasks. Also: MBPP n_think is saturated for the median task (~86 tokens at both 256 and 2048 budgets) but the top ~10% tail is budget-bound (p99 256->2048); escalation still fails that tail because those tasks are capability-hard, not merely compute-starved.
2026-07-13 — difficulty curve resolves the dependence: conf-select's edge is on HARD tasks
Pooled MBPP+HumanEval (312 tasks), binned by per-task pass rate, conf-select vs majority at k=6: hard (pass 0.08, n=68) +0.045; medium (pass 0.54, n=27) +0.038; easy (pass 0.97, n=217) -0.007. The P(True)-select advantage concentrates exactly on the hard tasks and vanishes on easy ones — self-consistent with abstention (which flags the same hard tasks). Deployable rule: use conf-select where the model is uncertain; majority-vote is fine when it's already ~0.97 accurate.
2026-07-13 — CAPSTONE: compute-optimal = confidence-gated adaptive ALLOCATION (not escalation)
The escalation (spend on depth) was null, but ALLOCATION (spend breadth only where needed) is the real win. Confidence-gated adaptive sampling — commit the greedy answer if its P(True) >= threshold, else sample K=8 + conf-select — dominates uniform-k conf-select on the accuracy-vs-avg-compute frontier: reaches the k=9 ceiling 0.762 at ~4.25 avg samples (uniform needs 9), strictly beating uniform at 7/9 operating points (avg 1.79: 0.742 vs 0.720; avg 4.25: 0.762 vs 0.747). ~2x compute saving. Mechanism: spend samples only on low-confidence (hard) tasks — exactly where the difficulty curve shows conf-select helps. C57 -> Supported. Deployable policy: sample greedily once, read P(True); high -> commit; low -> sample K + argmax-P(True); abstain below a floor. One logit drives selection, abstention, AND allocation.
2026-07-13 — adaptive allocation generalizes to HumanEval
adaptive_compute.py --src humaneval: the confidence-gated adaptive frontier beats uniform at 7/9 operating points on HumanEval too (reaches the 0.941 ceiling at ~5 avg samples vs 9 uniform; one dip at avg 1.82). So the ~2x-compute-saving allocation win holds on both a moderate (MBPP) and an easy (HumanEval) benchmark.
2026-07-13 — capstone generalizes to REASONING (cross-domain, cross-signal)
adaptive_toy.py on the original C41 pool (qwen35_4b_confidence_guided_compute, 240 records, confidence = P(answer)/C40 not a P(True) judge): confidence-gated adaptive allocation beats uniform at 6/8 operating points (mid-compute +0.02..0.03; small dips only at the k=1 and k=12 extremes). So the compute-optimal-allocation result is domain-general — code (MBPP, HumanEval, P(True) judge) AND reasoning (toy, P(answer)) — a robust, verifier-free, provenance-clean deployable policy.
2026-07-13 — agentic-domain arbitration: deferred to a separate follow-up
Began testing whether the confidence policy generalizes from static code/reasoning to the multi-step GYM (gym_confidence.py in ../qwen35_4b_gauntlet_frontier/scripts). Friction found: the gym's induction/exploration atoms make the BASE model think heavily (~3200 think tokens at budget 4096, rarely closing </think>), so it only emits parseable ANSWER lines at the 8192 budget — slow in eager, and a standalone two-phase sampler does not reproduce the proven gym harness's emission handling (0 parseable answers at 4096). CLEAN PATH (owed): reuse the proven harness.run_atoms (fast, correct answers at 8192) and add a confidence signal — either answer-span logprobs (P(answer)/C40) exposed from the VLLMRunner, or a P(True) judge pass over gym answers. That is a ~half-day build vs the post-hoc code/reasoning analyses; deferred to a separate prospective experiment. gym_confidence.py is retained only as a historical WIP starting point; it does not keep this completed C57 experiment open.
2026-07-14 — capstone closed
- C57 is the terminal result of this experiment: confidence-gated adaptive breadth allocation wins at matched compute, while depth escalation failed replication.
- The n=400 correction, HumanEval check, and reasoning-domain check are complete. Agentic-domain arbitration remains a prospective follow-up with a new instrument and must receive its own experiment boundary.
Data files 2
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
Reproduce
Smoke test
python experiments/qwen35_4b_confidence_policy/scripts/frontier.pyFull run
python ../qwen35_4b_code_confidence/scripts/gen_budget.py --n 120 --k 6 --budgets 256 2048 && python experiments/qwen35_4b_confidence_policy/scripts/escalation.pyRunnable scripts exist in the experiment folder, but the exact invocation was not written down.