Qwen Prefix-State Process Verifier
The one idea you need
To answer each puzzle the system writes a tiny step-by-step program for a calculator-like machine, one instruction at a time. A judge model rates each half-written program for whether it still looks on-track, like a proofreader checking a derivation line by line, sharp at flagging detours but unsure which promising draft actually finishes right.
The question
Can a helper that scores half-finished programs steer the search toward a correct one, without ever peeking at the right answer while it decides?
What we found
Barely. The judge got genuinely good at telling promising partial programs from dead ends, yet it lifted first-try accuracy only a few points, with hard problems going from 41% to 44%. The real gap: on hard problems a correct program sat among the candidates 70% of the time but got chosen only 41%. Knowing good from bad is not the same as surfacing the winner.
Why it matters
If you rerank partial solutions with a helper model, first confirm the right one is even in your candidate pool. When it is there but unranked, the fix is scoring whether a draft can still reach the answer, not whether it matches one textbook path.
On this page
Results at a glance 6
How to read
Each group is one way of choosing the final program: plain first guess, ranking by the model's own confidence, adding the judge's opinion, checking against the answer, and the best any candidate could reach; bar height is how often the chosen program is correct (higher is better), split into everyday and harder problems.
Takeaway → The judge-guided bar sits just above the plain first-guess bar, while the answer-checked and best-possible bars tower far higher, so most of the room went uncaptured.
Data table
| decoder | fresh paired | hard composition |
|---|---|---|
| greedy | 64.1% | 41.4% |
| logprob beam | 64.1% | 41.4% |
| verifier beam (w=2) | 64.8% | 44.5% |
| answer-verified repair | 82% | 70.3% |
| beam oracle | 84.4% | 70.3% |
Numbers from analysis/final_metrics.csv (main_prefix_state_verifier_s512 rows)
Technical framing
Prefix verifier barely closes the gap to answer-verified repair — Verifier-guided beam adds only ~1-3pp over greedy; answer-checked repair and the beam oracle show far more headroom.
How to read
Left to right the judge's opinion counts more toward the choice (zero means ignore it); height is how often the final program is correct, one line per problem set, higher is better.
Takeaway → Every line rises slightly at low weight then falls, so leaning lightly on the judge helps but leaning hard makes picks worse.
Data table
| verifier weight in beam score (0 = logprob only) | fresh standard | fresh paraphrase | fresh paired | hard composition |
|---|---|---|---|---|
| 0 | 63.3% | 60.2% | 64.1% | 41.4% |
| 0.5 | 65.6% | 62.5% | 63.3% | 42.2% |
| 1 | 64.1% | 63.3% | 63.3% | 43.8% |
| 2 | 66.4% | 61.7% | 64.8% | 44.5% |
| 4 | 63.3% | 59.4% | 64.1% | 44.5% |
| 8 | 60.9% | 57.8% | 64.1% | 43% |
| 16 | 57.8% | 56.2% | 63.3% | 43% |
Numbers from analysis/final_metrics.csv (main_prefix_state_verifier_s512 rows)
Technical framing
Verifier weight sweep: small gains, then decay — Moderate verifier weight (1-2) gives the best top-1 accuracy on every split; heavy weighting (8-16) hurts.
How to read
For each problem set, the lighter bar is how often the judge's chosen program is correct and the taller bar is how often the candidate set held a correct program at all; higher is better.
Takeaway → The taller bar stands well above the chosen-program bar in every set, showing the right program is usually present but not selected.
Data table
| evaluation split | verifier beam top-1 (w=2) | beam oracle |
|---|---|---|
| fresh standard | 66.4% | 82.8% |
| fresh paraphrase | 61.7% | 84.4% |
| fresh paired | 64.8% | 83.6% |
| hard composition | 44.5% | 73.4% |
Numbers from analysis/final_metrics.csv (main_prefix_state_verifier_s512, beam_verifier_w2 rows)
Technical framing
Correct programs sit in the beam but reranking misses them — The beam contains a correct program 73-84% of the time, but top-1 selection lags by 16-29pp — the reranking gap remains.
How to read
Bars compare a small and a large training set, for both the plain first guess and the answer-checked pick, across problem sets; height is accuracy, higher is better.
Takeaway → Bars for the larger training set dwarf the smaller one, so adding training data lifts accuracy far more than any picking trick.
Data table
| Evaluation split | Greedy, 128 train programs | Answer-verified repair, 128 train programs | Greedy, 512 train programs | Answer-verified repair, 512 train programs |
|---|---|---|---|---|
| val_mixed | 12.5% | 46.9% | 63.3% | 79.7% |
| fresh_standard | 12.5% | 53.1% | 63.3% | 82% |
| fresh_paraphrase | 9.4% | 43.8% | 60.2% | 82.8% |
| fresh_paired | 9.4% | 46.9% | 64.1% | 82% |
| hard_composition | 18.8% | 59.4% | 41.4% | 70.3% |
Numbers from analysis/final_metrics.csv
Technical framing
Training set size matters more than any decoder: pilot (128 programs) vs main (512) — Quadrupling compiler training data lifts greedy accuracy roughly 5x on most splits, dwarfing every search-time intervention.
How to read
Left to right is more training passes; height is how well the judge separates promising partial programs from dead ends (higher is better), with one line for the large training set and one for a smaller pilot.
Takeaway → Both lines climb then flatten near the top, so the judge became a strong rater, yet that skill never turned into better final picks.
Data table
| Training epoch | Main run (31,574 prefix samples) | Pilot run (6,841 prefix samples) |
|---|---|---|
| 1 | 0.926 | 0.849 |
| 2 | 0.929 | 0.892 |
| 3 | 0.93 | 0.913 |
| 4 | 0.934 | 0.921 |
| 5 | 0.937 | — |
| 6 | 0.937 | — |
| 7 | 0.934 | — |
| 8 | 0.936 | — |
Numbers from analysis/verifier_train_logs.csv
Technical framing
Verifier learns quickly then plateaus: held-out prefix AUC by training epoch — Prefix-state AUC saturates near 0.937 by epoch 5; a strong classifier still fails to steer beam top-1, so AUC is not the bottleneck.
How to read
Left to right is training progress; height is accuracy, with one line for getting individual instructions right and one for getting the entire program exactly right, higher is better.
Takeaway → The instruction line climbs well above the whole-program line, so the model nails pieces more often than complete programs, the gap the judge was meant to close.
Data table
| Training step (main run) | Bytecode token accuracy | Whole-program exact match |
|---|---|---|
| 1 | 0% | 0% |
| 150 | 18.8% | 0% |
| 300 | 51.6% | 26.6% |
| 450 | 60.9% | 31.2% |
| 600 | 65.6% | 35.9% |
Numbers from analysis/compiler_train_logs.csv
Technical framing
Compiler head training: bytecode tokens are learned faster than whole programs — By step 600 the compiler gets 66% of opcodes right but only 36% of full programs exact, leaving room search was meant to fill.
In the author’s words from the Report · “Abstract”
The verifier learned the prefix classification task well, reaching held-out prefix AUC 0.937 on the main run. It produced modest top-1 decoding gains, especially on hard-composition prompts: hard accuracy improved from 41.4% greedy to 44.5% with beam_verifier_w2. Fresh paired accuracy improved from 64.1% greedy to 64.8% with beam_verifier_w2. The much larger gap remained between no-answer top-1 decoding and answer-verified repair: fresh paired local answer repair reached 82.0%, and hard-composition local answer repair reached 70.3%.
Overview
Standalone experiment for prefix-level verification and typed beam search over executable bytecode.
The experiment asks whether a learned verifier over partial programs and VM state can close the gap between greedy bytecode decoding and answer-verified local search. Qwen is used as the prompt encoder, a bytecode compiler head proposes opcode/argument distributions, and a prefix-state verifier reranks typed beam-search prefixes.
Layout
src/ training, evaluation, and analysis code
runs/ per-run JSON/CSV logs
analysis/ aggregate tables and figures
reports/ standalone Markdown and HTML reportLarge checkpoints are stored outside this directory:
large_artifacts/qwen_prefix_state_process_verifier/checkpoints/Reading Order
reports/qwen_prefix_state_process_verifier_paper.mdanalysis/summary.mdanalysis/final_metrics.csvexperiment_log.md
Report
Rendered from reports/qwen_prefix_state_process_verifier_paper.md
Abstract
This experiment tests a learned prefix-state verifier for executable bytecode search. A frozen Qwen 4B model encodes the prompt, a trained compiler head emits opcode and argument distributions, and a prefix verifier scores partial bytecode prefixes together with the current VM stack. The verifier is then used to guide typed beam search without using the final answer at decode time.
The verifier learned the prefix classification task well, reaching held-out prefix AUC 0.937 on the main run. It produced modest top-1 decoding gains, especially on hard-composition prompts: hard accuracy improved from 41.4% greedy to 44.5% with beam_verifier_w2. Fresh paired accuracy improved from 64.1% greedy to 64.8% with beam_verifier_w2. The much larger gap remained between no-answer top-1 decoding and answer-verified repair: fresh paired local answer repair reached 82.0%, and hard-composition local answer repair reached 70.3%.
Experimental Question
The question is whether a process verifier over partial programs can convert a high-oracle beam into better deployable top-1 bytecode. The verifier sees a prompt representation, the current bytecode prefix, the VM stack after the proposed next action, and the proposed action. It predicts whether the prefix remains consistent with a known correct executable trace.
Runtime And Decoder
The runtime is a bounded typed stack machine over integers modulo 97. Programs have at most 16 slots and use arithmetic, comparison, min/max, modulo, and lookup opcodes. Search expands only type-valid actions, so invalid stack programs are pruned before scoring.
The evaluated decoders are:
greedy: constrained greedy compiler decoding;local_answer: complete-program local repair selected by final-answer verification;beam_logprob: typed beam search using compiler log probability only;beam_verifier_w*: typed beam search using compiler log probability plus a weighted sum of prefix-verifier log-scores.
Data And Training
The task generator creates prompts across modular arithmetic chains, weekday offsets, unit scaling, list aggregation, boolean threshold checks, and table lookup. The main run used 512 compiler-trace prompts, 512 verifier prompts, 128 examples per evaluation split, and ran on NVIDIA RTX 6000 Ada Generation.
The verifier training set contained 31,574 prefix samples with 9.2% positives. Positives are gold-consistent executable prefixes; negatives are off-path prefixes generated by the compiler's own typed beam distribution.
Results
| split | decoder | accuracy | program_exact | oracle_accuracy | mean_expansions |
|---|---|---|---|---|---|
| fresh_standard | greedy | 63.3% | 43.8% | 63.3% | 1 |
| fresh_standard | local_answer | 82.0% | 47.7% | 82.0% | 31.8 |
| fresh_standard | beam_logprob | 63.3% | 43.8% | 80.5% | 305.2 |
| fresh_standard | beam_verifier_w0.5 | 65.6% | 44.5% | 82.8% | 304.7 |
| fresh_standard | beam_verifier_w1 | 64.1% | 43.8% | 82.8% | 304 |
| fresh_standard | beam_verifier_w2 | 66.4% | 45.3% | 82.8% | 304.7 |
| fresh_standard | beam_verifier_w4 | 63.3% | 44.5% | 83.6% | 304.8 |
| fresh_standard | beam_verifier_w8 | 60.9% | 43.8% | 83.6% | 304.4 |
| fresh_standard | beam_verifier_w16 | 57.8% | 41.4% | 84.4% | 304.4 |
| fresh_paraphrase | greedy | 60.2% | 38.3% | 60.2% | 1 |
| fresh_paraphrase | local_answer | 82.8% | 45.3% | 82.8% | 30.5 |
| fresh_paraphrase | beam_logprob | 60.2% | 38.3% | 82.8% | 298 |
| fresh_paraphrase | beam_verifier_w0.5 | 62.5% | 39.1% | 82.8% | 295.7 |
| fresh_paraphrase | beam_verifier_w1 | 63.3% | 39.1% | 82.0% | 295 |
| fresh_paraphrase | beam_verifier_w2 | 61.7% | 38.3% | 84.4% | 297.4 |
| fresh_paraphrase | beam_verifier_w4 | 59.4% | 37.5% | 85.2% | 298.1 |
| fresh_paraphrase | beam_verifier_w8 | 57.8% | 37.5% | 84.4% | 297.6 |
| fresh_paraphrase | beam_verifier_w16 | 56.2% | 36.7% | 82.8% | 297.6 |
| fresh_paired | greedy | 64.1% | 47.7% | 64.1% | 1 |
| fresh_paired | local_answer | 82.0% | 55.5% | 82.0% | 28.1 |
| fresh_paired | beam_logprob | 64.1% | 47.7% | 84.4% | 288.4 |
| fresh_paired | beam_verifier_w0.5 | 63.3% | 47.7% | 83.6% | 288.1 |
| fresh_paired | beam_verifier_w1 | 63.3% | 47.7% | 84.4% | 286.4 |
| fresh_paired | beam_verifier_w2 | 64.8% | 49.2% | 83.6% | 290 |
| fresh_paired | beam_verifier_w4 | 64.1% | 49.2% | 85.2% | 290.3 |
| fresh_paired | beam_verifier_w8 | 64.1% | 47.7% | 85.2% | 289.7 |
| fresh_paired | beam_verifier_w16 | 63.3% | 48.4% | 85.9% | 289.8 |
| hard_composition | greedy | 41.4% | 29.7% | 41.4% | 1 |
| hard_composition | local_answer | 70.3% | 34.4% | 70.3% | 33.5 |
| hard_composition | beam_logprob | 41.4% | 29.7% | 70.3% | 315.5 |
| hard_composition | beam_verifier_w0.5 | 42.2% | 30.5% | 72.7% | 312.7 |
| hard_composition | beam_verifier_w1 | 43.8% | 31.2% | 73.4% | 312.4 |
| hard_composition | beam_verifier_w2 | 44.5% | 31.2% | 73.4% | 313.6 |
| hard_composition | beam_verifier_w4 | 44.5% | 29.7% | 71.9% | 315 |
| hard_composition | beam_verifier_w8 | 43.0% | 28.1% | 72.7% | 314.1 |
| hard_composition | beam_verifier_w16 | 43.0% | 28.1% | 72.7% | 313.8 |
Main run top-1 executable accuracy by decoder.
Top-1 beam accuracy compared with beam oracle accuracy.
Held-out prefix verifier AUC during training.
Prefix sample counts for pilot and main runs.
Interpretation
The prefix verifier does learn a meaningful process signal: AUC above 0.93 is not a weak classifier. It also raises some top-1 no-answer accuracy: hard-composition accuracy improved by +3.1 pp, and fresh paired improved by +0.8 pp. However, these gains are much smaller than the available beam/search headroom.
The central result is the distinction between containment and selection. On fresh paired prompts, logprob beam top-1 was 64.1%, but the same beam contained a correct executable program 84.4% of the time. On hard composition, logprob beam top-1 was 41.4%, while the beam oracle was 70.3%. The correct programs are frequently in the beam; this verifier formulation does not yet rank them aggressively enough.
The answer-verified local repair remains a strong upper comparison, reaching 82.0% fresh paired and 70.3% hard. That condition uses final-answer feedback at decode time, so it is not the deployable no-answer setting, but it shows that the compiler's nearby candidate space is much better than greedy decoding.
Failure Analysis
This verifier is trained mostly as a gold-prefix classifier. That is a useful process signal, but it is not the same as semantic reachability. A prefix can deviate from the canonical trace and still complete to a correct program, while a gold-looking prefix can still lose because of later argument choices. The scorer also accumulates prefix log-sigmoid penalties, which can over-penalize longer correct programs and does not directly optimize final top-1 answer accuracy.
Next Step
The next version should train a semantic value model rather than an exact-prefix classifier. Labels should come from suffix completion search: for a partial prefix, ask whether any bounded continuation can still execute to the correct answer. That would turn the verifier from "does this match the teacher trace?" into "is this prefix still live?" The search policy should then optimize expected completion success, not prefix exactness.
A second improvement is to distill successful verifier-beam or answer-verified beam programs back into the compiler head, so the direct compiler learns from the high-oracle beam instead of relying on expensive search at inference time.
Experiment log 1
Show the running log (1 entry, 2026-06-23)
2026-06-23
- Created standalone experiment directory with
src/,runs/,analysis/,reports/, and a separate large-artifact checkpoint root. - Design choice: use frozen Qwen prompt features plus a trained bytecode compiler head, then test whether a prefix-state process verifier can improve typed beam search over greedy decoding and simple local answer-verified search.
Planned conditions:
- compiler-only greedy constrained decoding;
- fixed local answer-verified repair over complete programs;
- typed beam search scored only by compiler log probability;
- typed beam search scored by compiler log probability plus the learned prefix-state verifier.
- Implemented the self-contained experiment harness in
src/qwen_prefix_state_process_verifier_experiment.py. - Verified VM execution and prefix-sample generation locally.
- Ran
smoke_prefix_state_verifier, an end-to-end Qwen-backed smoke test with tiny data. It loadedQwen/Qwen3-4B, extracted frozen features, trained a tiny compiler, collected prefix samples, trained the verifier, wrote metrics, and saved the checkpoint under the large-artifact root. - Ran
pilot_prefix_state_verifier_s128. The verifier reached held-out prefix AUC 0.921, but top-1 beam gains were modest with the weak 128-example compiler. Fresh paired greedy was 9.4%, local answer repair was 46.9%, compiler-logprob beam was 9.4%, and the best verifier beam was 12.5%. This suggests the main run needs a stronger compiler and a wider verifier-weight sweep. - Started
main_prefix_state_verifier_s512; interrupted the first attempt during final evaluation because the naive beam implementation scored every prefix expansion with a separate verifier forward pass. - Patched typed beam search to score each beam layer in batches and reran
main_prefix_state_verifier_s512cleanly. - Main result: verifier AUC reached 0.936 on held-out prefix states. Fresh paired greedy was 64.1%, local answer repair was 82.0%, compiler-logprob beam was 64.1%, and the best verifier beam was 64.8%. Hard-composition greedy was 41.4%, local answer repair was 70.3%, compiler-logprob beam was 41.4%, and the best verifier beam was 44.5%.
Figures 4
Data files 7
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
analysis/final_metrics.csv17 kBruns/main_prefix_state_verifier_s512/metrics.csv6.5 kBruns/main_prefix_state_verifier_s512/results.json21 kBruns/pilot_prefix_state_verifier_s128/metrics.csv4.7 kBruns/pilot_prefix_state_verifier_s128/results.json16 kBruns/smoke_prefix_state_verifier/metrics.csv2.9 kBruns/smoke_prefix_state_verifier/results.json12 kB
Reproduce
This entry is source code or analysis only — there is no separate run to reproduce.