Qwen Latent Beam Program Compiler
The one idea you need
Instead of doing arithmetic out loud in words, the model punches a compact program — a start value and a list of operations — into fixed hidden slots, like holes in a player-piano roll that a separate built-in calculator plays back step by step.
The question
Can a small model solve multi-step math not by writing the steps out in words, but by quietly filling hidden slots with a program a built-in calculator then runs?
What we found
Yes, up to a point. It wrote exact programs that a fixed calculator ran perfectly at eight and twelve steps, and its answers matched those programs — so it truly computed rather than guessing, where chance is about one in a hundred. But at twenty-four steps it ran roughly 85% of the chain, then erred late. Producing four candidate programs and keeping the best stayed at chance.
Why it matters
Offloading multi-step reasoning to a fixed runtime works for short and medium chains and gives exact, checkable traces. Don't reach for candidate sampling to fix depth — picking the best of four is useless when none are correct. Grow depth gradually instead.
On this page
Results at a glance 7
How to read
Horizontal axis is program length in steps (8, 12, 24); vertical axis is the share of programs written exactly right, higher is better. One line is plain-worded prompts, the other reworded prompts.
Takeaway → Both lines sit near 100% through twelve steps, then split hard at twenty-four — reworded holds near 69% while plain collapses to zero, exposing a depth wall.
Data table
| program length (steps) | standard prompts | paraphrase prompts |
|---|---|---|
| 8 | 100% | 100% |
| 12 | 87.5% | 100% |
| 24 | 0% | 68.8% |
Numbers from reports/aggregate_final_metrics.csv (run pilot_single_compiler_len24_unpaired_s600), matches report table
Technical framing
Exact program recovery vs program length (single compiler, max-24 mixed training) — The compiler writes exact executable programs up to 12 steps, but 24-step behavior is brittle: 68.8% on paraphrase, 0% on standard.
How to read
Bars group four training setups on twelve-step plain prompts. Two bars each: the picked answer and the best of all candidates. Vertical axis is answer accuracy, higher is better.
Takeaway → Both candidate-search setups sit at the floor even for their best candidate, while the two single compilers reach 88 to 100 percent — search never held a correct program.
Data table
| training condition | selected answer | oracle over beams |
|---|---|---|
| 4 prompt beams | 0% | 3.1% |
| 4 latent beams | 1.6% | 1.6% |
| single compiler (mixed) | 87.5% | 87.5% |
| single compiler (paired) | 100% | 100% |
Technical framing
Beam search failed where a single supervised compiler succeeded (length-12 standard split) — Beam runs stay at chance even for the oracle, so beams never produced correct candidate programs; one supervised compiler solves the split.
How to read
Three twenty-four-step splits (plain, reworded, matched-pair prompts). Each has two bars: share of fully exact programs and share of the step-by-step trace matched before the first error. Higher is better.
Takeaway → Trace bars all stand near 85% while exact-program bars sit at zero — the compiler executes most steps, then makes a late fatal error.
Data table
| evaluation split | exact program | state prefix matched |
|---|---|---|
| standard len 24 | 0% | 84.6% |
| paraphrase len 24 | 1.6% | 84.4% |
| paired len 24 | 0% | 84.6% |
Technical framing
Length-24 failure mode after paired-template training: long correct prefixes, late errors — The paired-trained compiler executes ~85% of the 24-step trace correctly but almost never finishes exactly, so paired consistency alone did not fix depth.
How to read
Horizontal axis is training step; vertical axis is answer accuracy on twenty-four-step splits, higher is better. Four lines cross two training runs with plain and reworded prompts.
Takeaway → Most lines hug zero; the mixed run's reworded line jumps to 69% only at the final step, and the paired plain line peaks near 23% mid-run then crashes to zero.
Data table
| Training step | Mixed run, paraphrase-24 | Mixed run, standard-24 | Paired run, standard-24 | Paired run, paraphrase-24 |
|---|---|---|---|---|
| 1 | 0% | 0% | 0% | 0% |
| 150 | 1.6% | 3.1% | 0% | 1.6% |
| 151 | 3.1% | 0% | 0% | 0% |
| 300 | 0% | 3.1% | 1.6% | 0% |
| 301 | 0% | 3.1% | 0% | 1.6% |
| 450 | 1.6% | 0% | 15.6% | 1.6% |
| 451 | 9.4% | 1.6% | 23.4% | 14.1% |
| 600 | 68.8% | 4.7% | 0% | 1.6% |
| 750 | — | — | 0% | 3.1% |
Technical framing
Length-24 accuracy during training: gains arrive late and are unstable — The mixed run's 24-step win appears only at the final step; the paired run peaks at 23% mid-training and collapses back to zero.
How to read
Six evaluation splits (twelve- and twenty-four-step across three prompt styles). Two bars each: the selected candidate and the best of four. Vertical axis is answer accuracy, higher is better.
Takeaway → Every bar hugs the floor, with best-of-four never topping about 5% — the four sampled programs held no correct one, so selection could not help.
Data table
| Evaluation split | Selected beam | Oracle (best of 4 beams) |
|---|---|---|
| standard-12 | 1.6% | 1.6% |
| standard-24 | 1.6% | 3.1% |
| paraphrase-12 | 1.6% | 4.7% |
| paraphrase-24 | 0% | 1.6% |
| paired-12 | 0% | 1.6% |
| paired-24 | 1.6% | 2.3% |
Numbers from reports/aggregate_final_metrics.csv (pilot_latent_beam4_s160 rows)
Technical framing
Why beams failed: even the oracle over 4 latent beams is near chance — Oracle accuracy never exceeds 4.7%, so the beams contained no correct programs — the failure was candidate generation, not selection.
How to read
Four twenty-four-step measures: plain and reworded answer accuracy, plain and reworded trace-prefix. Two bars each: mixed-template training versus balanced paired-template training. Higher is better.
Takeaway → Paired training lifted plain trace prefixes from 49% to 85% but dropped reworded answers from 69% to 3% — a swap with no net gain.
Data table
| Length-24 split and metric | Mixed max-24 training | Paired-template training |
|---|---|---|
| standard-24 answer accuracy | 4.7% | 0% |
| paraphrase-24 answer accuracy | 68.8% | 3.1% |
| standard-24 state prefix | 48.6% | 84.6% |
| paraphrase-24 state prefix | 92.8% | 84.4% |
Technical framing
Paired-template training erased the mixed run's 24-step win — Paired training raised standard-24 trace prefixes from 49% to 85% yet dropped paraphrase-24 answers from 69% to 3% — no net win.
How to read
Two program lengths (4 and 8 steps). Two bars each: plain template versus reworded template. Vertical axis is answer accuracy, which equals exact program recovery here, higher is better.
Takeaway → At eight steps the plain bar drops to 78% while the reworded bar holds at 94.5% — prompt wording alone shifts reliability by about sixteen points.
Data table
| Program length | Standard template | Paraphrase template |
|---|---|---|
| length 4 | 91.4% | 94.5% |
| length 8 | 78.1% | 94.5% |
Numbers from reports/aggregate_final_metrics.csv (pilot_single_compiler_len8_s300 rows)
Technical framing
Max-8 compiler baseline: paraphrase prompts were easier than standard ones — The first working compiler already showed template sensitivity: standard prompts lose 13 points at length 8 while paraphrase holds at 94.5%.
In the author’s words from the Report · “Summary”
It also solved one 24-step paraphrase split in the mixed-template run. However, multi-beam candidate search did not create useful oracle candidates, and balanced paired-template training did not solve 24-step programs. The next experiment should therefore focus on structural depth expansion or denser long-depth supervision, not on adding more beams or paired consistency alone.
Overview
This experiment tests whether a Qwen 4B backbone can write a compact latent program into fixed register slots, execute that program with a differentiable modular-arithmetic runtime, and improve by selecting among multiple latent candidate programs.
Small metadata, logs, plots, and reports live in this directory. Large adapter and head checkpoints are stored under:
large_artifacts/qwen_latent_beam_program_compiler/checkpoints/
Layout
src/qwen_latent_beam_program_compiler_experiment.py- training and evaluation script.runs/- per-run metrics and JSON summaries.reports/- final standalone write-up and generated figures.checkpoint_manifest.csv- checkpoint locations for the latest completed run written by the training script.experiment_log.md- running lab notebook for decisions and results.
Main Measurements
selected_accuracy: answer accuracy of the learned selector.oracle_accuracy: answer accuracy if any beam is allowed to be selected.selected_oracle_gap_recovered: how much of the oracle-over-beams advantage the selector recovers.avg_distinct_answers: whether beams collapse or explore distinct programs.program_exactandstate_prefix_fraction: whether the compiled program matches the executable trace, not just the final answer.
Current Result
The standalone report is in reports/standalone_report.md.
The short version: beam-set training did not create useful candidate programs, but a single supervised compiler did learn exact executable programs. It solved 8- and 12-step modular programs reliably, reached 68.8% exact program accuracy on one 24-step paraphrase split, and failed to make 24-step behavior robust under paired-template training. The next experiment should use staged compiler expansion/resume rather than more beams.
Report
Rendered from reports/standalone_report.md
Summary
This experiment tested whether a small posttraining attachment can make a Qwen 4B language model configure a fixed invisible runtime instead of solving a task only through generated text. The task was modular arithmetic with known operation traces. The model saw a natural-language prompt plus latent register tokens, predicted a compact program into those registers, and a differentiable runtime executed the predicted program.
The main result is positive but bounded. A single supervised compiler learned to emit exact executable programs up to 12 steps with high reliability. It also solved one 24-step paraphrase split in the mixed-template run. However, multi-beam candidate search did not create useful oracle candidates, and balanced paired-template training did not solve 24-step programs. The next experiment should therefore focus on structural depth expansion or denser long-depth supervision, not on adding more beams or paired consistency alone.
Setup
- Base model:
Qwen/Qwen3-4Bloaded as a 4-bit causal LM with LoRA adapters. - Runtime: exact modular arithmetic over modulus 97 with operations
ADD,SUB, andMUL. - Compiler output: initial value, per-step operation, and per-step argument.
- Primary supervision: exact trace loss, soft runtime answer loss, and state trajectory loss.
Metrics:
selected_accuracy: final answer accuracy.selected_program_exact: exact executable program recovery.selected_state_prefix_fraction: fraction of the gold execution trace
matched before the first error.
oracle_accuracy: whether any beam was correct. With one beam, oracle and
selected accuracy are identical.
Large checkpoints are stored separately under large_artifacts/qwen_latent_beam_program_compiler/checkpoints/.
Results
Beam Search Did Not Create Useful Programs
The prompt-bank four-beam run and compact latent four-beam run both stayed near chance. The compact latent run ended with selected accuracy of 0-1.6% and oracle accuracy of 1.6-4.7% across the evaluated splits. Because oracle accuracy was also near chance, the failure was not mainly selector failure. The beams were not producing correct candidate programs.
One Supervised Compiler Worked
Removing beam search exposed the core mechanism. With one latent program bank and max 8 program steps, the compiler learned exact executable traces:
| Split | Answer Accuracy | Exact Program | State Prefix |
|---|---|---|---|
| standard length 4 | 91.4% | 91.4% | 94.9% |
| standard length 8 | 78.1% | 78.1% | 86.3% |
| paraphrase length 4 | 94.5% | 94.5% | 94.5% |
| paraphrase length 8 | 94.5% | 94.5% | 94.5% |
This is the most important positive result: the attachment was not merely predicting final answers. Answer accuracy tracked exact program recovery.
Max-24 Mixed Training Was Powerful but Brittle
The unpaired max-24 mixed-template run solved length 8 and 12, and it partially solved length 24:
| Split | Answer Accuracy | Exact Program | State Prefix |
|---|---|---|---|
| standard length 8 | 100.0% | 100.0% | 100.0% |
| standard length 12 | 87.5% | 87.5% | 97.8% |
| standard length 24 | 4.7% | 0.0% | 48.6% |
| paraphrase length 8 | 100.0% | 100.0% | 100.0% |
| paraphrase length 12 | 100.0% | 100.0% | 100.0% |
| paraphrase length 24 | 68.8% | 68.8% | 92.8% |
The asymmetry matters. It shows that the fixed-runtime compiler can learn 24-step programs, but the learned interface is still template- and curriculum-sensitive.
Paired Template Training Did Not Fix Full Depth
The paired run used standard/paraphrase pairs of the same latent program and a longer 300-step long stage. It solved length 8 and 12 across all prompt families but failed at length 24:
| Split | Answer Accuracy | Exact Program | State Prefix |
|---|---|---|---|
| standard length 12 | 100.0% | 100.0% | 100.0% |
| standard length 24 | 0.0% | 0.0% | 84.6% |
| paraphrase length 12 | 100.0% | 100.0% | 100.0% |
| paraphrase length 24 | 3.1% | 1.6% | 84.4% |
| paired length 12 | 100.0% | 100.0% | 100.0% |
| paired length 24 | 0.8% | 0.0% | 84.6% |
The high length-24 state-prefix score with low exact-answer accuracy means the compiler often executes much of the long trace correctly but still makes a late program error. Paired consistency alone did not remove those late errors.
Figures
- Final accuracy by run and length
- Length-24 training curves
- Program exactness vs answer accuracy
- State-prefix heatmap
- Training loss curves
Interpretation
The fixed-runtime idea is viable in a narrow but meaningful sense. A Qwen 4B backbone plus a small LoRA-trained compiler can write an exact program into latent registers, and a non-language runtime can execute that program. This is a concrete test-time-compute substrate: the model configures a computation graph instead of spelling out every intermediate state in text.
The current bottleneck is depth robustness. The model learns short and medium programs cleanly, but full 24-step behavior is fragile. The beam experiments showed that candidate search does not help until the compiler can reliably create correct candidates. The paired experiment showed that template consistency does not automatically produce full-depth reliability.
Next Experiment
The highest-value next experiment is structural compiler expansion:
- Train a max-8 compiler until exact program recovery is high.
- Expand the same compiler to max 16 by copying learned slot/head structure and initializing new step slots near the learned distribution.
- Continue training on length 8-16.
- Expand to max 24 and continue on length 12-24.
- Evaluate standard, paraphrase, and paired prompts at every expansion point.
This tests whether the failure is caused by asking a 24-slot compiler to learn all depths from scratch. It also matches the engineering shape of the desired posttraining tweak: grow a reusable latent execution scaffold rather than sampling independent candidate programs.
Experiment log 13
Show the running log (13 entries)
Objective
Train a small posttraining attachment on top of a Qwen 4B causal LM so the model can configure an invisible fixed runtime instead of emitting a chain-of-thought solution token by token. The immediate testbed is modular arithmetic with known program traces, because it gives exact supervision for program compilation, runtime execution, beam oracle coverage, and length generalization.
Artifact Policy
Large files are kept outside the experiment directory in large_artifacts/qwen_latent_beam_program_compiler/checkpoints/. The experiment directory keeps source code, CSV logs, JSON summaries, figures, and reports.
Runs
smoke_beam2
- Purpose: verify the initial prompt-bank implementation with two visible candidate banks.
- Outcome: plumbing passed. This was not intended as a learning result.
pilot_beam4_s120
- Purpose: test four actual prompt-side candidate banks on length-12 and length-24 modular programs.
- Setup: 4 beams, prompt-bank mode, 120 curriculum steps, max 24 program steps.
- Runtime note: sequence length was 2225 tokens for one example, so evaluation and training were expensive.
- Result: selected and oracle accuracy stayed near chance. Final oracle accuracy was 0-9.4% across splits, with no reliable selected-vs-oracle gap to recover.
- Decision: actual prompt-side beam banks are too token-expensive and do not create a useful candidate set in this form.
smoke_latent_beam4
- Purpose: verify compact latent-beam mode, where the prompt contains one bank and learned beam embeddings create multiple candidate programs inside the compiler.
- Outcome: plumbing passed. Sequence length dropped to 193 tokens in the tiny smoke configuration.
pilot_latent_beam4_s160
- Purpose: test whether compact latent beams can create useful candidate coverage without prompt-bank token cost.
- Setup: 4 latent beams, 160 curriculum steps, max 24 program steps, length-12 and length-24 evaluation.
- Runtime note: full-task sequence length was 597 tokens and the run completed in 838 seconds on an RTX 6000 Ada.
- Result: final selected accuracy stayed at 0-1.6%, and oracle accuracy stayed at 1.6-4.7%. The executor assigned roughly chance probability to the correct answer throughout training (
best_soft_answer_massstayed near 1/97). - Decision: latent beams create some output diversity but do not learn correct executable candidate programs under the current set-level objective.
Current Diagnosis
The failure is not primarily selector selection. Oracle-over-beams is also near chance, so there is no hidden high-quality beam for the selector to find. The next run should test the necessary subproblem directly: can the Qwen-attached compiler learn to emit one supervised executable program trace at all?
pilot_single_compiler_len8_s300
- Purpose: isolate the necessary subproblem by removing beam search and training one latent program compiler with direct supervised trace/runtime losses.
- Setup: 1 latent beam, no paired training, max 8 program steps, 300 curriculum steps over length 1-4 then 1-8.
- Runtime note: sequence length was 262 tokens and the run completed in 499 seconds on an RTX 6000 Ada.
- Result: strong positive signal. Final exact program accuracy matched final answer accuracy: 91.4% on standard length 4, 78.1% on standard length 8, 94.5% on paraphrase length 4, and 94.5% on paraphrase length 8. Correct-answer probability rose from chance near 1/97 to 0.997 on the final train row.
- Decision: the Qwen-attached fixed-runtime interface can learn executable programs. The highest-value next run is a deterministic compiler curriculum to 24 steps, with standard, paraphrase, and paired-template evaluation.
Updated Diagnosis
Beam set training failed because useful candidates were not being created, not because the fixed-runtime interface is unusable. A direct supervised compiler can write exact executable programs. The main question is now whether that competence survives longer programs and template variation at 24 steps.
pilot_single_compiler_len24_s400 (aborted diagnostic)
- Purpose: try the deterministic supervised compiler directly at max 24 steps with paired standard/paraphrase training.
- Setup: 1 latent beam, max 24 steps, paired training, eval at lengths 8, 12, and 24.
- Runtime note: sequence length was 580 tokens and the nine-split eval grid was too expensive for frequent in-training evaluation.
- Observed before stopping: at step 200, selected accuracy was still at chance across standard, paraphrase, and paired splits, and loss remained around 44.
- Decision: do not use this exact setup as the main run. Before adding more machinery, test the cleaner scaling variable: unpaired max-24 training with a larger batch and lighter eval.
pilot_single_compiler_len24_unpaired_s600
- Purpose: test whether the deterministic compiler scales to 24 slots when trained without paired-template batches and with a larger batch.
- Setup: 1 latent beam, max 24 program steps, unpaired mixed-template training, 600 curriculum steps, batch size 8, eval at lengths 8, 12, and 24.
- Runtime note: sequence length was 602 tokens, peak observed GPU memory was roughly 30 GB, and the completed run took 2294 seconds.
- Result: strong scaling to length 12 and partial length-24 success. Final exact program accuracy was 100% on standard length 8, 87.5% on standard length 12, 100% on paraphrase length 8, and 100% on paraphrase length 12. Length 24 was asymmetric: standard length 24 reached only 4.7% answer accuracy with 0% exact program accuracy, while paraphrase length 24 reached 68.8% exact program accuracy.
- Decision: max-24 compilation is viable, but template robustness is not solved. Run a balanced paired-template version with a longer long stage before treating this as the main result.
main_single_compiler_len24_paired_s750
- Purpose: test whether balanced paired standard/paraphrase training fixes the long-depth template asymmetry from the unpaired run.
- Setup: 1 latent beam, max 24 program steps, paired standard/paraphrase training, paired evaluation, 750 curriculum steps with a 300-step long stage.
- Runtime note: sequence length was 581 tokens, observed GPU memory was roughly 19-28 GB during training/eval, and the completed run took 3358 seconds.
- Result: length 8 and 12 were solved across standard, paraphrase, and paired splits. Final selected/exact program accuracy was 100% for standard length 8, standard length 12, paraphrase length 8, paraphrase length 12, paired length 8, and paired length 12. Length 24 did not solve: final selected accuracy was 0% on standard length 24, 3.1% on paraphrase length 24, and 0.8% on paired length 24.
- Decision: paired consistency is not enough for full-depth generalization. The current strongest positive result is deterministic fixed-runtime compilation through 12 steps and one unpaired 24-step paraphrase split. A better next experiment should use structural expansion/resume or denser long-depth supervision rather than more paired consistency alone.
Figures 5
Data files 16
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
reports/aggregate_final_metrics.csv7.1 kBreports/run_summary.csv1.4 kBruns/main_single_compiler_len24_paired_s750/metrics.csv1.7 kBruns/main_single_compiler_len24_paired_s750/results.json83 kBruns/pilot_beam4_s120/metrics.csv1.4 kBruns/pilot_beam4_s120/results.json53 kBruns/pilot_latent_beam4_s160/metrics.csv1.5 kBruns/pilot_latent_beam4_s160/results.json54 kBruns/pilot_single_compiler_len24_unpaired_s600/metrics.csv1.1 kBruns/pilot_single_compiler_len24_unpaired_s600/results.json48 kBruns/pilot_single_compiler_len8_s300/metrics.csv968 Bruns/pilot_single_compiler_len8_s300/results.json27 kBruns/smoke_beam2/metrics.csv787 Bruns/smoke_beam2/results.json11 kBruns/smoke_latent_beam4/metrics.csv805 Bruns/smoke_latent_beam4/results.json11 kB
Reproduce
The run commands are documented inside the experiment folder (see the README).