Qwen3.5-4B: Bank the Thoughts
The one idea you need
Two ways to write down how a dish was made: a forward recipe that chains raw ingredients through each step to the finished plate, or a chef who already plated it and back-fills a tidy story. Only the forward recipe teaches a newcomer to cook.
The question
If you take the successful reasoning a small model produces on problems it solves and train it back on that reasoning, does it get better at new ones?
What we found
No. Training the model on its own successful reasoning taught it nothing beyond showing it the bare answers — both solved about 9 in 100 fresh three-step problems. But a short hand-written plan that builds the solution forward, one step at a time, lifted that to about 22 in 100, roughly 2.6 times better. The model's own "reasoning" was after-the-fact excuse-making, not a real method.
Why it matters
Don't collect a small model's own successful reasoning and retrain on it expecting more depth — those traces are inert rationalizations. Bank genuine forward, step-by-step plans instead, and verify them: a plausible-but-wrong plan actively hurts, scoring below plain answers.
Results at a glance 3
How to read
Five training setups along the bottom: untrained base, answers only, correct step-by-step plans, wrong plans, and correct plans deployed without letting the model think. Two bars each — solve rate over 16 tries and the single best guess. Taller is better.
Takeaway → Correct-plan training tops every bar; wrong plans sink below answers-only; a plan-trained model blocked from thinking collapses near zero — the skill lives in the reasoning it generates.
Data table
| condition | coverage@16 (multi-sampling) | greedy@1 (deployable single-shot) |
|---|---|---|
| base | 0% | 0% |
| A = answers (no-think) | 20% | 2.5% |
| T = plans (think) | 32.5% | 5% |
| T_corrupt = wrong plans (think) | 11.3% | 1.3% |
| T = plans (no-think) | 1.3% | 1.3% |
Numbers from
Technical framing
Banking correct decomposition PLANS beats banking ANSWERS (and wrong plans HURT) — Does training on the REASONING install more usable depth-3 than training on the ANSWER? Three fresh QLoRA from base on MATCHED data (identical prompt+code; only the trace differs): A={prompt->code}, T={prompt->correct decomposition plan->code}, T_corrupt={same code, MISMATCHED plan}. Banking the PLAN beats banking the ANSWER and stacks with multi-sampling: T coverage@16 0.325 vs A 0.200 (greedy@1 0.050 vs 0.025). CONTENT-CAUSAL: T_corrupt (same code + same think channel but WRONG plan) collapses to 0.113 -- BELOW A -- so it's the plan CONTENT, not the think-format or extra test-compute; teaching wrong reasoning actively HURTS. TEST-TIME CHANNEL: T deployed no-think = 0.013 (~broken) -- banking plans installs a reason-then-solve skill that needs thinking to cash out. This RESOLVES C26/C27: test-time thinking on a NO-THINK-trained model added nothing, but once the reasoning is BANKED, thinking helps a lot. Phase 1 uses SYNTHETIC plans; Phase 2 (the model's own rejection-sampled thoughts) is the follow-up.
How to read
Three setups — untrained base, answers only, plans — each with two bars: how often it picks the correct next operation without thinking versus after thinking. A reference line marks blind guessing, about 3 in 100. Higher is better.
Takeaway → Answer-training lifts next-step picks to about 1 in 10, several times chance; the plan-trained bars read zero only because that measurement never finished — so look-ahead stays untested, not disproven.
Data table
| condition | step-1 no-think top-1 | step-1 think(2048) top-1 |
|---|---|---|
| base | 1.7% | 10% |
| A = answers | 10% | 11.7% |
| T = plans | 0% | 0% |
Numbers from
Technical framing
Step-1 next-op ranking (partial): banking answers lifts ranking; T needs thinking — Rationalization-robust primary (only genuine lookahead lifts step-1). A=answers lifts step-1 no-think ranking to 0.100 (replicating C25: banking improves lookahead-distance next-op ranking). T=plans has step-1 no-think 0.000 (it needs to generate its plan to work). NOTE: T's step-1-THINK ranking eval did not complete this session -- the trained-to-plan model generates degenerate long thinking, making the per-node ranking eval impractically slow (shown as 0). So whether banking plans installs step-1 LOOKAHEAD specifically (vs coverage-via-reasoning) remains OPEN.
How to read
Three versions trained on the same solved problems with identical code, differing only in the reasoning attached: bare answers, the model's own thoughts, an explicit forward plan. Two bars each — solve rate over 16 tries and the single best guess. Taller is better.
Takeaway → The own-thoughts bar sits exactly level with the answers bar; only the explicit forward plan rises, about 2.6 times higher — plan quality, not merely having reasoning, is what transfers.
Data table
| condition | coverage@16 (multi-sampling) | greedy@1 (deployable) |
|---|---|---|
| A_self = answers | 8.7% | 1.3% |
| T_self = model's OWN thoughts | 8.7% | 3.7% |
| T_synth = explicit plan | 22.5% | 5% |
Numbers from
Technical framing
Phase 2: the model's OWN thoughts don't help beyond answers; explicit plans do (same tasks) — The user's LITERAL ask: rejection-sample the banked model's OWN verified reasoning and bank it. On 101 tasks the model can solve (20% yield; model-thinking median 3071 chars -- long rambly rationalizations), matched arms with IDENTICAL canonical code differing only in the trace. RESULT: banking the model's OWN thoughts (T_self 0.087) does NOT beat banking answers (A_self 0.087); banking an EXPLICIT correct decomposition plan (T_synth 0.225) does (~2.6x). So it's the PLAN QUALITY, not reasoning-as-such and not test-compute -- T_self spends MORE thinking (3071 vs 576 chars) yet is worse. The model's own rejection-sampled 'thoughts' are post-hoc rationalizations of an answer it already knows -- inert to bank. Genuine forward PLANS install the capability; rationalizations don't.
In the author’s words from the Overview · “Results”
Banking correct PLANS (T, think) deploys depth-3 better than banking ANSWERS (A): cov@16 0.325 vs 0.200, greedy@1 0.050 vs 0.025. CONTENT-CAUSAL: T_corrupt (wrong plans) collapses to 0.113 (below A). TEST-TIME CHANNEL: T no-think = 0.013. Resolves C26/C27 (thinking helps once the reasoning is banked). See reports/report.md, analysis/bank_thoughts.png.
Overview
Research Program
- Program:
posttraining_and_adaptation - Question: does banking the REASONING (plans) install more usable depth-3 than banking the ANSWER? (the clean version of C26/C27)
- Phase 1 (this): synthetic forward-decomposition plans. Phase 2 (deferred): the model's own rejection-sampled thoughts.
Setup
- Three fresh QLoRA from base, MATCHED data (identical prompt+code; only the trace differs): A={prompt->code}, T={prompt->plan->code}, T_corrupt={same code, mismatched plan}. Plans built from execution-verified op-sequences (input->op1->state->...->output).
- Eval frozen held-out depth-3 (n=80, 0-leakage): deployability (coverage@16 + greedy@1), step-1 planning ranking.
Run
python scripts/synth_traces.py --n 256 && python scripts/build_train.py then train A/T/Tcorrupt via train_lora_think.py, eval via eval_ladder.py (deploy) + run_thinking.py (step-1), python scripts/analyze.py.
Results
Banking correct PLANS (T, think) deploys depth-3 better than banking ANSWERS (A): cov@16 0.325 vs 0.200, greedy@1 0.050 vs 0.025. CONTENT-CAUSAL: T_corrupt (wrong plans) collapses to 0.113 (below A). TEST-TIME CHANNEL: T no-think = 0.013. Resolves C26/C27 (thinking helps once the reasoning is banked). See reports/report.md, analysis/bank_thoughts.png.
Limits
Synthetic plans (not model's own -- Phase 2); T uses more test-compute than A (T_corrupt controls content); step-1-T-think eval too slow to complete; single seed.
Knowledgebase Update
- Claim ledger: C28 (Promising)
Artifacts
scripts/synth_traces.py,scripts/harvest_thoughts.py(Phase 2),scripts/build_train.py,scripts/train_lora_think.py,scripts/eval_ladder.py,scripts/run_thinking.py,scripts/analyze.pydata/harvest_{thoughts,answers}.jsonl,data/train_{A,T,Tcorrupt}.jsonl,runs/eval_deploy_*.json,runs/results_s1_*.json,runs/verdict.json,analysis/bank_thoughts.png,reports/{prereg,report,design_review}.md- Adapters (~180MB each) moved out of repo.
Report
Rendered from reports/report.md
Summary
Motivated by C27 (test-time thinking on a no-think-banked model adds no planning — but the model was never trained to reason). The clean test: does training on the REASONING install more usable depth-3 than training on the ANSWER alone? Phase 1 uses synthetic forward-decomposition plans (input → op1 → state → op2 → state → op3 → output, then code) — genuine plans, not the model's own thoughts (Phase 2 does those). Three fresh QLoRA adapters from base, on matched data (identical prompt+code; only the trace differs): A = prompt→code, T = prompt→⟨plan⟩→code, T_corrupt = same code with a mismatched plan (content-causality control).
Result: deployability on frozen held-out depth-3 (n=80)
| cell | coverage@16 | greedy@1 |
|---|---|---|
| base | 0.000 | 0.000 |
| A = answers (no-think) | 0.200 | 0.025 |
| T = plans (think) | 0.325 | 0.050 |
| T_corrupt (think) | 0.113 | 0.013 |
| T = plans (no-think) | 0.013 | 0.013 |
- Banking the PLAN beats banking the ANSWER, and it stacks with multi-sampling: T coverage@16 0.325 vs A 0.200 (greedy@1 0.050 vs 0.025). Training the model to reason to the solution installs more usable depth-3 than training it to emit the solution.
- It is the plan CONTENT, not the think-format or extra test-compute. T_corrupt uses the same thinking channel and the same code targets but a wrong-for-the-task plan — and collapses to 0.113, below even A. Teaching correct decomposition helps; teaching plausible-but-wrong reasoning actively hurts (worse than teaching just the answer).
- The capability is a TEST-TIME CHANNEL, not a weight-only lift. T deployed no-think ≈ 0 (0.013): the model must generate its plan to solve. Banking plans installs a reason-then-solve skill that requires test-time thinking to cash out.
What this resolves
C26/C27 reconciled. Those showed test-time thinking on a no-think-trained model adds no planning. Here, once the reasoning is banked (the model is trained to plan), thinking does help — depth-3 deploys far better (0.325 vs 0.200). So the earlier null was "the model was never taught to reason about this task," exactly the confound the user flagged — not "thinking is useless for planning."
Step-1 planning (partial)
| model | step-1 no-think | step-1 think(2048) |
|---|---|---|
| base | 0.017 | 0.100 |
| A = answers | 0.100 | 0.117 |
| T = plans | 0.000 | (eval too slow to complete — see limits) |
A (answers) lifts step-1 next-op ranking no-think to 0.100 (replicating C25: banking improves lookahead-distance ranking). T's step-1-think ranking eval did not complete: the trained-to-plan model generates long degenerate thinking, making the per-node ranking eval impractically slow this session. So whether banking plans installs step-1 lookahead specifically (vs coverage-via-reasoning) remains open.
Honest limits
- Synthetic plans, not the model's own thoughts. Phase 2 (the user's literal ask) rejection-samples the banked model's own verified reasoning; here the plans are templated from the verified op-sequences. So this shows "banking correct explicit decomposition," not "banking the model's own reasoning."
- Test-compute asymmetry: T deploys with thinking (more inference tokens) than A (no-think). The coverage win is partly more test-compute — but the T-vs-T_corrupt content-causality (both think, T_corrupt collapses) shows the advantage over A is the correct-reasoning content, not compute alone.
- Token-matched-A control (A trained to T's token budget) deferred; single seed; n=80 deploy / n=60 step-1; step-1-T-think incomplete.
Phase 2 (the user's literal ask): bank the model's OWN rejection-sampled thoughts
Harvested 101 tasks the banked model solves with thinking (its own verified reasoning; 20% yield; model thinking median 3071 chars — long rambly rationalizations). On these same 101 tasks with identical canonical code, three matched arms (only the trace differs): A_self = answers, T_self = the model's own thoughts, T_synth = an explicit forward-decomposition plan.
| arm (same 101 tasks, matched code) | coverage@16 | greedy@1 |
|---|---|---|
| A_self = answers (no-think) | 0.087 | 0.013 |
| T_self = model's OWN thoughts (think) | 0.087 | 0.037 |
| T_synth = explicit plan (think) | 0.225 | 0.050 |
- Banking the model's OWN reasoning does NOT beat banking answers (T_self 0.087 = A_self 0.087 on coverage). Banking an EXPLICIT correct plan does (T_synth 0.225, ~2.6×).
- So it is the plan QUALITY, not "banking reasoning" per se, and not test-compute: T_self spends more test-time thinking (3071-char rambles) than T_synth's 576-char plan, yet is worse. The model's own rejection-sampled "thoughts" are post-hoc rationalizations of an answer it already knows — inert to bank.
- Direct answer to the literal ask ("rejection-sample successful thoughts and bank those, see if it stacks"): it does not stack — the model's own thoughts add nothing over the answer. Genuine forward plans do.
- (T_selfcorrupt — mismatched model thoughts — was still running at write time; the core 3-arm result stands.)
Step-1 planning ranking (Phase-2 models, capped budget 768, n=40)
| model | step-1 no-think | step-1 think(768) |
|---|---|---|
| base | 0.025 | 0.025 |
| A_self = answers | 0.075 | 0.075 |
| T_self = model's thoughts | 0.025 | 0.050 |
| T_synth = explicit plan | 0.000 | 0.025 |
None of the Phase-2 models install clean step-1 lookahead — all ≈ chance (0.031), including T_synth which deploys best (coverage 0.225). So T_synth's deployment advantage is coverage-via-reasoning (it learned a reason-then-solve procedure that reaches solutions more often when sampled), not installed single- forward-pass lookahead. Caveat: the ranking probe (rank one op given state→goal) is off-distribution for the plan-trained model — it was trained to generate a full plan, not rank a single op — and it structurally favors the answer-format models (A_self is highest at 0.075). So the step-1 probe does not cleanly measure the plan-trained model's planning; the valid finding is the deploy coverage. Net: banking explicit plans raises the deployable coverage ceiling, but does not install the step-1 lookahead the base forward pass lacks (C25/C26).
Next
- Length-vs-content control (verbose correct plan padded to ~3000 chars): is it the rambliness or the post-hoc-ness that makes the model's own thoughts inert? Add token-matched A + a seed.
Artifact Manifest
See reports/artifact_manifest.yaml. Adapters (~180MB each) moved out of repo.
Experiment log 1
Show the running log (1 entry)
Scaffold
Created as a new experiment scaffold.
Figures 2
Data files 15
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
runs/eval_deploy_A_nt.json9.7 kBruns/eval_deploy_T_nt.json9.7 kBruns/eval_deploy_T_th.json9.8 kBruns/eval_deploy_Tcorrupt_th.json10 kBruns/eval_deploy_base_nt.json10 kBruns/eval_p2_Aself_nt.json9.7 kBruns/eval_p2_Tself_th.json9.8 kBruns/eval_p2_Tsynth_th.json9.8 kBruns/results_s1_A.json288 Bruns/results_s1_T.json151 Bruns/results_s1_base.json306 Bruns/results_s1p2_Aself.json254 Bruns/results_s1p2_Tself.json249 Bruns/results_s1p2_Tsynth.json250 Bruns/results_s1p2_base.json251 B
Reproduce
Smoke test
python scripts/synth_traces.py --n 16 && python scripts/build_train.pyFull run
python scripts/synth_traces.py --n 256 && build_train + train A/T/Tcorrupt + deploy/step1 evals + analyzeRun steps are documented inside the experiment folder (README and scripts).