Qwen3.5-4B Thinking Content vs Compute
The one idea you need
Treat the model's written thinking as a recipe it cooks from, not a warm-up that merely revs its engine. A blank card, or the ingredients listed in scrambled order, changes the meal not at all; a clear recipe helps; another dish's recipe cooks the wrong meal.
The question
When a small model writes out its reasoning before answering, does that help because it truly reasons, or just because generating extra text warms up its computation?
What we found
It genuinely reasons; the boost is content, not compute. Blank filler of the same length, and the real thinking scrambled into nonsense, both scored like skipping thinking entirely, around 74 to 75 percent. Only coherent thinking lifted accuracy to 86 percent. And splicing in another task's reasoning crashed it to 4 percent, because the model solves whatever problem its notes describe.
Why it matters
Don't add reasoning tokens just to buy compute; padding and scrambled notes gain nothing here. Guard the scratchpad: a routing or retrieval bug that injects stale or off-topic reasoning is catastrophic, not harmless filler, because the model faithfully follows it.
On this page
Results at a glance 3
How to read
Five bars, one per thinking condition along the bottom (no thinking, filler dots, shuffled thinking, real thinking, another task's thinking), all matched to the same thinking length. Height is the share of Python tasks fully passed across eight tries; taller is better.
Takeaway → The first three bars sit together near three-quarters; only real thinking rises to about 86 percent, while foreign thinking collapses to a 4 percent stub, so content drives the result, not length.
Data table
| thinking condition (matched thinking length) | full-test pass (k=8) |
|---|---|
| no thinking | 74.9% |
| filler dots | 74.4% |
| shuffled thinking | 73.9% |
| real thinking | 86.1% |
| foreign thinking | 4% |
Numbers from experiments/qwen35_4b_thinking_content_vs_compute/analysis/decomposition.md
Technical framing
MBPP pass rate by thinking condition: only real thinking helps, foreign thinking is catastrophic — Filler and shuffled thinking match no-thinking; only coherent real thinking gains +11pp, and another task's thinking collapses to 4%.
How to read
Four bars, each the change in pass rate from adding one ingredient in turn: extra compute, having relevant words present, putting them in coherent order, and swapping in wrong-task content. Bars above zero help; bars below zero hurt.
Takeaway → Extra compute and scrambled words hover at zero; coherent order alone lifts about 12 points, the entire gain, while wrong content plunges 71 points below baseline, toxic rather than inert.
Data table
| ingredient isolated by each ladder step | change in full-test pass |
|---|---|
| pure compute + scaffold | -0.5% |
| token presence / relevance | -0.5% |
| coherent content | 12.2% |
| misleading content | -70.9% |
Numbers from experiments/qwen35_4b_thinking_content_vs_compute/analysis/decomposition.md
Technical framing
Attributing the thinking gain: coherent content is the entire effect — Extra compute and scrambled relevant tokens buy nothing; coherent reasoning content is 100% of the gain, and wrong content is actively harmful.
How to read
The line tracks how much coherent thinking beats the same words scrambled (vertical) as the thinking-token budget rises left to right across three settings, from about 500 to 2000 tokens. Higher means coherence matters more.
Takeaway → The line climbs from roughly 10 to 15 points, so more room to think widens coherence's advantage rather than diluting real reasoning into mere compute.
Data table
| thinking budget (tokens) | real - shuffled pass gap |
|---|---|
| 512 | 10.5% |
| 1024 | 10.8% |
| 2048 | 15% |
Numbers from report table (Refinement section of reports/report.md)
Technical framing
Coherence advantage (real minus shuffled) grows with thinking budget — Follow-up sweep refuted the overthinking prediction: the coherent-content advantage grows, not shrinks, at higher budgets.
In the author’s words from the Overview · “Results”
Behavioral ladder (full-pass, n=100) (table on the experiment page). Complete attribution (additive ladder no_think → filler → shuffle → real): pure compute + scaffold (filler − no_think): −0.005 — contentless “.” tokens buy nothing. token-presence / relevance (shuffle − filler): −0.005 — scrambled relevant tokens buy nothing. coherent content (real − shuffle): +0.122 — the entire gain. misleading content (foreign − no_think): −0.709 — the model follows foreign thinking to the wrong problem (verified: a string task + a matrix-sort thought emits sort_matrix). So at the efficient 512 budget on MBPP, 100% of the behavioral thinking gain is coherent reasoning content — not compute, not scaffold, not token-presence. … Read the full result →
Overview
Research Program
- Program:
test_time_reasoning_budget - Program question: of the native-thinking gain, how much is coherent reasoning vs token-presence/ relevance vs pure compute + scaffold?
- Prior anchors:
qwen35_4b_thinking_budget_scaling(shuffle control: coherent order reproduced part of the gain) andqwen35_4b_thinking_separability_probe(shuffled ≈ real in separability). Both kept the same thinking tokens; neither removed relevance — this experiment does.
Question
The shuffle control destroys coherent order but keeps the same thinking-token multiset (relevant variable names, values, operations stay in context). This experiment adds the foreign-task-thinking control — splice a different task's thinking into this task — which removes relevance/token-presence while keeping count + scaffold + compute. Where does foreign land on the ladder?
no_think → foreign → shuffle → real
+compute +relevance +coherent
+scaffold (presence) order- foreign ≈ real ≈ shuffle → thinking is pure compute + format; even irrelevant thinking helps as much.
- foreign ≈ no_think (≪ shuffle) → relevance/token-presence is the active ingredient (just not order).
Hypothesis
Given C9 (order doesn't matter), the open question is whether the relevant tokens matter. If foreign collapses to (or below) no_think while shuffle stays high, the thinking benefit is "the relevant tokens being in context," not reasoning and not pure compute.
Setup
- Model Qwen3.5-4B frozen (bf16, fast path). MBPP sanitized
test, 100 tasks, k=8, thinking budget 512. - Real thinking generated once (capturing its thinking tokens); shuffle (permute those tokens) and foreign (a cyclically-shifted other task's thinking tokens, same sample slot) reuse the tokens and regenerate only the answer from the modified prefix — so all conditions share the same thinking-token multiset and matched thinking length (compute).
- Measures: behavioral full-test pass (k=8) AND per-layer answer-token separability probe (as in the separability experiment: right-padded activations, GroupKFold-by-task logistic, bootstrap CI, shuffled-label control).
Run
HF_HUB_OFFLINE=1 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
../../.venv/bin/python scripts/run.py --tasks 100 --k 8 --budget 512
../../.venv/bin/python scripts/verify.py # execution labels
../../.venv/bin/python analysis/probe.py # per-layer separability
../../.venv/bin/python analysis/decompose.py # behavioral + separability ladderResults
Full results in reports/report.md. Behavioral ladder (full-pass, n=100):
| no_think | filler | shuffle | real | foreign |
|---|---|---|---|---|
| 0.749 | 0.744 | 0.739 | 0.861 | 0.040 |
Complete attribution (additive ladder no_think → filler → shuffle → real):
- pure compute + scaffold (filler − no_think): −0.005 — contentless
.tokens buy nothing. - token-presence / relevance (shuffle − filler): −0.005 — scrambled relevant tokens buy nothing.
- coherent content (real − shuffle): +0.122 — the entire gain.
- misleading content (foreign − no_think): −0.709 — the model follows foreign thinking to the wrong problem (verified: a string task + a matrix-sort thought emits
sort_matrix).
So at the efficient 512 budget on MBPP, 100% of the behavioral thinking gain is coherent reasoning content — not compute, not scaffold, not token-presence. Separability is noisy (overlapping CIs; the foreign AUC is an imbalance artifact), so only the behavioral ladder is robust.
Interpretation
This conclusively corrects the earlier "much of the gain is compute/scaffold, not reasoning": pure compute (filler) contributes ~0. At the efficient budget the gain is entirely coherent reasoning the model uses. The "compute/scaffold" reading survives only at high budgets (overthinking) and in the noisy decodability slice. Next: a high-budget ladder to confirm the coherence advantage shrinks under overthinking.
Artifacts
src/ladder_lib.py(gen_real captures thinking tokens; gen_answer regenerates answers; activations),src/tasks.py.scripts/run.py,scripts/verify.py.analysis/probe.py,analysis/decompose.py.data/records.jsonl,data/labels.jsonl,data/tasks.json(small, in-repo).- Activations in
large_artifacts/qwen35_4b_thinking_content_vs_compute/(external, gitignored, regenerable).
Report
Rendered from reports/report.md
Summary
A full content/compute decomposition of the native-thinking gain — and the definitive correction of the earlier "much of the gain is compute/scaffold, not reasoning." On a tight ladder where all thinking conditions share matched thinking length (and the relevant ones share the same thinking-token multiset), MBPP full-test pass is:
| no_think | filler | shuffle | real | foreign |
|---|---|---|---|---|
| 0.749 | 0.744 | 0.739 | 0.861 | 0.040 |
filler ≈ shuffle ≈ no_think, real is +12pp above all of them, and foreign collapses to 4%. So at the efficient (512) budget, 100% of the behavioral thinking gain is coherent reasoning content: pure forward compute (contentless . filler) buys ~nothing (−0.005), relevant-but-scrambled tokens buy ~nothing (−0.005), coherent order/content is the entire +0.122, and misleading content (another task's thinking) is catastrophic (−0.709) because the model follows it to the wrong problem.
Research Program Fit
Fourth experiment of test_time_reasoning_budget; the decisive content control + filler arm. It revises C9: the earlier "mostly compute/scaffold" reading was a greedy-metric artifact that held mainly at high budgets (2048 shuffle ≈ real, overthinking) and at the representational level; the filler arm shows pure compute contributes ~0, so at the efficient budget the gain is reasoning content.
Method
- Model Qwen3.5-4B frozen (bf16, fast path). MBPP sanitized
test, 100 tasks, k=8, thinking budget 512. Real thinking generated once (capturing its thinking tokens). The other arms reuse those tokens / matched length and regenerate ONLY the answer from the modified prefix:
- filler: that example's real-thinking length worth of contentless
.tokens (pure compute + scaffold). - shuffle: the real thinking tokens, permuted (relevant content, scrambled order).
- foreign: a cyclically-shifted other task's thinking tokens (misleading content).
- real: the model's own coherent thinking. no_think: enable_thinking=False.
- filler: that example's real-thinking length worth of contentless
- Behavioral full-test pass + per-layer answer-token separability probe (right-padded activations, GroupKFold-by-task logistic, bootstrap CI, shuffled-label control).
Results
| rung | full-pass | visible-pass | probe AUC | isolates |
|---|---|---|---|---|
| no_think | 0.749 | 0.806 | 0.646 | baseline |
| filler | 0.744 | 0.789 | 0.703 | pure compute + scaffold (contentless .) |
| shuffle | 0.739 | 0.781 | 0.645 | relevant tokens, scrambled order |
| real | 0.861 | 0.902 | 0.722 | relevant tokens, coherent order |
| foreign | 0.040 | 0.041 | 0.987\* | misleading content (another task's thinking) |
\* foreign probe AUC is a class-imbalance artifact (33/800 passes).
Behavioral attribution (additive ladder no_think → filler → shuffle → real):
- pure compute + scaffold (filler − no_think): −0.005
- token-presence / relevance (shuffle − filler): −0.005
- coherent order / content (real − shuffle): +0.122
- total (real − no_think): +0.112
- [off-ladder] misleading content (foreign − no_think): −0.709
Finding 1 — pure compute buys nothing
Contentless . filler (matched to each example's real thinking length) lands at 0.744 ≈ no_think 0.749. The "dot-by-dot" extra-compute effect does not appear for this 4B on MBPP; the <think> scaffold + extra forward passes alone add nothing.
Finding 2 — the model uses thinking as CONTENT
Foreign thinking collapses full-pass to 0.040 — far below no-think — by following the foreign reasoning to the wrong problem (verified: task remove_Occ fed a matrix-sort thought emits sort_matrix). The model conditions its answer on the thinking content, so irrelevant content is actively harmful, not inert.
Finding 3 — only coherent content helps; it is the entire gain
Scrambled relevant thinking (shuffle 0.739) ≈ filler ≈ no_think — token-presence without coherent order buys nothing on sampled full-pass. Coherent order (real 0.861) adds +12pp and accounts for the whole gain. So at the efficient budget the thinking benefit is genuine coherent reasoning content.
Finding 4 — representational side is noisy
Best-layer separability AUCs (no_think 0.646, filler 0.703, shuffle 0.645, real 0.722) have heavily overlapping CIs; point estimates hint filler ≈ real > no_think ≈ shuffle but this is within noise. The foreign AUC is a degenerate artifact. Only the behavioral ladder is robust.
Controls
The ladder is the control structure: filler isolates pure compute, foreign isolates misleading content, shuffle isolates relevance, real adds coherent order. Shuffled-label probes ≈ 0.44–0.56 (no leakage). The complete attribution leaves no unexplained component — earlier the missing piece was filler (the only contentless arm); it is now run and ≈ 0.
Oracle Versus Deployable Evidence
Behavioral full/visible-pass are deployable; the probe is a non-deployable, here-inconclusive decodability diagnostic.
Interpretation
This conclusively overturns the program's earlier sharpest claim. At the efficient budget on MBPP, the thinking accuracy gain is entirely coherent reasoning content — not compute (filler ≈ baseline), not scaffold, not token-presence (shuffle ≈ baseline) — and the model genuinely uses that content (foreign → wrong problem). The "thinking ≈ compute/scaffold" reading survives only as the high-budget regime (the scaling experiment's 2048 shuffle ≈ real overthinking result) and as the noisy representational slice. The honest, complete picture for the program: thinking is a real deployable lever (scaling), cheaply routable (controller), and — at the efficient budget — genuine reasoning the model uses (this experiment), whose advantage washes out under overthinking and is not clearly mirrored in internal correctness-decodability.
Limitations
- Efficient budget (512) only; the coherence advantage is expected to shrink at high budgets (overthinking — confirm with a high-budget ladder). MBPP is basic, likely partly contaminated, n=100 single seed. Minor protocol asymmetry: real's answer is its original generation; filler/shuffle/foreign answers are regenerated from the modified prefix (same prefix → same answer distribution).
Next Experiments
- High-budget (1024/2048) ladder to confirm the coherence advantage shrinks under overthinking.
- Contamination-controlled / harder substrate where the no-think baseline is weaker (more headroom).
- A learned controller using internal/uncertainty signals vs the visible-test C2 wall.
Artifact Manifest
See artifact_manifest.yaml. Activations (~0.65 GB, 5 conditions) external/regenerable; records, labels, decomposition, probe results, and figure in-repo.
Refinement (added after the budget sweep)
This report says the coherence advantage is "expected to shrink at high budgets (overthinking)" and that the compute reading survives as the high-budget regime. The follow-up budget sweep (qwen35_4b_overthinking_content_ladder) refuted this: the coherence advantage real − shuffle does not shrink but grows with budget (+0.105 → +0.108 → +0.150 at 512/1024/2048), and pure-compute filler ≈ no-think at every budget. So the gain is coherent reasoning content at all budgets; the "compute" reading only ever appeared through a greedy-metric lens and the scaling run's shuffle-protocol artifact. See claim C9 (corrected).
Experiment log 5
Show the running log (5 entries)
Scaffold
Fourth experiment of test_time_reasoning_budget. Adds the foreign-task-thinking control the separability report flagged as the decisive next test (remove relevance/token-presence, not just order).
Design / method notes
- Ladder no_think -> foreign -> shuffle -> real at a fixed budget (512). Real thinking is generated ONCE (
gen_realcaptures the thinking-region tokens); shuffle permutes those tokens, foreign uses a cyclically-shifted other task's thinking tokens (same sample slot); both regenerate ONLY the answer (gen_answer) from the modified prefix. So all conditions share one thinking-token multiset and matched thinking length — only relevance (shuffle vs foreign) and order (real vs shuffle) vary. - Behavioral full-pass + per-layer answer-token separability (reusing the separability experiment's right-padded activation extraction + GroupKFold-by-task logistic probe + shuffled-label control).
- Attribution (behavioral full-pass): compute+scaffold = foreign - no_think; relevance = shuffle - foreign; order = real - shuffle.
Smoke
4 tasks x k=2, budget 512: ladder generated + activations (8,33,2560) + verify ran. Tiny-n hint (NOT evidence): foreign full-pass 0.000 vs shuffle/real 1.000, no_think 0.875 — suggesting relevance may matter a lot (foreign possibly below no_think). Needs the full run.
Results (see reports/report.md)
Behavioral ladder (full-pass): no_think 0.764, foreign 0.043, shuffle 0.739, real 0.859. Foreign collapses (spot-check: task remove_Occ fed a matrix-sort thought -> emits sort_matrix, i.e. solves the WRONG problem). Decomposition: irrelevant-content (foreign-no_think) -0.721; relevance (shuffle-foreign) +0.696; coherent order (real-shuffle) +0.120. So the model uses thinking as content, and the efficient-budget gain is coherent reasoning over relevant content — CORRECTING the earlier "mostly compute/scaffold" claim. Separability noisy (no_think 0.682, shuffle 0.636, real 0.676, overlapping CIs; foreign 0.994 is a 34/800-imbalance artifact). Robust result is behavioral.
Probe.py fix logged: the copied analysis/probe.py initially pointed ACTS at the sibling separability experiment's large_artifacts dir (loaded wrong no_think, crashed on acts_foreign); fixed to this experiment's dir.
Filler arm (added)
Added the filler/pause-token arm via scripts/add_filler.py (reuses the existing data: contentless "." tokens matched per-example to real thinking length, answer-only regeneration). Complete behavioral attribution (no_think 0.749, filler 0.744, shuffle 0.739, real 0.861, foreign 0.040): pure compute (filler-no_think) -0.005; relevance (shuffle-filler) -0.005; coherent content (real-shuffle) +0.122; misleading content (foreign) -0.709. So pure compute buys NOTHING — the efficient-budget gain is 100% coherent reasoning content, conclusively correcting the earlier "mostly compute/scaffold" claim. Separability remained noisy (overlapping CIs; foreign AUC artifact).
Figures 1
Data files 1
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
runs/probe_results.json3.8 kB
Reproduce
Smoke test
../../.venv/bin/python scripts/run.py --smokeFull run
HF_HUB_OFFLINE=1 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True ../../.venv/bin/python scripts/run.py --tasks 100 --k 8 --budget 512Run steps are documented inside the experiment folder (README and scripts).