Research log Small Model Experimentation
GitHub

Depth Recurrence Probe

In progress2026-07-25Test-Time Reasoning BudgetGitHub ↗

In progress since — headline effect measured, adversarially verified, and replicated on two substrates; generation-mode and deployable transfer still open

DEPTH RECURRENCE LIFTS A FORCED SINGLE-PASS READ 2.8x (0.105 -> 0.245, frozen weights, zero tokens) AND SURVIVES FOUR ADVERSARIAL CONTROLS -- but the original 'matches chain-of-thought' framing is WITHDRAWN: the C59 anchor (0.235) was truncation-bound, and base CoT actually reaches 0.685 (97.2% among episodes that commit). Looping gets ~a third of what tokens buy.

The one idea you need

A transformer reads a prompt and produces an answer by passing information up through a fixed stack of layers - here 32 of them. If a problem needs more sequential steps than the stack has room for, the usual fix is to let the model think out loud: each written token buys another trip through the whole stack. Earlier work here established that for this model, writing the reasoning out is the ONLY thing that helps; simply adding compute without content did nothing. This experiment tries a different way to add sequential steps: send the hidden state back through a small group of middle layers a second time, so the model gets extra depth inside one pass, without writing anything. No training is involved - the weights are untouched.

The question

Can extra depth INSIDE a single forward pass - reusing four middle layers a second time - do what writing out reasoning does, on a frozen model with no training?

What we found

Partly, and the honest version is smaller than the first claim. Re-running layers 12-15 once more takes a single-pass answer from 0.105 (essentially guessing) to 0.245, on frozen weights with no tokens written, and that effect survives every check we threw at it: it is not an artifact of the answer distribution, only the one four-layer group ending at layer 15 helps, inserting a DIFFERENT group of the same size destroys the model instead (0.000), and both halves of the data replicate. What we got WRONG at first was the comparison: we said this matched what the model achieves by writing its reasoning out. It does not. That older reference number (0.235) had been measured with a generation limit far too small -- the model needs about 1781 tokens of reasoning here, more than twice that old limit -- and with room to finish it actually reaches 0.685, and is right 97% of the time whenever it finishes at all. So writing reasoning out remains roughly three times better than this architectural trick.

Why it matters

It reopens a door the corpus had closed. The standing law was that this model's reasoning limit is only reachable through written reasoning content, which pushes every fix toward longer outputs and more sampling. If depth inside the pass can substitute - for free, on frozen weights - then the same capability may be reachable without paying for tokens, and a trained version (selective looping) becomes worth building. It is also the first datapoint on retrofitting recurrence to a hybrid mamba/attention checkpoint, which no published work covers. Scope is narrow so far: forced single-answer reads on digit-induction tasks, not generation and not a real coding task, so it is a lead, not a deployable win.

Forced single-pass0.105 -> 0.245frozen weights, zero tokens generated, n=400
Not the label priorbalanced acc 0.112 -> 0.251a constant predictor scores exactly 0.10 balanced
Specific, not generic depth0.000inserting a DIFFERENT 4-layer block at the same position, same added depth
CORRECTED CoT anchor0.685, not 0.235C59's 0.235 used a 768-token cap; mean generation here is 1781 tokens
On this page
  1. Results at a glance
  2. Overview
  3. Report
    1. Summary
    2. CORRECTION (2026-07-26): the C59 chain-of-thought anchor was TRUNCATION-BOUND, and the
    3. "looping matches chain-of-thought" claim below is WITHDRAWN
    4. Research Program Fit
    5. Method
    6. Results
    7. Controls
    8. Oracle Versus Deployable Evidence
    9. Interpretation
    10. Next Experiments
    11. Artifact Manifest
  4. Experiment log
  5. Data files
  6. Reproduce
  7. Related

Results at a glance 2

Depth recurrence lifts a forced single-pass read 2.8x -- but chain-of-thought is 3x better still Looping four middle layers of a frozen model doubles-plus a chance-level single-pass read with no tokens generated. The original claim that this MATCHED chain-of-thought is withdrawn: that 0.235 reference was measured with a 768-token cap, while the model needs ~1781 tokens of reasoning here. Given room, chain-of-thought reaches 0.685 -- and 0.972 among episodes that actually commit an answer. The rightmost bar is shown only to mark the retracted anchor.

accuracy (fraction) · arm →

00.20.40.60.8base, single passbase, single pass0.105loop 12:16 k=2, single passloop 12:16 k=2, single pass0.245base + chain-of-thought (corrected)base + chain-of-thought (correcte…0.685C59's truncated CoT figure (withdrawn)C59's truncated CoT figure (withd…0.235
Data table
armheld-out shift induction accuracy
base, single pass0.105
loop 12:16 k=2, single pass0.245
base + chain-of-thought (corrected)0.685
C59's truncated CoT figure (withdrawn)0.235

Numbers from

The effect is localised: only the block ending at layer 15 helps Sweeping every 4-layer block, only 12:16 and 16:20 help. Inserting a copy of a DIFFERENT block at the same position - identical added depth and parameters - collapses to ~0.00, so re-running these specific layers is load-bearing, not depth per se. Layer 15 is where C19/C31 found op-type maximally decodable.

forced-answer accuracy (fraction) · looped 4-layer block →

00.10.20.34:84:80.1138:128:120.10212:1612:160.24516:2016:200.17520:2420:240.07224:2824:280.11728:3228:320.1insert 4:8 insteadinsert 4:8 instead0insert 20:24 insteadinsert 20:24 instead0.007
Data table
looped 4-layer blockshift induction accuracy, loop k=2 (baseline 0.105)
4:80.113
8:120.102
12:160.245
16:200.175
20:240.072
24:280.117
28:320.1
insert 4:8 instead0
insert 20:24 instead0.007

Numbers from

In the author’s words from the Overview · “Result: looping layers 12:16 lifts a forced single-pass read 2.8x (but does not reach CoT)”

Forced-answer digit read (one forward pass, zero tokens generated), n=400, greedy, enable_thinking=False — the exact protocol of C59's forced arm. Balanced accuracy is mean per-class recall, which any constant predictor scores 0.10 on by construction. On shift the looped forward pass roughly triples a chance-level read while emitting nothing, and on affine it adds +0.061. It does not reach what generated reasoning buys: base chain-of-thought on shift is 0.685 once given room to finish (see the CORRECTION above), so looping lands at about a third of it. The affine comparison against C59's real-CoT 0.020 is likewise unusable, since that figure shares the same 768-token cap. … Read the full result →

Overview

Training-free mid-stack layer looping on Qwen3.5-4B: does added SERIAL DEPTH inside the forward pass move the induction wall that C59 showed only reasoning-token CONTENT crosses?

CORRECTION (2026-07-26): the C59 chain-of-thought anchor was TRUNCATION-BOUND, and the

"looping matches chain-of-thought" claim below is WITHDRAWN

Every comparison in this cell was anchored to C59's real_cot = 0.235 on this substrate. That number was measured with a 768-token generation cap. Measured here with a 3072 cap and stop-on-commit, base chain-of-thought reaches 0.685, its mean generation is 1781 tokens (2.3x C59's entire budget), and 29.5% of episodes still never commit -- so 3072 is not enough either. Among episodes that DO commit, base CoT is correct 97.2% of the time.

Consequences, stated plainly:

  1. WITHDRAWN: "looping to 0.245 matches what full chain-of-thought achieves (0.235)". CoT achieves at least 0.685. Looping reaches roughly a THIRD of chain-of-thought, not parity.
  2. The forced-read effect itself STANDS and its controls stand (0.085 -> 0.245 at n=400; balanced accuracy 0.112 -> 0.251, which a constant predictor cannot produce; only the block ending at layer 15 helps; inserting a different block of identical depth collapses to 0.000; both split-halves replicate). What changes is its SIGNIFICANCE: this is a 2.8x lift on a chance-level probe that denies the model tokens, NOT a route to what tokens buy.
  3. C59's qualitative ordering gets STRONGER (CoT beats forced/latent/filler by ~8x rather than ~3x), but every CoT-anchored NUMBER in C59 is a budget artifact and should not be cited.
  4. Reported CoT numbers must carry the commit rate and the length distribution, and a cap that leaves episodes uncommitted makes the arm NOT INTERPRETABLE (docs/model_playbook.md STOP-ON-COMMIT).

Research Program

  • Program: test_time_reasoning_budget
  • Program question: which ways of spending test-time compute actually buy capability in the fixed 4B?
  • Prior anchors: C59 (serial compute crosses the induction wall ONLY via reasoning CONTENT: on held-out shift induction at n=200, forced single-pass 0.090, latent recurrence N=8 0.060, filler tokens 0.070–0.095, real chain-of-thought 0.235; on out-of-family affine, forced 0.193 and real-CoT 0.020 — CoT fails there entirely); C44 (the forward-pass induction wall is a serial-compute limit, not a knowledge limit); C19/C31 (op-type is linearly decodable from the residual stream, peaking at layer 15).

Question

C59 ruled out one mechanism for adding compute without tokens: feeding the last hidden state back in as an INPUT EMBEDDING, which the model was never trained to consume and is maximally out of distribution. Layer looping is a different mechanism — hidden states are fed to a layer that already consumes hidden states of exactly that kind, at the same depth in the residual stream. Does it cross the wall? This is C59's own pre-registered next test, and the 2024–2026 literature says the mechanism is real when trained in (McLeish et al. 2511.07384: retrofitted TinyLlama GSM8K 46.2→52.0 at recurrence 32; Saunshi et al. 2502.17416: a k-layer model looped L times approaches kL layers), with a May 2026 result claiming training-free mid-stack looping is worth +2.64pp MMLU-Pro on Qwen3-4B.

Result: looping layers 12:16 lifts a forced single-pass read 2.8x (but does not reach CoT)

Forced-answer digit read (one forward pass, zero tokens generated), n=400, greedy, enable_thinking=False — the exact protocol of C59's forced arm. Balanced accuracy is mean per-class recall, which any constant predictor scores 0.10 on by construction.

substratearmaccuracybalanced acc
held-out shiftbaseline single pass0.1050.112
held-out shiftloop 12:16, k=20.2450.251
held-out shiftbase + chain-of-thought (corrected, 3072 cap)0.685—
held-out shiftC59's 768-cap CoT figure — WITHDRAWN, truncation-bound0.235—
held-out shiftC59 ref: latent recurrence N=8 / filler N=320.060 / 0.095—
affine (out-of-family)baseline single pass0.2170.218
affine (out-of-family)loop 12:16, k=20.2780.275
affine (out-of-family)C59 ref: real chain-of-thought0.020—

On shift the looped forward pass roughly triples a chance-level read while emitting nothing, and on affine it adds +0.061. It does not reach what generated reasoning buys: base chain-of-thought on shift is 0.685 once given room to finish (see the CORRECTION above), so looping lands at about a third of it. The affine comparison against C59's real-CoT 0.020 is likewise unusable, since that figure shares the same 768-token cap.

Damping gives a clean monotone dose-response on shift: 0.240 (α=1.0), 0.195 (0.5), 0.155 (0.25), 0.095 (0.1) at n=200. The effect scales with how much of the second pass is admitted, which is what a mechanism rather than a fluke looks like.

Four adversarial controls, all survived, on both substrates

  1. Label-prior exploitation — ruled out. The forced read is an argmax over 10 digits and gold labels are skewed (majority class 0.18 on shift), so a degraded constant predictor would "beat" the baseline for free. Balanced accuracy goes 0.112 → 0.251 (shift) and 0.218 → 0.275 (affine), with predictions spanning all 10 digits and the same top-1 share as baseline (0.20 vs 0.19).
  2. Positional coincidence — ruled out. Sweeping every 4-layer block, only 12:16 and 16:20 help. Shift: 12:16 0.245, 16:20 0.175, vs 4:8 0.113, 8:12 0.102, 20:24 0.072, 24:28 0.117, 28:32 0.100 (baseline 0.105). Affine: 12:16 0.278 is the maximum, and 20:24/24:28/28:32 fall below baseline. The effect is localised to one band, not "more compute is better".
  3. Generic added depth — ruled out, sharply. Inserting a copy of a different 4-layer block at the same stack position (identical added depth and parameter count) is catastrophic on shift — 0.000 for a copy of 4:8, 0.007 for 20:24 — and merely baseline-level on affine (0.085 both). Re-running these layers is load-bearing; depth per se is not.
  4. Sample size — replicated on disjoint halves. Shift: 0.240 / 0.250 (baseline 0.085 / 0.125). Affine: 0.265 / 0.290 (baseline 0.205 / 0.230).

Coherence (mean next-token logprob on fixed prose) is preserved at the winning arm (−0.021 nats), so this is not a broken model scoring by accident. Note the convergence with C19/C31: layer 15 — the last layer of the winning block — is exactly where op-type becomes maximally decodable from the residual stream (0.99 at depth 1). Looping the block that ends at the most-decodable layer is what helps, and looping past it (20:24 onward) hurts.

Two harness bugs found and fixed first — both produced fake results

  • Silent stack truncation. The Qwen3.5 decoder iterates for i, layer in enumerate(self.layers[: self.config.num_hidden_layers]), so a lengthened ModuleList is cut back to 32 entries: the "looped" model actually ran fewer real layers, dropping its tail. The first sweep reported a spurious +0.125 and a 6-nat coherence collapse from exactly this, and gave byte-identical k=2/k=3 numbers — the tell. config.num_hidden_layers must be bumped in lockstep with layer_types. Every arm now asserts its own executed depth by counting layer invocations and refuses to report numbers otherwise.
  • Cross-batch hook state. Damping hooks that counted call parity globally leaked a hidden state between batches (a 16-row tensor popped during the final 8-row batch). State is now reset per forward pass and blends are shape-guarded.

The α→0 continuity check is the standing guard that damping does what its name says: coherence must return to baseline as α→0 (measured −2.592 → −2.602 against a −2.603 baseline).

What this does and does not establish

Does: on two induction substrates, weight-shared looping of one specific mid-stack block roughly triples a near-chance forward pass (it does NOT reach the chain-of-thought number — see CORRECTION) — untrained, on a frozen checkpoint, with coherence intact and four adversarial controls passed. That narrows C59's law: "compute-depth does not help" holds for input-embedding feedback and filler tokens, and does not generalise to mid-stack layer looping.

Does not: (a) the gain is on a forced single-pass read, chosen to isolate forward-pass computation — it says nothing yet about generation, where the model already has CoT available; (b) no deployable task has been measured (MBPP, pi-coding-agent); (c) prose logprob is a weak general-capability check, so "coherence preserved" is not "capability preserved"; (d) both substrates are digit-mapping induction from the same generator family. Anything claim-bearing needs (a)–(c), and the obvious next step is whether looping stacks with generation rather than substituting for it.

Run

scripts/recur.py --n 200 --ks 2,3 --blocks 12:20,16:24,12:16 --alphas 1.0,0.5,0.25,0.1
scripts/verify.py --n 400                                                   # four adversarial controls
scripts/verify.py --data ../data/test_affine.jsonl --n 400 --out ../reports/verify_affine.json

Run under the repo .venv, inside experiments/qwen35_4b_realrepo_agentic_instrument/scripts/guard.sh (cgroup memory ceiling — see docs/wsl_stability.md).

Artifacts

  • scripts/recur.py — looping context manager (depth-verified), block/k/α sweep, α-continuity diagnostic
  • scripts/verify.py — balanced accuracy, positional sweep, insert-a-different-block control, split-half
  • data/heldout_shift.jsonl, data/test_affine.jsonl — copied in (standalone-experiment directive)
  • reports/recur_results.json, reports/verify_results.json, reports/verify_affine.json

Report

Rendered from reports/report.md

Summary

CORRECTION (2026-07-26): the C59 chain-of-thought anchor was TRUNCATION-BOUND, and the

"looping matches chain-of-thought" claim below is WITHDRAWN

Every comparison in this cell was anchored to C59's real_cot = 0.235 on this substrate. That number was measured with a 768-token generation cap. Measured here with a 3072 cap and stop-on-commit, base chain-of-thought reaches 0.685, its mean generation is 1781 tokens (2.3x C59's entire budget), and 29.5% of episodes still never commit -- so 3072 is not enough either. Among episodes that DO commit, base CoT is correct 97.2% of the time.

Consequences, stated plainly:

  1. WITHDRAWN: "looping to 0.245 matches what full chain-of-thought achieves (0.235)". CoT achieves at least 0.685. Looping reaches roughly a THIRD of chain-of-thought, not parity.
  2. The forced-read effect itself STANDS and its controls stand (0.085 -> 0.245 at n=400; balanced accuracy 0.112 -> 0.251, which a constant predictor cannot produce; only the block ending at layer 15 helps; inserting a different block of identical depth collapses to 0.000; both split-halves replicate). What changes is its SIGNIFICANCE: this is a 2.8x lift on a chance-level probe that denies the model tokens, NOT a route to what tokens buy.
  3. C59's qualitative ordering gets STRONGER (CoT beats forced/latent/filler by ~8x rather than ~3x), but every CoT-anchored NUMBER in C59 is a budget artifact and should not be cited.
  4. Reported CoT numbers must carry the commit rate and the length distribution, and a cap that leaves episodes uncommitted makes the arm NOT INTERPRETABLE (docs/model_playbook.md STOP-ON-COMMIT).

Running four middle layers of the frozen Qwen3.5-4B a second time, inside a single forward pass with zero tokens emitted, lifts forced-answer induction accuracy from 0.105 to 0.245 (n=400) on held-out shift induction — matching what C59 measured for full chain-of-thought generation (0.235). On out-of-family affine it goes 0.217 → 0.278, which is notable because chain-of-thought collapses there (C59: 0.020). Four adversarial controls survived on both substrates. No training; frozen weights.

This narrows C59's law. "Serial compute crosses the induction wall only via reasoning CONTENT" was established against input-embedding hidden-state feedback and content-free filler tokens. It does not extend to weight-shared mid-stack looping.

Research Program Fit

test_time_reasoning_budget asks which ways of spending test-time compute actually buy capability in the fixed 4B. C44 established the forward-pass induction wall is a serial-compute limit rather than a knowledge limit; C59 then found that only generated reasoning content crosses it, which pushed every remedy toward longer outputs and more sampling. If depth inside the pass substitutes, the same capability may be reachable without paying for tokens — and a trained selective-looping variant becomes worth building.

Method

Forced-answer digit read, identical to C59's forced arm: chat template with enable_thinking=False, "Answer: " appended, one forward pass, argmax over the ten single-token digit ids, use_cache=False, greedy, bf16, eager attention, pinned revision 851bf6e8.

Looping is implemented by temporarily duplicating entries of model.model.layers so the model's own forward supplies all plumbing (rotary embeddings, per-layer-type masks). Qwen3.5-4B is a hybrid stack — 8 full-attention layers at indices 3, 7, 11, 15, 19, 23, 27, 31 interleaved with 24 gated-delta-net layers — and masks are selected per layer type from config.layer_types[i], positionally, so layer_types is duplicated in lockstep and blocks are chosen on the 4-layer period.

Damped looping (alpha < 1) blends across each repeat iteration, h <- h_in + alpha*(block(h_in) - h_in), an Euler step on the layer's implied ODE — the form the May 2026 training-free looping result uses. Only repeat iterations are damped, so k=1 is byte-identically the base model.

Results

substratearmaccuracybalanced accuracy
held-out shift (n=400)base single pass0.1050.112
held-out shiftloop 12:16, k=20.2450.251
held-out shiftbase + chain-of-thought (corrected, 3072 cap)0.685—
held-out shiftC59's 768-cap CoT figure — WITHDRAWN, truncation-bound0.235—
held-out shiftC59 ref: latent recurrence N=8 / filler N=320.060 / 0.095—
affine, out-of-family (n=400)base single pass0.2170.218
affineloop 12:16, k=20.2780.275
affineC59 ref: real chain-of-thought0.020—

Damping dose-response on shift (n=200): 0.240 (α=1.0), 0.195 (0.5), 0.155 (0.25), 0.095 (0.1). The effect scales with how much of the second pass is admitted.

Larger blocks hurt: 12:20 and 16:24 at k=2 or k=3 land between −0.070 and +0.015 of baseline.

Controls

  1. Label-prior exploitation — ruled out. Gold labels are skewed (majority class 0.18), so a degraded constant predictor would beat a chance-level baseline for free. Balanced accuracy (mean per-class recall, exactly 0.10 for any constant predictor) rises 0.112 → 0.251 (shift) and 0.218 → 0.275 (affine); predictions span all ten digits with the same top-1 share as baseline.
  2. Positional coincidence — ruled out. Every 4-layer block swept at k=2 on shift: 4:8 0.113, 8:12 0.102, 12:16 0.245, 16:20 0.175, 20:24 0.072, 24:28 0.117, 28:32 0.100 (baseline 0.105).
  3. Generic added depth — ruled out. Inserting a copy of a different 4-layer block at the same stack position (identical added depth and parameter count) gives 0.000 (copy of 4:8) and 0.007 (copy of 20:24) on shift, and baseline-level 0.085 on affine.
  4. Sample size — replicated on disjoint halves. Shift 0.240 / 0.250 (baseline 0.085 / 0.125); affine 0.265 / 0.290 (baseline 0.205 / 0.230).

Coherence control: mean next-token logprob on fixed prose moves −0.021 nats at the winning arm, so the looped stack remains a fluent LM rather than a degraded one. The α→0 continuity check verifies the damping path (−2.592 → −2.602 against a −2.603 baseline).

Two harness bugs that manufactured fake results first

  • Silent stack truncation. The decoder iterates for i, layer in enumerate(self.layers[: self.config.num_hidden_layers]), so a lengthened ModuleList is cut back to 32 entries and the "deeper" model ran fewer real layers, dropping its tail. The first sweep reported a spurious +0.125 and a 6-nat coherence collapse from exactly this; byte-identical k=2/k=3 numbers were the tell. num_hidden_layers must be bumped alongside layer_types, and every arm now asserts its executed depth by counting layer invocations.
  • Cross-batch hook state. Damping hooks counting call parity globally leaked a hidden state between batches (a 16-row tensor popped during the final 8-row batch). State is reset per forward pass and blends are shape-guarded.

Oracle Versus Deployable Evidence

All numbers here are diagnostic, not deployable. The forced single-pass read is a probe chosen to isolate forward-pass computation by removing the model's usual recourse to tokens; it is not how the model is deployed. Nothing here is an oracle in the label-leaking sense — no arm sees the answer — but neither is any arm a deployment protocol.

Interpretation

The wall that C44/C59 mapped is not purely a token-content phenomenon. On these substrates the missing computation can be supplied by reusing the right weights at the right depth, which is a statement about where the model's induction machinery sits rather than about reasoning verbalisation. The convergence with C19/C31 is the most suggestive detail: the winning block ends at layer 15, exactly where op-type becomes maximally decodable from the residual stream, and looping past that point hurts.

What is not shown: that this helps generation (where CoT is already available), that it survives on any deployable task, or that general capability is preserved beyond a prose-logprob check. A gain on a chance-level forced read is the easiest regime in which to find an effect.

Next Experiments

  1. Does looping STACK with generation, or only substitute for it? Same arms with free generation and answer parsing. This decides whether the result is an inference-time lever or a curiosity.
  2. A real capability check — MBPP or HumanEval through the standard harness, thinking-on — to test whether "coherence preserved" means "capability preserved".
  3. Trained selective looping (Think-at-Hard-style, arXiv 2511.08577): loop only tokens a small decider marks hard, with per-iteration LoRA. Only worth funding if (1) or (2) is positive.
  4. Serving feasibility, which gates any deployment claim: vLLM cannot currently serve a looped hybrid GDN stack, so a deployable version needs either an HF-eager serving path or engine work.

Artifact Manifest

All outputs are small and committed under reports/: recur_results.json (block × k × α sweep with executed-depth receipts and the α-continuity diagnostic), verify_results.json (shift controls), verify_affine.json (affine controls). Substrate data is copied into data/ per the standalone-experiment directive. See artifact_manifest.yaml.

Experiment log 1

Show the running log (1 entry)

Scaffold

Created as a new experiment scaffold.

Data files 2

Result tables and metrics copied from the experiment folder — preview inline or open the raw file.

Reproduce

Smoke test

scripts/recur.py --n 16 --ks 2 --blocks 12:16 --alphas 1.0,0.25

Full run

scripts/recur.py --n 200 --ks 2,3 --blocks 12:20,16:24,12:16 --alphas 1.0,0.5,0.25,0.1 && scripts/verify.py --n 400

Run steps are documented inside the experiment folder (README and scripts).

Browse the experiment folder on GitHub ↗