Qwen3.5-4B Activation Steering: is the latent first-op causally usable?
The one idea you need
Picture the model keeping working notes on an internal notepad. A simple reader can read one note almost perfectly — which operation it plans to do first. Steering writes that same note back in, bolder. But reading a note and getting the model to obey it are separate channels.
The question
If a simple reader can see which step the model plans to do first, can we push that same signal back in to make it actually do it?
What we found
No. A simple reader picks the model's planned first operation out of its internal state almost perfectly — 99% of the time — yet pushing that exact signal back in during generation barely moves what the model does: at best 3 to 5 points of improvement, no better than pushing a meaningless random direction, and far under the 10-point bar set in advance. Even handing it the known-correct answer failed.
Why it matters
If a probe shows your small model internally "knows" something, don't assume you can elicit it by adding that direction back to its activations at inference — that failed even with the true answer handed in. Real capability needs weight training or a tool.
On this page
Results at a glance 2
How to read
Two clusters of bars — a clean setting where the reader finds the step 99% of the time, and a harder one (42%). Each has four bars: no steering, steer toward the true first step, toward a wrong one, toward random. Taller naming accuracy is better.
Takeaway → Within each cluster all four bars sit at nearly the same height; steering toward the true signal never beats no-steering or the random control — the injection does essentially nothing.
Data table
| condition | baseline (no steer) | steer → TRUE first-op | steer → wrong op | steer → random |
|---|---|---|---|---|
| depth 1 (probe 0.99) | 32.7% | 36% | 32% | 35.3% |
| depth 2 (probe 0.42) | 12% | 17.3% | 11.3% | 14.7% |
Numbers from
Technical framing
Steering the decodable first-op direction does NOT move behavior (best coef) — C19 showed the first op is linearly DECODABLE (depth-1 probe 0.99). Adding that same mean-difference direction back to the residual stream during generation does NOT make the model use it: at depth 1 steering toward the true op (0.36) is no better than baseline (0.33) or random (0.35); at depth 2 a faint whiff (0.17 vs 0.12) is within noise of the random control and below the pre-registered +0.10 bar. Decodability != steerability -- the latent signal is readable but not writable into behavior. Only weight edits (banking C18) and tools (C12) move the wall.
How to read
On the harder setting, three lines track naming accuracy as steering strength rises left to right: pushing toward the true step, a wrong one, or a random direction. The no-steer baseline sits at 12%. Higher, and pulling above the others, is better.
Takeaway → All three lines overlap in the same low band, then sag together past mid-strength as fluency breaks down — no strength setting makes the true direction win.
Data table
| steering coefficient | steer → TRUE | steer → wrong | steer → random |
|---|---|---|---|
| 2 | 16% | 12.7% | 12% |
| 4 | 16.7% | 11.3% | 13.3% |
| 6 | 16.7% | 12.7% | 14.7% |
| 8 | 17.3% | 11.3% | 14.7% |
| 12 | 14.7% | 7.3% | 12.7% |
| 20 | 13.3% | 7.3% | 8.7% |
Numbers from
Technical framing
Depth-2 naming vs steering coefficient (all conditions flat/overlapping) — steer_true sits marginally above steer_random and steer_wrong at low coef (a directional whiff) but the gap is within noise; beyond coef ~12 fluency degrades and all conditions fall. No usable steering effect. Baseline (no steer) = 0.12.
In the author’s words from the Overview · “Results”
INERT — decodability ≠ steerability. Depth 1 (probe 0.99): steer_true never beats baseline (max +0.03), degrades at high coef. Depth 2: faint whiff (+0.05, within noise of random, below the +0.10 bar). Null at earlier layers (8, 12) and on identification (0.03→0.03). All pre-registered predictions refuted. See reports/report.md, analysis/steering.png, runs/steer_results.json.
Overview
Research Program
- Program:
structured_execution_and_compilers - Program question: C19 found the first op is linearly decodable. Is it causally usable — can we steer it out of the residual stream (training-free) and make the model use it?
- Prior anchors: C19 (latent representation), C17 (selection is free), C18 (banking installs capability).
Question
Add the decoded "correct first-op" direction back to the residual stream during generation (ActAdd). Does the model then name / use it?
Hypothesis
Pre-registered (reports/prereg.md): steer_true ≥ baseline + 0.10 (usable); steer_wrong < baseline and steer_random ≈ baseline (specific); depth-1 sanity ≥ +0.15; identification lift (the prize).
Setup
- Model: Qwen3.5-4B (only permitted model). Inference only + a forward hook.
- Directions: mean-difference
d_c = mean(acts[first_op==c, L]) − mean(acts[all, L])from C19's cached activations, at the C19 probe-best layer (depth-1 L15, depth-2 L22). - Steering: forward hook on
model.model.layers[L−1]addscoef · d_cto the residual at all positions. - Tasks: fresh verified
list, depth 2 (primary) + depth 1 (sanity), n=150, disjoint from C19. - Readout: forced-answer first-op naming (fast, baseline parse ≈ 1.0). Conditions {baseline, steer_true, steer_wrong, steer_random} × coef {2,4,6,8,12,20}. Secondary: identification pass@1.
Run
Smoke: python scripts/steer.py --n 30 --coefs 0 6 12 --depths 1 Full: python scripts/steer.py --n 150 --coefs 0 2 4 6 8 12 20 --depths 2 1 && python scripts/analyze.py
Results
INERT — decodability ≠ steerability. Depth 1 (probe 0.99): steer_true never beats baseline (max +0.03), degrades at high coef. Depth 2: faint whiff (+0.05, within noise of random, below the +0.10 bar). Null at earlier layers (8, 12) and on identification (0.03→0.03). All pre-registered predictions refuted. See reports/report.md, analysis/steering.png, runs/steer_results.json.
Interpretation
The latent signal C19 found is readable but not writable into behavior via standard ActAdd. Strengthens the throughline: test-time interventions (selection C17, steering C20) don't move the wall; only weight edits (banking C18) and tools (C12) do. Limit: a clean negative for mean-difference steering; patching / optimized vectors untested.
Knowledgebase Update
- Program evidence updated:
research_programs/structured_execution_and_compilers/evidence.md(C20) - Claim ledger updated: C20 added
Artifacts
scripts/steer.py(ActAdd hook + readouts),scripts/analyze.pyruns/steer_results.json(naming sweep, ident arm, earlier-layer supplementary),analysis/steering.png- Directions built from C19's
scratchpad/probe_artifacts/acts.npy(external) + the probe experiment'slabels.json
Report
Rendered from reports/report.md
Summary
C19 found the composition's first operation is linearly DECODABLE from the residual stream far above the model's behavioral access (depth-1 probe 0.99 vs naming 0.44), but flagged that "decodable ≠ usable for generation." This is the decisive follow-up: add the decoded "correct first-op" direction back to the residual stream during generation (training-free ActAdd steering) — does the model then USE it? No. Decodability ≠ steerability. The pre-registered usability threshold is not met at any coefficient, layer, or depth.
- Depth 1 (the cleanest direction, probe 0.99): steering toward the true first-op never exceeds the no-steer baseline (0.33); at higher coefficients all conditions (true / wrong / random) decline together as fluency degrades. No usable effect where the signal is near-perfectly present.
- Depth 2: a faint, predicted-direction whiff —
steer_true(0.16–0.17) sits ~+0.05 above baseline (0.12) and abovesteer_wrong(0.07–0.13) across coefs 2–8 — but the gap over thesteer_randomcontrol (0.12–0.15) is ~0.02–0.03, within noise at n=150, and far below the pre-registered +0.10 bar. - Robustness: the null holds at earlier steering layers (8, 12) as well as the probe-best layer (22), and the identification arm is flat (0.03 → 0.03).
Verdict: INERT. The latent first-op signal is readable but adding it back does not make the model act on it — at least not via the standard mean-difference method.
Research Program Fit
Direct causal follow-up to C19 (the wall's representation). Tests whether the mission's "cleverer access" dream — steer out latent capability without training — works. It does not, which sharpens which levers can actually move deployable capability.
Method
Reuse C19's cached activations to build ActAdd directions; test on FRESH held-out tasks with a forward hook.
- Directions: per first-op class c,
d_c = mean(acts[first_op==c, L]) − mean(acts[all, L])at the C19 probe-best layer L (depth-1 L15, depth-2 L22), from C19's cached activations. - Steering: a forward hook on
model.model.layers[L−1]addscoef · d_cto the residual at all positions, every generation step. Per-example vector = the task's own true/wrong/random direction. - Tasks: fresh verified
listtasks (depth 2 primary, depth 1 sanity), n=150, disjoint from C19. - Readout — first-op naming: the prompt is forced to answer directly (
First:suffix, no-think, greedy) — a fast, clean readout (baseline parse ≈ 1.0). Conditions × coef ∈ {2,4,6,8,12,20}. - Conditions:
baseline(no steer),steer_true,steer_wrong(a random wrong op),steer_random(Gaussian vector scaled to ‖d_true‖). - Secondary — identification pass@1 (no-think, greedy): baseline vs
steer_trueat the best coef.
Results
| depth | baseline | best steer_true (coef) | steer_wrong | steer_random | max(true − baseline) |
|---|---|---|---|---|---|
| 1 | 0.33 | 0.36 (4) | 0.32–0.38 | 0.35 | +0.03 |
| 2 | 0.12 | 0.17 (8) | 0.07–0.13 | 0.12–0.15 | +0.05 |
Identification (depth 2, coef 8, no-think): baseline 0.027 → steer_true 0.027. Earlier layers (depth 2, coef 8): layer 8 true 0.15 / wrong 0.13 / random 0.13; layer 12 true 0.15 / wrong 0.17 / random 0.13 — no effect.
Pre-registered verdicts
- P1 (causal usability, steer_true ≥ baseline + 0.10): REFUTED — max gain +0.05 (depth 2), +0.03 (depth 1).
- P2 (specificity): PARTIAL/weak — depth-2
steer_wrongdoes dip below baseline (directionally correct), butsteer_random≈ baseline and thesteer_true−steer_randomgap is within noise; no clean specific effect. - P3 (depth-1 sanity, ≥ +0.15): REFUTED — steering the cleanest (0.99-decodable) direction moves naming by ≤ +0.03 and only degrades at higher strength.
- P4 (identification lift): REFUTED — 0.027 → 0.027.
All predictions refuted ⇒ INERT, with at most a marginal, non-significant directional whiff at depth 2.
Interpretation
- Decodability ≠ steerability. A linear probe reads the first op off the residual stream (C19), but adding that same direction back does not route it into behavior. The representation-expression gap C19 found is not bridged by simple linear intervention — the "unexpressed" information is not trivially writable into the output pathway.
- Strengthens the arc's throughline from a new angle. Test-time interventions keep failing to move the wall: sampling+selection is free but adds no coverage (C17); activation steering can't elicit the latent signal (C20). The only things that move deployable capability remain weight edits (banking, C18) and externalization (tools, C12) — installing the capability, not reading it out.
- Refines the C19 "latent" reading honestly. C19's latent signal is real but inert under this intervention: present-and-readable ≠ present-and-usable. The mission's "steer it out for free" hope does not pan out with the standard method.
Honesty notes / limits
- This is a negative for ONE steering method (mean-difference / ActAdd, single layer, all positions). More sophisticated steering — optimized/causal directions, multi-layer, activation patching rather than addition, or larger interventions with fluency repair — is not ruled out. The claim is that the simplest, most-cited method fails cleanly, including on the near-perfectly-decodable depth-1 direction.
steer_trueis an ORACLE (uses the known answer) — an upper bound on steering usefulness. Even the oracle fails, so deployable self-steering (toward the probe's prediction) is not worth pursuing here.- The identification arm is underpowered (no-think baseline floored at ~0.03); naming is the primary readout.
Next Experiments
- Activation patching (replace rather than add the class-c subspace) or optimized steering vectors (gradient-tuned to change the output, not the probe) — does a stronger intervention move it? A single negative for ActAdd doesn't fully close the door.
- Probe the C18 banked model: the complementary test — does banking raise the first-op probe (install the representation)? If yes, banking works by adding what steering can't inject.
Artifact Manifest
See reports/artifact_manifest.yaml. Key: scripts/steer.py, scripts/analyze.py, runs/steer_results.json, analysis/steering.png, reports/prereg.md. Steering directions are built from C19's cached activations (scratchpad/probe_artifacts/acts.npy, external) + qwen35_4b_latent_composition_probe/data/labels.json.
Experiment log 1
Show the running log (1 entry)
Scaffold
Created as a new experiment scaffold.
Figures 1
Data files 1
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
runs/steer_results.json3.6 kB
Reproduce
Smoke test
python scripts/steer.py --n 30 --coefs 0 6 12 --depths 1Full run
python scripts/steer.py --n 150 --coefs 0 2 4 6 8 12 20 --depths 2 1 && python scripts/analyze.pyRun steps are documented inside the experiment folder (README and scripts).