Qwen3.5-4B: Externalize the Latent Readout (probe-to-prompt)
The one idea you need
Picture reconstructing a lost recipe. The cook's instinct reveals the first step needs something sour — the category — but not the exact amount, two spoons of lime — the value. Given the category, they land it within a few tries; given the value, first try.
The question
A small model's hidden state already seems to know a task's right first step. Can you read it out and feed it into the prompt to solve harder tasks?
What we found
Partly. Writing the concrete first step — exact value and all — into the prompt lifted the single-best-guess solve rate on two-step tasks sixfold, from 3% to 19%, where editing the model's internal state did nothing. But its internals reliably reveal only the step's category, and category alone merely helps it stumble onto answers across several tries — not solve first-shot. The exact value is the real unlock.
Why it matters
When a small model seems to internally know something, don't extract it by editing its internal state — decode the fact into the prompt instead. And decode the concrete value, not just the category; category-only hints merely widen the search without fixing the first answer.
On this page
Results at a glance 2
How to read
Six prompt conditions run along the bottom, from no hint through category-only, the exact step, the model's own decoded guess, to a deliberately wrong hint. One color is single-best-guess solves, the other solved-within-six-tries; taller is better.
Takeaway → Only the exact-step hint (type plus value) raises single-best-guess solving; the category-only hint lifts just the six-try bar, and the wrong hint flattens both to zero.
Data table
| condition | coverage@6 (depth-2) | greedy@1 (depth-2, deployable) |
|---|---|---|
| no-hint | 5% | 3% |
| neutral (placebo) | 5% | 2% |
| oracle TYPE | 19% | 2% |
| oracle FULL (+param) | 31% | 19% |
| probe (C19 readout) | 6% | 1% |
| wrong (content ctrl) | 0% | 0% |
Numbers from
Technical framing
Externalize the latent readout: the FULL first op elicits depth-2 (6x), but the op-TYPE C19 decodes only narrows sampling — C19: the composition first-op is linearly DECODABLE from the base residual far above behavior (depth-2 0.42) but a thread at depth-3. C20: that direction is NOT steerable (ActAdd inert). C17: test-time selection is free but adds no coverage. This EXTERNALIZES the readout: decode the first-op with C19's probe and inject it as a PROMPT hint. RESULT: (1) externalization ELICITS deployable depth-2 where steering failed -- oracle_FULL (true op+param in the prompt) lifts greedy@1 6x (0.03->0.19) and coverage 6x (0.05->0.31), the FIRST test-time intervention in the arc to move deployable capability. (2) But the deployable bottleneck is the PARAMETER, not the op-TYPE C19 decodes: oracle_TYPE lifts COVERAGE (0.05->0.19, narrows sampling) but NOT greedy (0.02); only the full op deploys single-shot. (3) The type-only C19 probe (0.32 eval acc) nets to ~zero. Controls: neutral~=no-hint (format), wrong HURTS (content-causal), layer-0 probe at chance 0.05 (model-computed, not surface). Graded by depth: fades at depth-3 (thread).
How to read
Five prompt conditions along the bottom; height is the single-best-guess solve rate, taller is better. One color is two-step tasks, the other three-step, showing how far each hint carries by task length.
Takeaway → The exact-step hint lifts two-step tasks to 19% but leaves three-step tasks flat near zero — it can cash in latent knowledge but cannot create a step the model never computed.
Data table
| condition | depth-2 greedy@1 | depth-3 greedy@1 |
|---|---|---|
| no-hint | 3% | 0% |
| oracle TYPE | 2% | 1% |
| oracle FULL (+param) | 19% | 1% |
| probe | 1% | 0% |
| wrong | 0% | 0% |
Numbers from
Technical framing
Graded by depth: elicitation works at depth-2 (headroom) and vanishes at depth-3 (thread, C19) — The elicitation is bounded by the representation, exactly as C19 predicts. At depth-2 the first-op is latently present (probe 0.42) so giving the concrete op (oracle_full) lifts deployable greedy@1 6x. At depth-3 the representation is a thread (C19 real signal ~0.13) -- even oracle_full (telling the model the true first op) barely moves it (0.00->0.01). No readout conjures information the forward pass never computed: externalization can cash out latent headroom but cannot cross the true wall.
In the author’s words from the Overview · “Results”
Externalization ELICITS deployable depth-2 where steering (C20) failed -- oracle_full lifts depth-2 greedy@1 6x (0.03->0.19). BUT the deployable bottleneck is the PARAMETER, not the op-TYPE C19 decodes (oracle_type lifts coverage only); the type-only probe nets to ~zero. Graded by depth (fades at depth-3, thread). Controls clean (neutral~=no-hint, wrong hurts, layer-0 at chance). See reports/report.md, analysis/probe_to_prompt.png.
Overview
Research Program
- Program:
interpretability_and_diagnostics/structured_execution_and_compilers - Question: C19 showed the first-op is linearly DECODABLE from the base residual but C20 showed it is NOT steerable. Can we EXTERNALIZE the readout instead -- decode it with the probe and inject it as a PROMPT hint (shift the proposal, the lever C17 allows)?
Setup
- Refit C19's linear first-op probe (standardize+PCA128+L2-logistic) on 1500 training tasks (replicates C19).
- On FRESH fsig-disjoint eval tasks (n=100/depth 2,3), decode the first-op from the base model's own activation; generate under 6 arms: no-hint, neutral(placebo), oracle-type, oracle-full(+param), probe(decoded), wrong(random). Metrics greedy@1 + coverage@6, no-think.
Run
python scripts/fit_probe.py --n-per-depth 500 then python scripts/run_hints.py --n-per-depth 100 --depths 2 3 then python scripts/analyze.py.
Results
Externalization ELICITS deployable depth-2 where steering (C20) failed -- oracle_full lifts depth-2 greedy@1 6x (0.03->0.19). BUT the deployable bottleneck is the PARAMETER, not the op-TYPE C19 decodes (oracle_type lifts coverage only); the type-only probe nets to ~zero. Graded by depth (fades at depth-3, thread). Controls clean (neutral~=no-hint, wrong hurts, layer-0 at chance). See reports/report.md, analysis/probe_to_prompt.png.
Knowledgebase Update
- Claim ledger: C30
Artifacts
scripts/fit_probe.py(fits + saves C19 probe, mid-layer + layer-0),scripts/run_hints.py(6-arm hint experiment + leak control),scripts/analyze.py,scripts/capture.py(from C19)data/train_fsigs.json,runs/hint_results.json,runs/verdict.json,analysis/probe_to_prompt.png,reports/{report,design_review}.md- Probe pickle + activations moved out of repo.
Report
Rendered from reports/report.md
Motivation
The arc's throughline: deployable capability moves only via weight edits (banking) or externalization (tool-search); every pure test-time readout has failed. Specifically — C17: test-time SELECTION is free but adds no coverage. C19: the composition's first-op is linearly DECODABLE from the base residual far above behavior at depth-1/2 (0.99/0.42) but thins to a thread at depth-3. C20: that decodable direction is NOT steerable (ActAdd inert). The untried seam: externalize the latent readout — decode the first-op with C19's probe and put it in the CONTEXT (shift the proposal, the only lever C17 says works), rather than steering the residual (C20) or selecting post-hoc (C17).
Method
Refit C19's linear first-op probe (standardize+PCA128+L2-logistic) on 1500 training tasks — replicates C19 (depth-1 0.99@L15, depth-2 0.45@L21, depth-3 0.23@L19). On FRESH fsig-disjoint eval tasks (n=100/depth), decode the first-op from the base model's own activation and generate depth-2/3 code under six arms: no-hint; neutral (placebo line, format control); oracle-type (TRUE first-op type = the probe's ceiling); oracle-full (TRUE op WITH parameter = ceiling if param-binding is the bottleneck); probe (decoded type); wrong (RANDOM wrong type, content-causality control). Metrics: greedy@1 + coverage@6, no-think.
Results (no-think, fsig-disjoint eval, n=100/depth)
| arm | d2 greedy@1 | d2 cov@6 | d3 greedy@1 | d3 cov@6 |
|---|---|---|---|---|
| no-hint | 0.030 | 0.050 | 0.000 | 0.010 |
| neutral (placebo) | 0.020 | 0.050 | 0.000 | 0.000 |
| oracle-TYPE | 0.020 | 0.190 | 0.010 | 0.030 |
| oracle-FULL (+param) | 0.190 | 0.310 | 0.010 | 0.030 |
| probe (C19 readout) | 0.010 | 0.060 | 0.000 | 0.000 |
| wrong (content ctrl) | 0.000 | 0.000 | 0.000 | 0.000 |
Probe EVAL accuracy (fsig-disjoint): depth-2 0.32, depth-3 0.18 (vs majority 0.19/0.11). Layer-0 (embedding) probe: 0.05 / 0.02 = chance (leak control passes).
- Externalization ELICITS deployable depth-2 capability where steering (C20) was inert. oracle-full lifts depth-2 greedy@1 6× (0.030→0.190) and coverage 6× (0.050→0.310). Telling the model the concrete first op via the PROMPT works — the first test-time intervention in the whole arc to move deployable capability (decode→prompt succeeds where decode→steer, C20, failed).
- The deployable bottleneck is the PARAMETER, not the op-TYPE C19 decodes. oracle-TYPE lifts coverage (0.050→0.190 — knowing the type narrows the sampling search) but NOT greedy (0.020); only oracle-FULL (with the parameter) makes it single-shot deployable. So the quantity C19 found latent (op type) is coverage-relevant, not greedy-deployable.
- The C19 type-only probe can't cash out. probe-hint ≈ no-hint (greedy 0.010, cov 0.060). The effect is genuine self-elicitation — on the 32% probe-correct tasks, probe-hint coverage 0.156 vs no-hint 0.094 (+0.062); on probe-wrong tasks it slightly hurts (−0.015) — but at 0.32 accuracy it washes out.
- Graded by depth, exactly as C19 predicts. Everything is real at depth-2 (latent headroom 0.42) and ≈0 at depth-3 (thread — even oracle-full only 0.010): no readout conjures information the forward pass never computed.
- Controls clean. neutral placebo ≈ no-hint (not a format effect); wrong-hint HURTS (content-causal, like C28's T_corrupt); layer-0 probe at chance (the readout is the model's COMPUTATION, not surface-readable I/O).
Implication
The latent readout (C19) IS usable at test time — by externalizing it (decode→prompt), not by steering (C20). This adds the first test-time lever that moves deployable capability, bounded by the representation (works depth-2, fades depth-3). But it also finds a NEW wall: the decodable quantity (op TYPE) is not the deployable one (the PARAMETER). Knowing which op-type narrows sampling (coverage) without fixing the greedy mode; the concrete parameter is the missing piece.
Next
- Decode the (op, PARAMETER) — is the parameter latently decodable from the residual too? If a full-op probe reaches useful accuracy, a probe-hint would deliver the oracle-full lift (training-free elicitation). If the param is NOT decodable, that pinpoints exactly what the forward pass fails to compute.
- Compliance instrumentation (parse the generated first-op) and ≥2 seeds; depth-1 sanity (no disjoint tasks in the tiny op-space).
Artifact Manifest
See reports/artifact_manifest.yaml. Probe pickle + activations (~large) moved out of repo.
Experiment log 1
Show the running log (1 entry)
Scaffold
Created as a new experiment scaffold.
Figures 1
Data files 1
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
Reproduce
Smoke test
python scripts/fit_probe.py --n-per-depth 30 && python scripts/run_hints.py --n-per-depth 12 --k 2 --depths 2 3Full run
fit_probe --n-per-depth 500; run_hints --n-per-depth 100 --depths 2 3; analyzeRun steps are documented inside the experiment folder (README and scripts).