Research log Small Model Experimentation
GitHub

Qwen3.5-4B Semantic-Anchor Coordinate Branching

Finished2026-07-13Benchmark GeneralizationInterpretability and DiagnosticsStructured Execution and CompilersGitHub ↗
The late anchor writes names, but the consequence test is invalid

The one idea you need

Imagine pausing a scratchpad, placing a candidate name in a dedicated slot, and copying the model's internal state for a different candidate into that slot. The important test is whether later computation follows the copied candidate—not merely whether the next word changes.

The question

Can a copied internal candidate state at a named hypothesis slot drive a separately computed consequence inside real reasoning?

What we found

This run cannot establish that. The internal edit strongly changed the model's choice among candidate names, but none of 440 consequence outputs began with a valid answer token. Even in a restricted twelve-choice readout, the edited state chose the intended consequence only 5 of 44 times, nearly the 4-of-44 control rate. A post-run audit also found that two rotating maps canceled, leaving the final name-to-label relation fixed across tasks.

Why it matters

The numerical intervention controls all passed, so the failure is not hidden drift. But formatting failure and the fixed composed mapping make this an invalid consequence interface, not a clean negative about internal-state transport. The safe lesson is narrower: changing the next candidate name is not evidence that reasoning consumed it.

Unrestricted outputs parsed56 / 8806.36%, below the frozen 95% requirement
Consequence outputs parsed0 / 440every first token was a formatting opener
Edited consequence choices5 / 44versus 4/44 for source and generic controls
On this page
  1. Results at a glance
  2. Overview
  3. Report
    1. Summary
    2. Research Program Fit
    3. Method
    4. Results
    5. Controls
    6. Oracle Versus Deployable Evidence
    7. Interpretation
    8. Next Experiments
    9. Artifact Manifest
  4. Experiment log
  5. Data files
  6. Reproduce
  7. Related

Results at a glance 1

Writing a candidate name did not carry through to its consequence

How to read

For each method, compare the dark bar for choosing the supplied candidate name with the light bar for choosing the label produced by actually applying that candidate's operation. These are restricted-choice diagnostics; the unrestricted output gate failed separately.

0%25%50%75%100%literal textliteral text97.7%13.6%full donor statefull donor state97.7%13.6%donor coordinatesdonor coordinates95.5%11.4%source / non-Jsource / non-J0%9.1%

Takeaway → Visible text, full-state copying, and coordinate copying all changed the immediate name choice, while their computed consequences stayed near the control rate.

Data table
intervention armdirect candidate selectedcomputed consequence selected
literal text97.7%13.6%
full donor state97.7%13.6%
donor coordinates95.5%11.4%
source / non-J0%9.1%

Numbers from runs/mechanics/summary.json

Technical framing

Direct candidate writing does not become consequence transport — Each rate contains 44 label-free supplied-target rows. These restricted-choice diagnostics are not a passed mechanism result: unrestricted parse was 56/880 overall and 0/440 for consequence outputs, and post-run audit found the composed alias-to-label map was fixed across tasks. All 880 numeric and 2,240 intervention controls passed after canonical identity sorting.

In the author’s words from the Overview · “Results”

Terminal INVALID_MECHANICS_CONTROL at the frozen parse gate. All 880 numeric and 2,240 intervention rows reproduce the calibration after canonical identity sorting, but unrestricted parse is only 56/880 (6.36%); no consequence row parses. Text, full-donor, and donor-J direct constrained choices reach 43/44, 43/44, and 42/44, while their consequence choices reach only 6/44, 6/44, and 5/44. Donor-J consequence probability lift over source is +0.00170 versus the preregistered +0.15 gate. No continuation or correctness stage opened. The mandatory post-mechanics adversarial audit also found that the alias-to- operation and operation-to-label Latin rotations cancel: the composed alias-to- label mapping is identical in all four mechanics tasks. … Read the full result →

Overview

This experiment tests the last mechanism-specific bridge left by the native Jacobian line: put a named hypothesis token inside real reasoning and replace its context-local J coordinates with each candidate donor state.

Research Program

Question

Did arbitrary-position additive steering fail because J is not a native reasoning control, or because the replicated effect specifically requires an explicit semantic token and donor-coordinate replacement?

Hypothesis

The fixed early J dictionary carries a context-local concept state only when a token supplies a semantic workspace address. After 512 native thinking tokens, all 24 J coordinates at a forced Candidate first-operation alias: token are replaced with clean coordinates captured from another alias in the identical context. This should transport a task-randomized, separately computed operation consequence and then seed candidate-specific reasoning. Literal text anchors are the deployable baseline; full donor activation is the causal upper bound.

Setup

  • Model: only Qwen/Qwen3.5-4B, revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a.
  • Backend: Transformers bf16 SDPA for every arm. Activation mechanics use batch-one full recomputation; backend mixing is forbidden.
  • Lens: byte-identical 24-concept context lens, layers 4--8, with no refit, layer sweep, or amplitude sweep.
  • Fresh substrate: exact-depth-two procedural list transformations, split into four label-sealed mechanics, 24 qualification, and 48 untouched confirmation tasks, with complete fingerprint disjointness from ancestors.
  • Prompt mapping: the 12 public one-token aliases are assigned to operations by a frozen task-balanced permutation, so token identity cannot stand in for an operation across tasks.
  • Anchor bank: a task-ID-selected valid source alias is the surface recipient; every other public alias supplies a clean, context-local donor state.
  • Primary mechanics endpoint: apply the supplied candidate's prompt-local operation to [3,-1,2,0] (k=2) and select a task-randomized one-token label for the resulting list. Task correctness is never loaded.
  • Arms: source, literal target text, full donor activation, all-24 donor J, mean donor J, frozen additive J, two exact live-bf16 non-J controls, rotated wrong donor, and ordinary concept-logit-lens replacement.
  • Capability baselines: literal text, clean shared-prefix continuations, and a frozen master pool of fully independent sample-more traces at sampled-token and total-forward-compute parity.
  • Hidden-label boundary: mechanics sees only public task prompt fields and its diagnostic mapping. Qualification writes every output and resource receipt before grading. Confirmation stays absent until qualification passes.

Run

Model-free smoke:

.venv/bin/python experiments/qwen35_4b_semantic_anchor_coordinate_branching/scripts/run.py --stage smoke

The terminal mechanics run was:

PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1 \
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
.venv/bin/python experiments/qwen35_4b_semantic_anchor_coordinate_branching/scripts/run.py --stage mechanics

Results

Terminal INVALID_MECHANICS_CONTROL at the frozen parse gate. All 880 numeric and 2,240 intervention rows reproduce the calibration after canonical identity sorting, but unrestricted parse is only 56/880 (6.36%); no consequence row parses. Text, full-donor, and donor-J direct constrained choices reach 43/44, 43/44, and 42/44, while their consequence choices reach only 6/44, 6/44, and 5/44. Donor-J consequence probability lift over source is +0.00170 versus the preregistered +0.15 gate. No continuation or correctness stage opened.

The mandatory post-mechanics adversarial audit also found that the alias-to- operation and operation-to-label Latin rotations cancel: the composed alias-to- label mapping is identical in all four mechanics tasks. The advertised task- randomized computation endpoint was therefore not realized.

Interpretation

This exact one-token consequence interface is invalid and unreachable. The conditional direct-choice pattern is hypothesis-generating evidence that an explicit anchor can write a name, not passed evidence of reasoning transport. Because literal text and full-state donors also fail the consequence interface, and because the composed mapping was fixed, this run cannot isolate a general J-space limitation. The experiment is frozen; any repair or continuation must use a fresh directory and fresh data.

Knowledgebase Update

  • Program evidence and shared synthesis record the terminal invalid result and retire this late opaque-anchor interface.
  • Program backlog redirects to a fresh deployable early-text hypothesis fork, not another J amplitude/layer sweep.
  • Claim ledger: no new claim while the repository claim re-grade is open.

Artifacts

  • idea_intake.md: closest-neighbor and novelty decision.
  • reports/preregistration.md: immutable stage logic and gates.
  • reports/design_review.md: adversarial review before any GPU run.
  • assets/context_lens.pt: byte-identical frozen lens.
  • scripts/run.py: frozen staged runner and automatic decision logic.
  • reports/pre_mechanics_adversarial_audit.md: authorization audit.
  • reports/post_mechanics_adversarial_audit.md: terminal audit and discovered composition confound.
  • runs/mechanics/: complete terminal mechanics receipts.
  • reports/artifact_manifest.yaml: artifact policy.

Report

Rendered from reports/report.md

Summary

Terminal INVALID_MECHANICS_CONTROL (parse). Continuation and all task- correctness stages remained sealed.

Research Program Fit

Primary interpretability_and_diagnostics; conditional secondary evidence is withheld until the respective mechanism/capability gates run.

Method

One shared 512-token native thought receives a valid source-alias anchor. Clean target text is compared with full activation, context-local all-24 donor coordinates, mean donor coordinates, the prior additive J method, two exact live non-J controls, wrong donor, and logit-lens replacement. A direct identity probe and a task-randomized computed operation consequence precede continuation.

Results

Model-free smoke passes: lens SHA/rank and all 12 distinct diagnostic results are valid; the 4/24/48 splits contain 76 unique new behaviors with zero overlap against 1,046 ancestor fingerprints. The original smoke had six tests; 15 now passed after the pre-model implementation audit (16 after the suffix-shape repair). At that original boundary, model/outcomes remained unloaded.

Outcome-blind model smoke now also passes after preserving one failed receipt. The equal-length retry has exact zero cross-suffix anchor difference, 20/20 live non-J controls, maximum norm error 9.3031e-6, maximum J-span fraction 0.0099543, and 60/60 intervention rows. No outcome has been measured.

The full outcome-blind calibration also passes all 880 live controls and 2,240 intervention rows. Maximum norm error is 9.9968e-6, maximum span fraction is 0.0099951, and at most three lattice pairs are required. All four exact native prefixes are locked.

The one authorized mechanics run reproduces all controls after canonical identity sorting: 880/880 numeric and 2,240/2,240 intervention rows pass, all four prefixes match, and donor tensors remain immutable. The automatic result is nevertheless INVALID_MECHANICS_CONTROL because unrestricted next-token parse is 56/880 (6.36%) against 95%; consequence parse is 0/440. Consequence full tops are always a formatting opener (294 backticks and 146 opening brackets).

Within the preregistered constrained token sets, text target and full donor each write the direct alias on 43/44 rows, donor J on 42/44, and wrong-donor J writes its own alias on 42/44 while the registered target is 0/44. The corresponding computed-consequence results are 6/44, 6/44, 5/44, and 5/44 wrong-donor-own. Donor J exceeds the worse non-J consequence rate by one row and lifts mean conditional target probability by only 0.00170.

Post-mechanics review found a second invalidity: both task maps advance by the same cyclic index, so alias -> operation -> label is a single fixed 12-pair map across all four tasks. Individual map rotation did not produce independent composition. Thus task/label breadth could not have certified computation even if the endpoint had passed.

Controls

The preregistration freezes task-local alias meaning, intended result-label rotation, live bf16 norm/span tolerances, source/donor position, no-alpha/no-layer-sweep rules, and compute accounting including every donor capture.

Mechanics and calibration files differ in row order; the runner compares sorted unique row objects, and those objects are exactly equal. Earlier audit wording that promised raw-file byte equality was too strong and is corrected here.

Oracle Versus Deployable Evidence

Mechanics is independent of task correctness but remains a write diagnostic. Literal text is deployable. Activation arms are white-box interventions and must beat literal text plus matched-compute sampling to count as elicitation.

Interpretation

No continuation is authorized. The formal invalid verdict is driven by the frozen parse contract, not numerical corruption. Counterfactually ignoring parse would still stop at ANCHOR_PROBE_UNREACHABLE, because literal text and full-state consequence rates are only 13.6%. This does not establish a clean negative about native J consequence transport: the readout itself is unreachable and the intended label-randomization control is broken. It does show that the late opaque anchor provides no deployable advantage over literal text in this setup.

Next Experiments

A fresh experiment may move a concrete textual hypothesis before the reasoning trajectory, generate full continuations, and compare visible-only selection against matched-compute sampling. Do not repair this result-bearing experiment, post-hoc accept formatting tokens, or sweep J layers/scales.

Artifact Manifest

See artifact_manifest.yaml; all terminal receipts are committed and no external artifact is required.

Experiment log 11

Show the running log (11 entries, 2026-07-13)

2026-07-13 — Intake and adversarial design

  • Created as a distinct follow-up to terminal additive native branching.
  • Related-work search names the short synthetic clamp as closest near-duplicate.
  • Two independent read-only adversarial reviews rejected a cloud recipient as non-neutral and rejected direct alias writing as a sufficient endpoint.
  • Replaced it with valid-source-to-target donor transport and a primary alias -> task-local operation -> computed result -> randomized label probe.
  • Fixed one model/revision, byte-identical lens, layers 4--8, all-24 donor coordinates, no scale sweep, fresh seed, and staged 4/24/48 splits.
  • Literal text, full donor, mean coordinate, additive J, two live non-J, wrong- donor, and logit-lens arms prevent a positive hiding a simpler explanation.
  • No model call is authorized until CPU smoke, tests, and a pushed hash boundary all pass.

2026-07-13 — Model-free smoke

  • Frozen lens SHA matches and all five intervention matrices retain rank 24.
  • The 12 preregistered diagnostic operation results are pairwise distinct.
  • Generated 4/24/48 task splits with task-local Latin-cycle alias mappings; qualification and confirmation balance every alias-operation pair exactly.
  • All 76 behavior fingerprints are unique and have zero overlap with 1,046 readable ancestor procedural fingerprints. Benchmark contents were not read.
  • Public mechanics rows expose only prompt fields, source alias, and diagnostic labels; sealed-field mutation leaves the prompt invariant.
  • Six tests pass and direct mechanics invocation fails before any model load.

2026-07-13 — Immutable design anchored

  • Pushed the reviewed design and generated registration through commit 9437bdc2664772f4ad2c50e8403740f11c28688c.
  • Anchored README, preregistration, adversarial review, public mechanics data, data manifest, and frozen lens by SHA-256.
  • A clean model-free rerun verifies the commit is an ancestor and both local and committed bytes match every expected digest. Implementation remains pending.

2026-07-13 — Pre-model implementation audit repair

  • An independent code audit caught that the preregistered label_map seed was present but not used. Corrected the generator before any model call, preserving every task behavior while changing only the diagnostic-label stream; added a mutation test and will supersede the public-data design hashes.
  • Removed an inherited close-suppression flag. Native 512-token prefix sampling uses ordinary cached decoding and fails on natural close/EOS; all anchor capture/intervention/readout forwards remain cache-free as preregistered.
  • Implementation now freezes scientific config semantics, exact prefix IDs, tokenizer/whole-scaffold contracts, clean donors, exact-once hooks, two live controls, and every patcher type before outcome retention.

2026-07-13 — Implementation boundary anchored

  • Pushed the audited implementation and stable catalogs through commit fa5696c5916024bee7936893c6b59eb738d3f769.
  • Anchored runner, model operations, coordinate math, additive geometry, pure mechanics evaluator, task data, all tests, and the implementation audit.
  • The canonical scientific-config hash excludes only boundary receipt blocks; changing any seed, gate, prompt, arm, model, lens, or control breaks unlock.
  • This boundary authorizes one outcome-blind representative model smoke. Full calibration and mechanics remain separately gated.

2026-07-13 — Model smoke 001 fails suffix-shape causality

  • All 20 non-J numeric rows and 60 intervention rows pass; no outcome was kept.
  • The frozen causal activation gate fails: direct/consequence max difference is 0.078125 versus 0.001, with suffix lengths 14 versus 216 tokens.
  • Preserved the full failed receipt and rejected threshold relaxation.
  • Geometry-only repair gives both probes the same public result table and an exactly length-balancing direct-control instruction. Both suffixes are now 216 pinned-tokenizer tokens; scientific endpoints and gates are unchanged.
  • A new implementation hash boundary is mandatory before retry.

2026-07-13 — Model smoke 002 passes equal-length repair

  • Both probe suffixes are exactly 216 pinned-tokenizer tokens; causal anchor difference falls from 0.078125 to exact zero on identical prefix IDs.
  • All 20 live numeric rows and all 60 intervention rows pass. Maximum norm error is 9.3031e-6; maximum span fraction is 0.0099543; no lattice repair.
  • No logits, probabilities, outcomes, or correctness fields were retained.
  • Passing receipt must be pushed before the one authorized 880-row calibration.

2026-07-13 — Full live-bf16 calibration passes

  • Exactly 880/880 numeric rows and 2,240/2,240 intervention rows pass.
  • Maximum norm error is 9.9968e-6; maximum J-span fraction is 0.0099951; maximum lattice repair is three pairs at layer 8.
  • Cross-probe causal difference remains exact zero. All four prefix ID lists, donor immutability, positions, tokenizer, model, lens, and boundary pass.
  • Outcomes/correctness/logits/probabilities remain absent.
  • Mechanics remains locked until this receipt and its adversarial audit are committed, pushed, and hashed in a separate boundary.

2026-07-13 — Mechanics boundary anchored

  • The passing model-smoke and full calibration artifacts are anchored to pushed commit 6e49856e75280c709294410da89d5fafc26d095d by seven exact SHA-256 hashes.
  • Local and committed bytes must both match, and the calibration commit must be an ancestor, before mechanics can load the model.
  • This authorizes exactly one label-free mechanics run. It does not authorize qualification, confirmation, or any capability claim.

2026-07-13 — Mechanics terminates invalid at parse

  • The one authorized run reproduces 880/880 numeric rows and 2,240/2,240 intervention rows after canonical identity sorting; all prefixes and donor immutability pass.
  • Automatic decision: INVALID_MECHANICS_CONTROL. Full-vocabulary parse is 56/880 and consequence parse is 0/440, below the frozen 95% gate.
  • Direct constrained selection is 43/44 text, 43/44 full donor, and 42/44 donor J. Consequence selection is only 6/44, 6/44, and 5/44; donor-J probability lift is +0.00170.
  • No continuation, qualification, confirmation, task correctness, or capability claim opens.

2026-07-13 — Post-mechanics adversarial audit

  • The parse failure is a real failure of the one-token interface: every consequence full top is a formatting opener. Optional-prefix parsing would be an unregistered rescue and was not run.
  • Independent review exposed a generator confound: both cyclic mappings advance by the same task index, leaving alias -> result label constant across tasks. The claimed randomized computation endpoint is invalid.
  • Calibration and mechanics row objects are exactly equal after frozen identity sorting, but raw JSONL bytes differ because their order differs. Correct the earlier byte-match wording; numerical validity is unaffected.
  • Freeze the experiment as terminal invalid/unreachable. A fresh successor must test composed-map independence and use a naturally parseable continuation.

Data files 4

Result tables and metrics copied from the experiment folder — preview inline or open the raw file.

Reproduce

Smoke test

.venv/bin/python experiments/qwen35_4b_semantic_anchor_coordinate_branching/scripts/run.py --stage smoke

Full run

PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True .venv/bin/python experiments/qwen35_4b_semantic_anchor_coordinate_branching/scripts/run.py --stage mechanics

Run steps are documented inside the experiment folder (README and scripts).

Browse the experiment folder on GitHub ↗