Learned Sparse Slot Executor Experiment

Finished2026-06-21imported · line YAlgorithmic Memory and Retrieval Structured Execution and Compilers GitHub ↗

The one idea you need

Picture a whiteboard holding a fixed set of candidate answers for two hidden numbers, each with a confidence score. As arithmetic steps and narrowing clues arrive, the system must rewrite every note so all still-possible answers stay in view.

The question

Can a small network learn to keep and rewrite a scratchpad tracking which values two hidden numbers might still be, as math steps and clues arrive?

What we found

Yes, but only small. With eleven possible values and a ready-made scratchpad, the network kept the fully correct answer in view 95.5% of the time, even on longer chains than it trained on. Widen to thirty-one values and the same design collapsed to 12.3%, while a hand-coded updater stayed perfect throughout.

Why it matters

For state-tracking or wrap-around arithmetic, a generic learned updater has no built-in arithmetic and falls apart as the value range grows. Bake the update rule in; learning only where to place memory proved far more robust than learning how to change it.

Correct answer kept in view, small task95.5%values wrap at eleven, chains longer than trained on

Same design, bigger task12.3%values wrap at thirty-one, chain of twenty-four steps

Best learned approach, bigger task36.5%learned where to place memory, not how to update it

Hand-coded ceiling100%proves the memory format can store the exact answer

On this page

Results at a glance
Overview
Report
Experiment log
Figures
Data files
Reproduce
Related

Results at a glance 8

On the bigger task, learned pieces trail the perfect hand-built version

How to read

Each line tracks the share of probability landing on the correct final answer (higher is better) as programs grow longer, left to right. The flat top line is fully hand-built; the two sloping lines each learn part of the job.

Takeaway → Both learned lines slope steadily down and sit far below the flat perfect line, and the gap widens as programs lengthen.

Data table

program length	oracle init + exact transition (ceiling)	learned init + exact transition	oracle init + neural transition
4	100%	84.8%	74.1%
8	100%	74.6%	60.3%
12	100%	66.4%	49.2%
16	100%	59.3%	41.2%
24	100%	49.1%	30.7%

Numbers from experiments/learned_sparse_slot_executor/analysis/first_k_ge_l_summary.csv (also analysis/summary.md table)

Technical framing

Modulus 31: learned components fall far below the exact oracle ceiling (query mass vs program length) — The slot format can represent exact belief (ceiling = 100%), but learned initializers and neural transitions both degrade with length at modulus 31.

Getting both hidden numbers exactly right collapses fastest for learned updates

How to read

Lines show the share of probability on the exact full answer state (higher is better) as programs lengthen. The flat top line is hand-built; the lowest line learns the step-by-step updates.

Takeaway → The learned-update line collapses to roughly one-tenth by the longest program, dropping well below the version that only learns the initial board setup.

Data table

program length	oracle init + exact transition (ceiling)	learned init + exact transition	oracle init + neural transition
4	100%	60%	34.4%
8	100%	54.5%	27.5%
12	100%	49%	21.3%
16	100%	44.2%	17.2%
24	100%	36.5%	12.3%

Numbers from experiments/learned_sparse_slot_executor/analysis/first_k_ge_l_summary.csv (also analysis/summary.md table)

Technical framing

Modulus 31 strict belief mass: neural transitions collapse hardest — On the stricter belief metric the neural transition drops to 12.3% at length 24; the learned initializer with exact transitions holds 36.5%.

On the small task, learned updates stay near perfect

How to read

Lines show the share of probability on the correct answer (higher is better) as programs lengthen. The top line learns step updates from a ready-made memory board; the lower lines also learn the board setup themselves.

Takeaway → Handed the starting board, the learned updater holds a nearly flat top line across lengths, while learning setup and updates together decays fastest.

Data table

program length	oracle init + neural transition	learned init + exact transition (final-query loss)	learned init + exact transition (full-belief loss)	learned init + neural transition (end-to-end)
3	98.4%	89.3%	84.1%	84.2%
6	97.6%	82.7%	75.3%	72.4%
9	97.2%	80.1%	72.3%	64.9%
12	97.1%	78.9%	70.2%	57.8%

Numbers from experiments/learned_sparse_slot_executor/analysis/first_k_ge_l_summary.csv (also analysis/summary.md table)

Technical framing

Modulus 11 pilot: neural transitions are learnable when slots start from oracle initialization — The one clear positive: with oracle slots, a learned transition holds 97% query mass out to held-out length 12; end-to-end learning composes worst.

The small-task success does not carry over to the bigger task

How to read

Bars show the share of probability on the exact answer state at the longest program tested (taller is better): learned updates on the small task, then the same learned updates and a learned setup on the bigger task.

Takeaway → The tall small-task bar shrinks to a stub on the bigger task, showing the same method fails to scale rather than being unable to store the answer.

Data table

condition	belief target mass
oracle init + neural transition (mod 11, L=12)	95.5%
oracle init + neural transition (mod 31, L=24)	12.3%
learned init + exact transition (mod 31, L=24)	36.5%

Numbers from experiments/learned_sparse_slot_executor/analysis/first_k_ge_l_summary.csv (also analysis/summary.md table)

Technical framing

Held-out belief mass at the longest evaluated length: pilot success does not scale — The same neural transition that reaches 95.5% belief mass at modulus 11 falls to 12.3% at modulus 31 - a scale failure, not a representation failure.

Memory slots lose focus as programs get longer

How to read

Lines track how sharply each memory slot is focused on a single value as programs lengthen — a readout of internal state, not a score. It shows how the slots behave, not which setup wins.

Takeaway → Most lines drift downward with length while the learned-update line stays flat, hinting its slots stop sharpening the way the task needs.

Data table

Program length	Learned init, exact transitions	Oracle init, exact ceiling	Oracle init, neural transitions
4	0.614	0.477	0.308
8	0.503	0.293	0.314
12	0.413	0.218	0.307
16	0.347	0.177	0.308
24	0.254	0.142	0.309

Numbers from analysis/summary.md table (First K >= L, main modulus 31)

Technical framing

Slot memory purity degrades with program length (modulus 31) — Every variant loses slot purity as programs lengthen; neural transitions plateau near 0.31 while others keep decaying.

Training keeps improving but never fully settles on the bigger task

How to read

Lines show the share of probability on the correct answer (higher is better) as training proceeds, left to right. The upper line learns only the board setup; the lower line learns the step updates.

Takeaway → Both lines are still climbing when training stops, with the learned-update line trailing by roughly ten points the whole way.

Data table

Training step	Learned init, exact transitions	Oracle init, neural transitions
1	45.4%	33.2%
150	55.4%	47.7%
300	58%	48.7%
450	66.3%	51.6%
600	74.9%	51.6%
750	79.3%	61.5%
900	82.2%	65.5%
1050	—	71.2%
1200	—	71.7%

Numbers from analysis/train_log.csv

Technical framing

Training progress on modulus 31: query mass climbs but never converges — Both modulus-31 runs are still improving at budget end, but neural transitions lag exact transitions by roughly 10 points throughout.

On the small task, both setup and update learning matter

How to read

Lines show the share of probability on the correct answer (higher is better) as programs lengthen, for four combinations of what the model learns versus what it is handed ready-made.

Takeaway → Only the version handed the starting board keeps a flat top line; learning both setup and updates together drops the most across length.

Data table

Program length	Learned init, exact, final-query loss	Learned init, exact, full-belief loss	Learned init, neural, full-belief loss	Oracle init, neural, full-belief loss
3	89.3%	84.1%	84.2%	98.4%
6	82.7%	75.3%	72.4%	97.6%
9	80.1%	72.3%	64.9%	97.2%
12	78.9%	70.2%	57.8%	97.1%

Numbers from analysis/summary.md table (First K >= L, pilot modulus 11)

Technical framing

Modulus 11 ablation: supervision and initialization both matter — Only oracle-initialized slots hold query mass across length; learning init and transitions together decays fastest (89% to 58%).

Even the easiest task shows learned updates lagging

How to read

Bars show the share of probability on the exact answer state at a short program length (taller is better) for five versions, from fully hand-built to fully learned.

Takeaway → The hand-built bar reaches the top while every learned-update bar falls short, so the shortfall shows up even at the easiest setting.

Data table

Variant	Belief target mass
Learned init, exact, final-query	76.8%
Learned init, exact, full-belief	75.8%
Learned init, neural	68.3%
Oracle init, exact	100%
Oracle init, neural	82.2%

Numbers from analysis/summary.md table (First K >= L, smoke modulus 7)

Technical framing

Smoke test at modulus 7: neural transitions already lag at length 3 — Even at the easiest modulus and shortest programs, swapping exact for neural transitions costs 8-18 points of belief mass.

In the author’s words from the Report · “Abstract”

At modulus 11, a neural transition module with oracle slot initialization learns strong recurrent execution, reaching 97.1% query mass and 95.5% belief mass at held-out length But this transition learner does not scale cleanly to modulus 31: at length 24 it reaches only 30.7% query mass and 12.3% belief mass. Learned initialization with exact transitions scales better at modulus 31, reaching 49.1% query mass and 36.5% belief mass at length 24, but it also remains far from the exact oracle ceiling.

Overview

This experiment tests whether a neural recurrent runtime can learn to use an explicit slot memory for modular belief-state execution.

src/learned_sparse_slot_experiment.py: task generator, trainable slot executor, checkpointing, and evaluation harness.
src/analyze_learned_sparse_slot.py: analysis and figure generation.
reports/learned_sparse_slot_experiment_log.md: chronological experiment log.
reports/learned_sparse_slot_paper.md: standalone written report.
reports/learned_sparse_slot_paper.html: standalone HTML report.
runs/: JSON and CSV run outputs.
analysis/: generated summaries and figures.
checkpoint_manifest.csv: checkpoint paths and sizes.

Checkpoints are written outside the experiment directory under:

../../large_artifacts/learned_sparse_slot_executor/checkpoints/

Download this experiment directory for the normal research bundle. Download ../../large_artifacts/learned_sparse_slot_executor/ only when saved model weights are needed.

Report

Rendered from reports/learned_sparse_slot_paper.md

Abstract

This experiment tests whether a neural recurrent model can learn to use an explicit sparse slot memory for modular belief-state execution. Each example starts from a hidden relation over two registers, B = A + d (mod p), then applies arithmetic updates and observation filters. The model keeps S slots; each slot represents a distribution over one candidate A, one candidate B, and a slot weight. The decoded belief is the weighted mixture of slot-local pair distributions.

The experiment separates two learnability questions: learning to initialize the slot memory and learning to update slots recurrently. At modulus 11, a neural transition module with oracle slot initialization learns strong recurrent execution, reaching 97.1% query mass and 95.5% belief mass at held-out length 12. But this transition learner does not scale cleanly to modulus 31: at length 24 it reaches only 30.7% query mass and 12.3% belief mass. Learned initialization with exact transitions scales better at modulus 31, reaching 49.1% query mass and 36.5% belief mass at length 24, but it also remains far from the exact oracle ceiling.

Task

Each example starts from:

B = A + d (mod p), with A unknown

The initial belief has p possible (A,B) states. Programs contain arithmetic updates and observation filters:

A=A+c, A=A-c, B=B+c, B=B-c
A=A+B, B=B+A, A=A-B, B=B-A
A % m = r, B % m = r

Observation residues are sampled from the current support, so the target belief is never empty. The final query asks for the distribution of A, B, A+B mod p, or A-B mod p.

Model

The model maintains S slots. Each slot has:

logits over A values,
logits over B values,
a scalar slot-weight logit.

The decoded pair belief is:

P(A,B) = sum_s softmax(w)_s * P_s(A) * P_s(B)

Two axes are varied:

Axis	Choices	Meaning
Initialization	oracle, learned	Oracle slots store the initial support; learned slots are produced by an MLP from `d` and a slot id.
Transition	exact, neural	Exact transitions apply the known arithmetic/filter update; neural transitions use an MLP conditioned on slot state, op, and argument.

Two supervision modes are used:

full_belief: cross-entropy to the exact belief at every prefix.
final_query: cross-entropy only to the final query distribution.

Protocol

Phase	Modulus	Main purpose	Evaluation lengths
Smoke	7	Validate all staged paths.	2, 3
Pilot	11	Compare learnability of initialization, transition, and end-to-end composition.	3, 6, 9, 12
Main	31	Test scale behavior of the strongest staged conditions.	4, 8, 12, 16, 24

For each length L, the headline rows report the first recurrent budget K such that K >= L.

Metrics:

decoder_query_target_mass: probability assigned to the exact final query support after projecting the decoded slot belief.
decoder_belief_target_mass: probability assigned to the exact final (A,B) support.
mean_slot_purity: average product of the largest A probability and largest B probability per slot.
mean_weight_entropy: entropy of the slot-weight distribution.

Smoke Results

All model paths compile, train, evaluate, and write checkpoints where applicable. The exact oracle ceiling is perfect. The learned paths recover substantial signal but remain below exact after short smoke budgets.

Variant	Init	Transition	Supervision	L=2 query	L=2 belief	L=3 query	L=3 belief
Oracle exact ceiling	oracle	exact	full belief	100.0%	100.0%	100.0%	100.0%
Learned init, exact transition	learned	exact	full belief	90.0%	75.7%	87.5%	75.8%
Oracle init, neural transition	oracle	neural	full belief	93.9%	84.4%	90.8%	82.2%
Learned init, neural transition	learned	neural	full belief	90.5%	72.8%	83.6%	68.3%
Learned init, exact transition	learned	exact	final query	91.2%	77.3%	88.0%	76.8%

Pilot Results

At modulus 11, the staged transition-learning condition is strongly positive: with oracle slot initialization, a neural transition reaches 95.5% belief mass at held-out length 12.

Variant	Init	Transition	Supervision	L=3 query	L=6 query	L=9 query	L=12 query	L=12 belief
Learned init, exact transition	learned	exact	full belief	84.1%	75.3%	72.3%	70.2%	58.1%
Oracle init, neural transition	oracle	neural	full belief	98.4%	97.6%	97.2%	97.1%	95.5%
Learned init, neural transition	learned	neural	full belief	84.2%	72.4%	64.9%	57.8%	39.3%
Learned init, exact transition	learned	exact	final query	89.3%	82.7%	80.1%	78.9%	70.2%

The end-to-end learned model is much weaker than either staged success. This shows that the two subskills do not compose automatically under the current parameterization and losses.

Pilot query mass

Pilot belief mass

Main Results

At modulus 31, the exact oracle path remains perfect, but the learned staged conditions are no longer close to exact.

Variant	Init	Transition	Supervision	L=4 query	L=8 query	L=12 query	L=16 query	L=24 query	L=24 belief
Oracle exact ceiling	oracle	exact	full belief	100.0%	100.0%	100.0%	100.0%	100.0%	100.0%
Oracle init, neural transition	oracle	neural	full belief	74.1%	60.3%	49.2%	41.2%	30.7%	12.3%
Learned init, exact transition	learned	exact	final query	84.8%	74.6%	66.4%	59.3%	49.1%	36.5%

Strict belief mass:

Variant	L=4 belief	L=8 belief	L=12 belief	L=16 belief	L=24 belief
Oracle exact ceiling	100.0%	100.0%	100.0%	100.0%	100.0%
Oracle init, neural transition	34.4%	27.5%	21.3%	17.2%	12.3%
Learned init, exact transition	60.0%	54.5%	49.0%	44.2%	36.5%

Main query mass

Main belief mass

Main slot purity

Interpretation

The experiment separates representational sufficiency from learnability. The oracle exact ceiling proves that the slot representation can express the exact belief state. The learned models show that this state format is not automatically discovered at scale by generic MLP initializers and transitions.

The clearest positive result is local: at modulus 11, neural recurrent transitions are learnable when the initial support is already placed into slots. The learned transition generalizes from training lengths up to 6 to held-out length 12 with 95.5% belief mass.

The main negative result is scale: the same transition parameterization fails to learn clean modular updates at modulus 31. With oracle slots and full belief traces, length-24 belief mass is only 12.3%.

The learned initializer is more robust at modulus 31 than the neural transition. With exact transitions and final-query supervision, it reaches 36.5% belief mass at length 24. This is a useful approximate slot state, but not the exact one-slot-per-support-element machine.

Limitations

The neural transition is a generic MLP over expected slot embeddings. It has no hard modular arithmetic inductive bias. Failure at modulus 31 should therefore be read as a failure of this parameterization, not proof that learned slot execution is impossible.

The slot decoder factorizes each slot as P_s(A)P_s(B). This is appropriate when slots become sharp, but it is a weak representation for diffuse slots. The drop in slot purity at longer lengths is therefore directly relevant to the belief-mass drop.

The main runs do not include a full-budget end-to-end modulus-31 model. The pilot already showed that end-to-end learning is weaker than both staged conditions, and the main staged transition itself failed to scale.

Reproducibility

Run outputs are in:

experiments/learned_sparse_slot_executor/runs/

Checkpoints are stored externally:

large_artifacts/learned_sparse_slot_executor/checkpoints/

Regenerate analysis:

PYTHONDONTWRITEBYTECODE=1 python experiments/learned_sparse_slot_executor/src/analyze_learned_sparse_slot.py

Example main run:

PYTHONDONTWRITEBYTECODE=1 python experiments/learned_sparse_slot_executor/src/learned_sparse_slot_experiment.py \
  --variant_name main_learned_init_exact_final_query \
  --modulus 31 \
  --observe_mod 5 \
  --observe_prob 0.3 \
  --slot_capacity 31 \
  --init_mode learned \
  --transition_mode exact \
  --supervision final_query \
  --slot_dim 96 \
  --hidden_dim 192 \
  --train_min_len 1 \
  --train_max_len 8 \
  --train_steps 900 \
  --batch_size 512 \
  --eval_lengths 4,8,12,16,24 \
  --eval_k 0,1,2,4,8,12,16,24 \
  --output_dir experiments/learned_sparse_slot_executor/runs/main_learned_init_exact_final_query \
  --checkpoint_dir large_artifacts/learned_sparse_slot_executor/checkpoints/main_learned_init_exact_final_query

Experiment log 11

Show the running log (11 entries, 2026-06-21)

Objective

Test whether a neural recurrent executor can learn to represent and update a sparse slot memory for modular belief-state execution.

Each example starts from:

B = A + d (mod p), with A unknown

Programs apply arithmetic updates and observation filters over hidden registers A and B. The model maintains S slots. Each slot represents a distribution over one candidate A value, one candidate B value, and a slot weight. The decoded belief distribution is the weighted mixture of slot-local (A,B) distributions.

Primary Questions

With oracle slot initialization, can a learned transition module recover the arithmetic and filter updates?
With exact transitions, can a learned initializer populate a support memory from the initial relation?
Can the fully learned slot executor combine learned initialization and learned transitions?
Does final-query supervision alone induce the same slot machinery, or is prefix belief supervision required?

Metrics

decoder_query_target_mass: probability assigned to the exact final query support after projecting the decoded slot belief.
decoder_belief_target_mass: probability assigned to the exact final (A,B) support.
decoder_belief_top1_on_support: whether the highest-probability pair is in the exact final support.
mean_slot_purity: mean product of the strongest A and B probabilities per slot.
mean_weight_entropy: entropy of the slot-weight distribution.

The headline metric is decoder_query_target_mass, with decoder_belief_target_mass as the stricter execution metric.

Artifact Layout

Code and lightweight outputs: experiments/learned_sparse_slot_executor/
Checkpoints: large_artifacts/learned_sparse_slot_executor/checkpoints/
Run outputs: experiments/learned_sparse_slot_executor/runs/<variant>/
Analysis outputs: experiments/learned_sparse_slot_executor/analysis/

Planned Sequence

Smoke tests on modulus 7 to validate oracle initialization, learned initialization, exact transition, and neural transition paths.
Pilot runs on modulus 11 to compare staged learning conditions.
Main runs on modulus 31 for the most informative staged variants.
Generate tables, figures, checkpoint manifest, standalone report, and HTML report.

Variant Plan

smoke_oracle_init_exact_transition: ceiling path for output and metrics.
smoke_learned_init_exact_transition: learned slot initialization with exact recurrent updates.
smoke_oracle_init_neural_transition: oracle slots with learned recurrent updates.
smoke_learned_init_neural_transition: fully learned slot executor.
pilot_*: modulus-11 versions of the staged variants that pass smoke.
main_*: modulus-31 runs for the highest-signal staged variants.

2026-06-21 Setup

Created the standalone experiment directory:

experiments/learned_sparse_slot_executor/src/
experiments/learned_sparse_slot_executor/reports/
experiments/learned_sparse_slot_executor/runs/
experiments/learned_sparse_slot_executor/analysis/figures/
large_artifacts/learned_sparse_slot_executor/checkpoints/

CUDA is available on the current machine. Checkpoints will be kept outside the experiment directory.

Next action: implement the trainable slot executor and run smoke tests.

2026-06-21 Smoke Tests

Implemented the training and evaluation harness:

Oracle or learned slot initialization.
Exact or neural recurrent transitions.
Full prefix-belief supervision or final-query supervision.
Decoded query and belief metrics.
Slot purity and weight-entropy diagnostics.
External checkpoint writing.
Analysis script and figures.

Ran five modulus-7 smoke variants:

Variant	Init	Transition	Supervision	Steps	L=2 query	L=2 belief	L=3 query	L=3 belief
`smoke_oracle_init_exact_transition`	oracle	exact	full belief	0	100.0%	100.0%	100.0%	100.0%
`smoke_learned_init_exact_transition`	learned	exact	full belief	200	90.0%	75.7%	87.5%	75.8%
`smoke_oracle_init_neural_transition`	oracle	neural	full belief	500	93.9%	84.4%	90.8%	82.2%
`smoke_learned_init_neural_transition`	learned	neural	full belief	700	90.5%	72.8%	83.6%	68.3%
`smoke_learned_init_exact_final_query`	learned	exact	final query	700	91.2%	77.3%	88.0%	76.8%

Smoke interpretation:

The ceiling path is exact, confirming the decoder, target generation, and K-indexed evaluation are correct.
Learned initialization with exact transitions recovers much of the support state but is not exact after a short run.
Learned neural transitions with oracle slots are stronger than the fully learned path, so transition learning and initialization learning should stay separated in the pilot.
Final-query supervision is enough to recover substantial belief mass on the small task when transitions are exact.

Pilot decision:

Run modulus-11 staged variants for learned initialization, learned transition, full end-to-end learning, and final-query-only initialization.
Use longer budgets than smoke, but keep the sweep scoped enough to inspect results before any modulus-31 run.

2026-06-21 Pilot Sweep

Ran four modulus-11 pilot variants with evaluation lengths 3, 6, 9, and 12:

Variant	Init	Transition	Supervision	Steps	L=3 query	L=6 query	L=9 query	L=12 query	L=12 belief
`pilot_learned_init_exact_full_belief`	learned	exact	full belief	1400	84.1%	75.3%	72.3%	70.2%	58.1%
`pilot_oracle_init_neural_full_belief`	oracle	neural	full belief	900	98.4%	97.6%	97.2%	97.1%	95.5%
`pilot_learned_init_neural_full_belief`	learned	neural	full belief	1100	84.2%	72.4%	64.9%	57.8%	39.3%
`pilot_learned_init_exact_final_query`	learned	exact	final query	900	89.3%	82.7%	80.1%	78.9%	70.2%

Pilot interpretation:

Learned transition rules are highly learnable when the slot support is initialized correctly. The oracle-initialized neural transition reached 97.1% query mass and 95.5% belief mass at held-out length 12.
Learned initialization is the harder subproblem. Even with exact transitions, the full-belief initializer plateaued near 58.1% belief mass at length 12.
Final-query supervision trained the initializer better than full prefix belief supervision in this pilot, reaching 70.2% belief mass at length 12.
The fully learned model did not compose the two subskills well at this scale; it fell to 57.8% query and 39.3% belief at length 12.

Main sweep decision:

Include an exact oracle ceiling at modulus 31.
Scale the successful learned-transition condition: oracle init + neural transition + full belief.
Scale the strongest learned-initialization condition: learned init + exact transition + final query.
Do not spend a full modulus-31 budget on the fully learned condition unless the two staged main runs leave enough time and evidence requires it.

2026-06-21 Main Sweep

Ran three modulus-31 main variants with evaluation lengths 4, 8, 12, 16, and 24:

Variant	Init	Transition	Supervision	Steps	L=4 query	L=8 query	L=12 query	L=16 query	L=24 query	L=24 belief
`main_oracle_init_exact_ceiling`	oracle	exact	full belief	0	100.0%	100.0%	100.0%	100.0%	100.0%	100.0%
`main_oracle_init_neural_full_belief`	oracle	neural	full belief	1200	74.1%	60.3%	49.2%	41.2%	30.7%	12.3%
`main_learned_init_exact_final_query`	learned	exact	final query	900	84.8%	74.6%	66.4%	59.3%	49.1%	36.5%

Strict belief mass:

Variant	L=4 belief	L=8 belief	L=12 belief	L=16 belief	L=24 belief
`main_oracle_init_exact_ceiling`	100.0%	100.0%	100.0%	100.0%	100.0%
`main_oracle_init_neural_full_belief`	34.4%	27.5%	21.3%	17.2%	12.3%
`main_learned_init_exact_final_query`	60.0%	54.5%	49.0%	44.2%	36.5%

Main interpretation:

The exact oracle ceiling remains perfect at modulus 31, so the slot format and evaluation path can represent the target state exactly.
Neural transitions learned well at modulus 11 but did not scale to modulus 31 with this generic MLP transition. Even with oracle slot initialization, held-out length-24 belief mass reached only 12.3%.
Learned initialization with exact transitions and final-query supervision scaled better than the neural transition at modulus 31, reaching 49.1% query mass and 36.5% belief mass at length 24.
Neither learned component recovered the exact sparse support machine at modulus 31. The main bottleneck is now learnability and inductive bias, not representational sufficiency.

Analysis artifacts generated:

analysis/all_metrics_long.csv
analysis/all_metrics_query_mean.csv
analysis/first_k_ge_l_summary.csv
analysis/train_log.csv
analysis/summary.md
analysis/figures/

2026-06-21 Final Audit

Report artifacts generated:

reports/learned_sparse_slot_paper.md
reports/learned_sparse_slot_paper.html

Consistency checks:

Source compilation passed for src/learned_sparse_slot_experiment.py and src/analyze_learned_sparse_slot.py.
The experiment directory is 3.5M and contains no .pt, .pth, or .ckpt files.
External artifacts are stored under large_artifacts/learned_sparse_slot_executor/ and total 2.9M.
checkpoint_manifest.csv contains 10 rows; every listed checkpoint exists and matches the recorded byte count.
The HTML report references 5 local figure files, and all referenced figures exist.
The standalone paper markdown and HTML were checked for backward-looking phrases such as "previous", "prior", "original", "follow-up", "earlier", "older", "past experiment", and "another experiment"; no matches were found.

Final status:

The experiment has its own subdirectory with source, runs, analysis, reports, and a progress log.
Large model checkpoints are separated from the experiment directory and indexed by manifest.
The write-up is standalone and does not rely on any earlier experiment.