Qwen3.5-4B verified-macro capacity-fit vLLM rerun
The one idea you need
A server holds each active prompt's growing memory in a fixed pool of fast storage — a parking garage with set spaces. Raising the context limit lets each car park taller but adds no spaces; admit more full prompts than fit, and cars get towed and re-parked, wasting work.
The question
If you give a small model a huge context and a huge thinking budget, is that enough to make it actually reason over long inputs?
What we found
No. Two hidden limits bit first. Configuring the server for 64 simultaneous long prompts demanded about 2.4 million tokens of fast memory from a pool holding roughly 1 million, forcing constant re-reading. Cutting to 19 prompts fixed that overflow — but the model then looped instead of finishing: 37 of 48 attempts fell into exact repeating cycles, and none stopped on their own.
Why it matters
Before serving long-context batches, don't just raise the length limit. Measure your live fast-memory pool and cap concurrent prompts so worst-case demand fits with room to spare. And separately, guard against models that loop forever instead of stopping.
On this page
Results at a glance 2
How to read
Two setups on the horizontal axis: the over-committed 64-prompt config and the fitted 19-prompt config. For each, one bar is worst-case memory demand and the other is memory actually available, in cache tokens; demand below available is safe, and lower demand is better.
Takeaway → At 64 prompts, demand towers to about 2.4 million against a roughly 1-million cache — guaranteed overflow and re-reading. At 19, demand drops just under the cache.
Data table
| condition | block-rounded worst-case demand | measured live KV capacity |
|---|---|---|
| predecessor max-seqs 64 | 2.433M | 995.3k |
| capacity-fit max-seqs 19 | 963.1k | 997.9k |
Technical framing
Capacity fitting moves worst-case demand below live KV capacity — The predecessor's 48 logical sequences could demand 2.43M tokens from a 995,328-token cache. The completed fresh K=4 probe used max-seqs 19; with 528-token blocks and a 50,688-token rounded worst sequence, it required 963,072 of 997,888 live tokens, leaving 34,816. The strict capacity-fit run was nevertheless 19.6% slower than the old max-seqs-64 diagnostic, so cache-safe concurrency is not automatically throughput-optimal.
How to read
Three failure types along the horizontal axis: prompts that never resolved, prompts stuck repeating themselves, and prompts that maxed out the answer length. Each shows its observed share of the 48 samples beside its allowed ceiling; staying below the ceiling passes.
Takeaway → Every observed bar overshoots its ceiling — repeating loops hit about 77% against a 25% cap — so the model kept rambling instead of finishing, and nothing could be scored.
Data table
| condition | observed share | registered maximum |
|---|---|---|
| unresolved boundary contacts | 22.9% | 5% |
| exact periodic loops | 77.1% | 25% |
| answer-limit contacts | 18.8% | 5% |
Numbers from analysis/scientific_smoke_49k_termination_audit.json
Technical framing
Capacity fitting did not clear the 49k termination boundary — All 48 samples contacted the reasoning boundary. The fresh 49k rung failed all three content-blind thresholds. Its 61k successor was stopped before a receipt after the implicit CUDA-graph list proved not to cover active width 15; it left no reusable rows. No decoded output or score was inspected, so this is not a macro result.
In the author’s words from the Report · “Summary”
Stopped infrastructure status: the fresh capacity-fit K=4 probe at think@49,152 passed its live KV-cache fit check but failed the registered content-blind termination gate. The subsequent think@61,440 K=4 probe passed preflight, then was manually stopped before any result when a CUDA-graph capture-geometry mismatch was identified. Its preflight-only artifact is preserved. No decoded or scored content has been inspected, and no macro claim is eligible. The key correction is to treat long-context inference as a joint context-and-concurrency problem. The direct parent's larger thinking allowance retained max_num_seqs=64, which did not conservatively fit 64 maximum-length contexts in the observed KV cache. … Read the full result →
Overview
Status: the capacity-fit 49,152-token K=4 probe is termination-inadequate. The fresh 61,440-token K=4 probe passed its live-capacity preflight, then was manually stopped before any result after a CUDA-graph capture-geometry problem was identified. Its preflight-only artifact is preserved; no decoded or scored content has been inspected, and there is no macro claim.
Research program
- Primary program:
operator_and_skill_inventories. - Direct parent:
qwen35_4b_verified_macro_long_context_rerun. - Closest near-duplicate: the direct parent. This is a separate experiment because it uses a new scheduler geometry, fresh external namespace, and independent stop rule. Its bytes may never be pooled with or promoted from the parent's
max_num_seqs=64diagnostics.
Question
The preceding long-context attempt showed that a 768-token cap was plainly too small, then reached another setup boundary at much larger contexts. Did those later observations reflect a verified-macro limitation, or did max_num_seqs=64 ask vLLM's scheduler to support more simultaneous near-65k contexts than the live KV cache could hold?
This follow-up tests the latter possibility before drawing any semantic conclusion. It freezes capacity-fit concurrency at 19 sequences for a 49,152-token thinking allowance and 15 for 61,440, then verifies the actual vLLM cache capacity and block rounding after engine construction and before generation. Only a complete, termination-adequate K=12 base/designed matrix may be decoded and graded.
The 12 smoke-v2 tasks are the frozen, previously unscored v2 smoke matrix under an independent capacity-fit protocol. They are not newly generated or model-unseen: the stateless model received these prompt identities during predecessor diagnostics. No predecessor output is imported, read, pooled, or scored here.
Frozen protocol
- Model: only
Qwen/Qwen3.5-4B, revision851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. - Backend: only the experiment-local vLLM runner, SHA-256
fd9972bdcb3a9e8b9841b45ed8e2849017a6e80b601e924817cdaaa5144b8782, byte-identical to the direct parent. - Engine:
max_model_len=65536,gpu_memory_utilization=0.9,max_num_batched_tokens=32768, prefix caching off, asynchronous scheduling off. - Rungs:
(thinking_budget, max_num_seqs) = (49152, 19), (61440, 15). - Sampling: temperature 0.6, top-p 0.95, top-k 20, answer allowance 512, run seed 2701.
- Probe: fresh base K=4, termination-only. Probe rows are never K=12 evidence.
- Selectable matrix: fresh base K=12 plus fresh designed-ceiling K=12 at one rung.
- Storage: a new fail-closed external root, receipts written last, and a tracked checksum catalog.
At the 995,328-token cache capacity measured on the predecessor engine, the longest frozen designed prompt gives conservative block-rounded requirements of 963,984 tokens at 49k and 945,360 at 61k. Those are planning numbers, not assumed facts: each invocation must pass the same calculation using the newly constructed engine's live kv_cache_size_tokens, block size, model context, and rendered prompt lengths.
Setup with uv
From the repository root, create the pinned vLLM environment only if it is absent:
uv venv --python 3.12 .venv-vllm
uv pip sync --python .venv-vllm/bin/python --torch-backend=cu129 requirements-vllm.lock.txt
uv pip check --python .venv-vllm/bin/pythonRun the model-free gates before reserving the GPU:
.venv-vllm/bin/python -m unittest discover \
-s experiments/qwen35_4b_verified_macro_capacity_fit_rerun/tests -v
.venv-vllm/bin/python \
experiments/qwen35_4b_verified_macro_capacity_fit_rerun/scripts/run.py --validateGPU runbook
Coordinate with the owner of any existing GPU process before launching. Each command below creates exactly one vLLM engine, invokes generate_vllm_batch once for one experiment phase, commits one bundle, and exits. The budget wrapper may make a stage-one vLLM call and a stage-two continuation call for forced closures within that phase. The default root is /workspace/large_artifacts/qwen35_4b_verified_macro_capacity_fit_rerun/scientific_smoke_v1; QWEN35_MACRO_CAPACITY_FIT_ARTIFACT_ROOT may point to another absolute, nonsymlinked root that does not contain or overlap the predecessor root.
The commands below record the frozen protocol that produced this experiment's artifacts. Do not resume the interrupted 61k probe in this experiment. Correcting the resolved CUDA-graph capture geometry changes the inference protocol and therefore requires fresh rows in a separate follow-up namespace rather than filling this preflight-only bundle.
Start only with:
.venv-vllm/bin/python \
experiments/qwen35_4b_verified_macro_capacity_fit_rerun/scripts/run.py \
--phase probe --budget 49152If its returned termination audit is adequate, run the two new K=12 arms:
.venv-vllm/bin/python experiments/qwen35_4b_verified_macro_capacity_fit_rerun/scripts/run.py \
--phase base --budget 49152
.venv-vllm/bin/python experiments/qwen35_4b_verified_macro_capacity_fit_rerun/scripts/run.py \
--phase designed --budget 49152If the probe or either K=12 arm is rejected by the registered termination gate, move to the next rung with a new probe; do not reuse any 49k row:
.venv-vllm/bin/python experiments/qwen35_4b_verified_macro_capacity_fit_rerun/scripts/run.py \
--phase probe --budget 61440Run base and designed at 61k only if that probe passes, using the same command shape as above. The runner rejects out-of-order phases, an already terminal history, partial/unknown external files, runtime drift within a rung, and any capacity audit that does not fit.
After a passing K=12 matrix has been selected, and only then, run:
.venv-vllm/bin/python \
experiments/qwen35_4b_verified_macro_capacity_fit_rerun/scripts/analyze.pyThe analyzer re-verifies every receipt and recomputes the complete lower-rung termination history before decoded text is parsed or inspected and before hidden task data is loaded. A terminal 61k rejection writes pass:false with no selected bundle; that history is a setup boundary, not a negative macro result.
Capacity-fit 49k result
The fresh base-only K=4 probe at think@49,152 passed the live scheduler-capacity check before generation: max-seqs 19 required 963,072 block-rounded tokens from 997,888 live KV-cache tokens, leaving 34,816 tokens of headroom with 528-token blocks. Its receipt and tracked catalog verify against the external bytes.
Termination nevertheless remained inadequate. All 48 samples ended by stage-one length, were force-closed, and contacted the reasoning boundary. Token-ID-only periodicity classified 37 as exact periodic loops and left 11 unresolved; 9 answer stages reached their limit. These rates fail all three registered thresholds. The probe produced 2,364,643 sampled tokens in 5,012.451 seconds, or 471.754 tokens/s. No decoded output, parser result, correctness value, or hidden example was inspected.
The strictly capacity-fit run was 19.6% slower than the predecessor's max-seqs-64 diagnostic (471.754 versus 586.471 sampled tokens/s), but that comparison is confounded by CUDA-graph geometry and cannot isolate the cost of avoiding cache recomputation. The predecessor initially admitted all 48 logical sequences and had a captured width-48 decode graph. Its 995,328-token cache could hold only about 20,736 total context tokens per sequence at that width, after which cache pressure could preempt requests and recompute prefixes. The max-seqs-19 run bounded that cache demand, but vLLM resolved the runner's requested graph maximum of 19 to [1, 2, 4, 8, 16]; decode widths 17--19 therefore ran without a CUDA graph. The runs differed by only 1,977 sampled tokens (less than 0.1%) while wall time differed by 24.2%. Sampled-token throughput also excludes any recomputed prefix tokens, so the faster overcommitted diagnostic does not establish lower actual model compute.
Thus cache-safe concurrency and capture-aware throughput tuning are separate requirements. The 49k rung remains rejected by its termination gate, and its throughput comparison remains infrastructure diagnosis only, not evidence for or against verified macros.
Clean 61k interruption
The fresh base-only K=4 probe at think@61,440 started under the frozen max-seqs-15 protocol. Before generation, its live preflight passed: 15 active sequences required 950,400 block-rounded tokens from 997,888 live KV-cache tokens, leaving 47,488 tokens of headroom with 528-token blocks. The 7,053-byte preflight is preserved at smoke_budget_probes/think_61440/base.preflight.json, SHA-256 a2a3ef1f4ba9e68909374460030bc947712f10a488870a0db1bf081e368b8a5a.
While the model call was running, a source-level audit found that the runner requested max_cudagraph_capture_size=15, but vLLM 0.24 constructs default capture sizes at 1, 2, 4, and then multiples of 8. It therefore resolved the effective maximum to 8, so decode widths 9--15 used no CUDA graph. The process was manually stopped before rows, runner metadata, or a last-written receipt existed. The tracked catalog consequently and correctly records this bundle as preflight_only. There is no 61k termination count, timing result, decoded output, score, or scientific outcome to infer from the interrupted call.
Decision rule
Termination selection uses token counts, finish metadata, and token-ID periodicity, never decoded answers, parser outcomes, correctness, or hidden examples. A probe/arm is adequate only when:
- unresolved reasoning-boundary contact rate is strictly below 5%;
- answer-limit contact rate is strictly below 5%; and
- detected periodic-loop contact rate is at most 25%.
The first rung with an adequate probe and both adequate K=12 arms is selected. Semantic smoke then requires parse rate at least 0.5 in each arm, valid macro candidates on at least two reuse tasks, and designed reuse oracle coverage no lower than base. Smoke is an interface gate, not a capability-gain claim; matched-compute sampling remains mandatory for any later scientific claim.
Artifacts
data/source_provenance.json: exact copied-file and predecessor-boundary provenance.data/prompt_manifest.json: ordered content-blind prompt identities; no model output.reports/preregistration.md: frozen hypotheses, branch logic, and thresholds.reports/design_review.md: adversarial pre-launch review and required fixes.analysis/scientific_smoke_49k_termination_audit.json: receipt-bound, content-blind 49k capacity, termination, exact-period, and throughput audit.analysis/scientific_smoke_artifact_catalog.json: complete 49k bundle plus the preserved, checksum-bound 61k preflight-only interruption state.src/scientific_artifacts.py: external bundle, receipt, catalog, and selection validator.scripts/run.py: one-engine/one-phase vLLM runner.scripts/analyze.py: content-blind termination finalizer and post-gate semantic analyzer.reports/artifact_manifest.yaml: external-storage and regeneration contract.
Report
Rendered from reports/report.md
Summary
Stopped infrastructure status: the fresh capacity-fit K=4 probe at think@49,152 passed its live KV-cache fit check but failed the registered content-blind termination gate. The subsequent think@61,440 K=4 probe passed preflight, then was manually stopped before any result when a CUDA-graph capture-geometry mismatch was identified. Its preflight-only artifact is preserved. No decoded or scored content has been inspected, and no macro claim is eligible.
The key correction is to treat long-context inference as a joint context-and-concurrency problem. The direct parent's larger thinking allowance retained max_num_seqs=64, which did not conservatively fit 64 maximum-length contexts in the observed KV cache. This follow-up uses 19 sequences at 49,152 thinking tokens or 15 at 61,440 and requires the newly constructed vLLM engine to prove live block-rounded cache fit before generation.
Research program fit
This experiment belongs to operator_and_skill_inventories. It does not change the macro mechanism or claim that designed macros improve capability. It repairs the inference-validity gate needed before the frozen base/designed smoke can say anything about the interface.
Method
- Only
Qwen/Qwen3.5-4Bat the pinned revision, through the byte-identical local vLLM runner. - Frozen 12-task v2 smoke, base and designed libraries, demonstrations, prompts, sampling, and semantic rules.
- A new K4 base termination probe at each rung, permanently excluded from semantic selection.
- New K12 base and designed arms at the same first adequate rung.
- Live KV token capacity, block rounding, context, and worst rendered prompt checked before rows.
- Content-blind termination selection followed by full receipt/history re-verification before decoded text is parsed or inspected and before hidden task data is loaded.
- Fresh external namespace with receipt-last commits, checkpoint catalog, fixed lock, and strict crash reconciliation.
These v2 tasks are frozen and previously unscored under this independent protocol, but are not model-unseen: predecessor diagnostics already called the same prompt identities. No predecessor output is imported or scored.
Results
Model-free gates
- Frozen protocol and exact record identities: pass.
- Unit tests: 37/37 pass.
- vLLM runner byte parity with direct parent: pass.
- GPU generation: 49k K=4 complete; 61k K=4 stopped before result.
- Termination selection: 49k rejected; no 61k termination result exists.
- Semantic smoke: ineligible until a complete K12 matrix passes termination.
Scientific result
The 49k probe's live capacity audit passed before generation:
- live KV capacity: 997,888 tokens;
- cache block size: 528 tokens;
- capacity-fit concurrency: max-seqs 19;
- block-rounded demand: 963,072 tokens; and
- remaining margin: 34,816 tokens.
Its complete receipt binds 12 records × K=4 = 48 samples. Every sample ended by stage-one length, was force-closed, and contacted the reasoning boundary. The frozen token-ID periodicity rule found 37 exact periodic loops, with periods recorded in analysis/scientific_smoke_49k_termination_audit.json; the other 11 contacts remained unresolved. Nine answer stages reached the 512-token limit. Thus unresolved (22.92%), loop (77.08%), and answer limit (18.75%) rates each fail its registered threshold. This rung is rejected before decoding or scoring.
The probe sampled 2,364,643 tokens in 5,012.451 seconds (471.754 sampled tokens/s). It was 19.6% slower than the predecessor's max-seqs-64 diagnostic at 586.471 tokens/s, but this is not a clean test of cache-fit overhead. Max-seqs 64 initially admitted all 48 logical sequences and supplied a captured width-48 decode graph. Its 995,328-token cache could hold only about 20,736 context tokens per sequence at that width, so later cache pressure could preempt requests and recompute prefixes. Max-seqs 19 bounded worst-case cache demand, but vLLM resolved its requested CUDA-graph maximum of 19 to [1, 2, 4, 8, 16], leaving decode widths 17--19 uncaptured. The two probes differed by only 1,977 sampled tokens (less than 0.1%) while wall time differed by 24.2%, and sampled-token throughput does not count recomputed prefix work. The comparison therefore distinguishes neither actual model compute nor a throughput optimum.
This is a termination/provisioning result only. It is not evidence for or against verified macros, and it does not authorize semantic analysis.
Clean 61k interruption
The fresh max-seqs-15 61k invocation passed its live preflight before generation. Fifteen active sequences required 950,400 block-rounded tokens from 997,888 live KV-cache tokens, leaving 47,488 tokens of margin with 528-token blocks. The preserved preflight is 7,053 bytes with SHA-256 a2a3ef1f4ba9e68909374460030bc947712f10a488870a0db1bf081e368b8a5a.
The installed vLLM 0.24 source then exposed a second geometry constraint: the runner requested a maximum CUDA-graph capture size of 15, while the default capture ladder contains 1, 2, 4, and then multiples of 8. The effective maximum was consequently 8, and decode widths 9--15 used no CUDA graph. The process was manually stopped before rows, runner metadata, or a receipt existed. The catalog correctly preserves only probe/think_61440/base.preflight.json and marks the bundle preflight_only. No 61k token counts, timing, termination verdict, decoded output, or score exists; the interruption cannot be interpreted as either a positive or negative result.
Controls
The implementation rejects the predecessor root and old max-seqs-64 metadata, proves K4 cannot resolve a selected K12 arm, enforces same-rung runtime/engine identity, disallows phase carryover, recomputes the entire first-adequate history, and writes an explicit unselected terminal history if 61k fails. Tests also establish that changing decoded answers or correctness fields cannot change termination selection.
Oracle versus deployable evidence
If termination clears, sample selection is visible-only: earliest valid candidate among those with maximal visible score. Hidden examples grade only after that index is frozen. Oracle hidden coverage is a nondeployable smoke diagnostic and cannot support a capability claim.
Interpretation
The current learned lesson is operational: increasing max_model_len or generation allowance is not sufficient for long-context batched inference. max_num_seqs must be fitted to live block-rounded KV capacity at the intended per-sequence reserve, and CUDA-graph capture sizes must cover the intended decode concurrency. Both resolved values are part of the inference protocol. Maximum cache-safe concurrency is not automatically a throughput optimum, but the present timings cannot quantify that tradeoff because cache overcommit and graph coverage changed together.
The eventual branches are preregistered:
- live fit failure means infrastructure mismatch;
- terminal 61k termination failure remains setup-inconclusive;
- termination pass plus semantic failure indicates an interface/induction ceiling under the frozen protocol; and
- smoke pass licenses a separate matched-compute mechanism experiment, not a direct positive claim.
Next action
Do not resume the preflight-only 61k bundle. A capture-aware rerun changes Ada batch geometry and must begin with fresh rows under a separate follow-up protocol. This experiment remains stopped with one rejected 49k termination probe and one preserved 61k preflight; no K12 matrix is authorized.
Experiment log 4
Show the running log (4 entries, 2026-07-10)
2026-07-10 — experiment split and protocol freeze
- Created a new experiment rather than changing the result-bearing direct parent.
- Copied the frozen tasks, demonstrations, libraries, macro DSL, model harness, and vLLM runner. The six byte identities are recorded in
data/source_provenance.json; the runner remains byte-identical at SHA-256fd9972bdcb3a9e8b9841b45ed8e2849017a6e80b601e924817cdaaa5144b8782. - Imported no predecessor model-output artifact. Extracted only the ordered prompt ID, input hash, rendered-prompt hash, and prompt-token count from predecessor preflights into
data/prompt_manifest.json. - Corrected the research language: these are frozen, previously unscored v2 smoke tasks under an independent protocol, not newly model-unseen tasks.
- Froze the capacity-fit ladder at 49,152/19 and 61,440/15 with a 65,536-token model context and 512-token answer allowance. The live engine, not the historical cache number, must prove fit.
- Separated termination-only K=4 probes from selectable K=12 base/designed bundles by namespace, role, receipt geometry, catalog ID, and selection validation.
- Added a fresh external root and a fixed nonblocking experiment lock. The predecessor root, parent/child aliases, symlinks, unknown files, and partial bundles fail closed.
- Added checkpointed catalog reconciliation. The only legal mutation is a registered preflight-only entry becoming receipt-complete; disappearing or silently appearing bundles fail.
- Added whole-history first-adequate selection and a terminal
pass:false, selected=nullrecord for 61k exhaustion. The analyzer recomputes every recorded lower-rung termination metric before any decoded or hidden-content access.
2026-07-10 — model-free verification
- Frozen-protocol validation passed with the exact smoke-v2 record hashes: base
bd66aa64942f9e57e1fe55ae716c154ea1231480d6163f1811a07828ba364907and designedc5a6cd00d9600b7a63c8e2c132e202b25da30f30af299afb3735a8f5525d9e86. - All 37 CPU tests passed. Coverage includes cache-fit rejection, live-context enforcement, predecessor-root exclusion, old
max_num_seqs=64rejection, same-rung runtime identity, K4 nonpromotion, phase transitions, receipt/catalog crash reconciliation, first-adequate success, terminal 61k failure, idempotence, lower-tier outcome-shopping detection, and hidden-content nonaccess before termination eligibility. - No GPU engine or model call was launched from this experiment. Launch remains pending explicit coordination with the owner of the current GPU process.
2026-07-10 — 49k capacity-fit probe rejected; 61k probe started
- The fresh base-only K=4 think@49,152 preflight passed against the constructed engine's live cache: max-seqs 19, 997,888 KV-cache tokens, 528-token blocks, 963,072 block-rounded tokens of demand, and 34,816 tokens of margin.
- The last-written receipt and tracked catalog verify against all external file sizes and SHA-256 digests. The 12 records produced 48 completions and 2,364,643 sampled tokens in 5,012.451 seconds (471.754 sampled tokens/s).
- Recomputed the frozen content-blind gate without exposing decoded or scored content. All 48 samples ended by stage-one length, were force-closed, and contacted the reasoning boundary; token-ID periodicity classified 37 exact loops, 11 contacts remained unresolved, and 9 answers reached the 512-token limit. The exact periods and receipt hashes are preserved in
analysis/scientific_smoke_49k_termination_audit.json. - The 49k rung fails all three registered termination thresholds and is excluded before parsing, correctness, hidden labels, or any macro comparison. This is not a negative macro result.
- Strict capacity fitting did not maximize throughput: 471.754 tokens/s is 19.6% below the predecessor max-seqs-64 diagnostic's 586.471 tokens/s. Cache-safe scheduler geometry is necessary for validity but is not automatically throughput-optimal.
- Advanced exactly one rung. A fresh base-only K=4 think@61,440 probe at max-seqs 15 is now active; no 49k K=12 arm was launched.
2026-07-10 — 61k probe manually stopped pre-receipt after capture audit
- The 61k engine constructed successfully and wrote its checkpointed preflight before generation. The live audit passed with max-seqs 15, 997,888 KV-cache tokens, 528-token blocks, 63,360 rounded tokens per sequence, 950,400 tokens of demand, and 47,488 tokens of margin.
- The preserved 7,053-byte preflight has SHA-256
a2a3ef1f4ba9e68909374460030bc947712f10a488870a0db1bf081e368b8a5a. The tracked catalog recordsprobe/think_61440/baseaspreflight_only; there are no rows, runner metadata, or receipt. - A content-blind audit of the installed vLLM 0.24 scheduler/dispatcher found a separate capture-configuration mismatch. The frozen runner requested
max_cudagraph_capture_size=15, but vLLM's default capture ladder retained only[1, 2, 4, 8]and resolved the effective maximum to 8. Decode token batches wider than 8 therefore dispatched without CUDA graphs. The same issue affected the completed max-seqs-19 probe at widths 17--19 because its effective graph maximum was 16. - The 61k process was manually stopped before it could produce a result. No decoded output, token termination classification, parse result, correctness value, hidden label, or score was read or inferred. The interruption is not a failed 61k termination probe and does not advance the frozen state machine.
- The prior 49k throughput comparison is consequently confounded: max-seqs 64 supplied a captured width-48 decode shape but overcommitted the KV cache, whereas max-seqs 19 fit the cache but lost CUDA graphs at its maximum widths. Its 1,977-token output-count difference was below 0.1%, versus a 24.2% wall-time difference. Moreover, sampled-token throughput omits model work spent recomputing preempted prefixes. Neither timing identifies the throughput optimum or changes the termination-only scientific verdict.
- This result-bearing experiment, its source, and its frozen protocol remain unchanged. A corrected capture list changes Ada batch geometry and must be tested with fresh rows under a separate follow-up protocol rather than by resuming this preflight.
Reproduce
Run steps are documented inside the experiment folder (README and scripts).