Process Control and Tool Use

Train or evaluate small models as controllers over tools, verifiers, budgets, and intermediate actions.

What we have learned

Seed Experiments

Current Read

Tool-state policies are a promising way to make small models useful inside iterative systems, but they need visible-only evaluation discipline.

qwen35_4b_repo_search_compress_bank: exact marginal balance over inspect/patch/verify/commit installed a perfect four-step trained-family path but regressed family-disjoint repository success 49/72→25/72. After failed tests the control revised with another patch 24/26 times; compact revised 0/48 times, while commit after pass stayed intact. Process-control curricula must preserve verifier-conditioned transitions and changed recovery actions, not merely terminal success traces or operator totals.
qwen35_4b_specialist_policy_integration: no new tool policy was trained. The only registered tools family scored 0.994 under the installed incumbent, leaving 0.006 maximum headroom against a frozen +0.10 qualification rule. This is a substrate-selection negative: tool-policy experiments need an unsaturated core before adaptation quality is testable.

Scorecard

Program: charter
Current read: small models need explicit control policies for tools, budgets, and commit/repair decisions. Exact operator marginals are not enough: a compact repository bank preserved commit after pass but learned zero patch recovery after 48 failed tests, regressing held-out success 49/72→25/72.
Best next experiment: compare STOP/MORE, commit/repair, and tool-choice policies with strict cost accounting and explicit failed-patch/failed-test transition gates; a recovery curriculum must contain changed second actions, not only successful terminal traces.
Strong anchors: qwen35_4b_adaptive_tool_controller, qwen35_4b_tool_state_policy_lora, qwen35_4b_adaptive_evidence_budget_policy, qwen35_4b_repo_search_compress_bank.
Avoid repeating: tool-use demos that omit the no-tool, fixed-tool, and random-tool baselines, or banks that balance action counts while deleting verifier-rejection contingencies.
Evidence that advances the program: policy lift survives latency, token, and tool-call ceilings.

Charter

Show charter.md

Purpose

Use small models as controllers over tools, verifiers, budgets, repair loops, and intermediate actions rather than single-shot answer generators.

Why This Is A Program

Many future lines will need models that decide what to do next: call a tool, gather evidence, stop, repair, search, or commit. That deserves a program-level treatment across substrates.

Progress Signals

Controllers improve utility at fixed budgets.
Tool-state representations transfer across pools or tasks.
STOP/MORE and commit/repair decisions are calibrated.
Policies remain useful when oracle labels are removed.

Boundaries

This line owns process decisions. It often collaborates with Active Evidence Acquisition and Evidence-Conditioned Selection.

Backlog

Show backlog.md

Next Experiments

The specialist-policy integration attempt stopped before policy learning: its sole tools core was already at 0.994, making a mandatory +0.10 gain impossible under the 1.0 score cap. Any successor must add a harder, independently calibrated tools/provenance core and run a per-core ceiling/headroom gate before best-of-k or training.
Build a common tool-state schema for direct answer, executable program, verifier result, repair history, and budget.
Compare learned controllers with simple heuristics on identical pools.
Train STOP/MORE policies under visible-only labels and evaluate under hidden labels.
Stress process policies with noisy tools and misleading visible examples.
Measure whether controllers transfer from table tasks to code or text transformations.

Required Controls

Fixed budget baseline.
Always-stop and always-continue policies.
Random action policy.
Oracle policy ceiling.

Stop Conditions

Do not claim process control progress unless the learned policy beats simple budget heuristics under the same evidence constraints.

Experiments 58

2026-07-19 Qwen35 4B Agentic RLVR Feasibility
Yes to the machinery: with three fixes (force eager mode so the hybrid architecture doesn't hang, load the model in bf16 not fp32, and split GPU memory 55% to the inference engine with sleep-mode) the loop CLOSES - a mov
2026-07-18 → Qwen35 4B WHY-Think Scale
This phase built and proved the machinery; the GPU training sweep has not run yet. The generator now emits, for every example, a real hidden reasoning trace generated mechanically from the program's shape - parse the spe
2026-07-18 → Qwen35 4B WHY Scale Ladder
This phase built and proved the machinery; the GPU training sweep has not run yet. The core blocker was that the original WHY generator saturated fast (about 75 distinct reasons, 438 distinct programs at 504 examples), s
2026-07-16 Repair-Verifier Signal Probe
The gate said no, cleanly. Handed two candidate repairs and the full failure evidence — a task solvable by mentally running each candidate through both trials — the model picked the working one 51.5 percent of the time w
2026-07-15 Interleaved-Replay Dose with Medium Pilot
The review round did not prevent forgetting: the dosed model lost nine to ten retained answers against both comparisons — almost exactly the cost of dosing directly — so the theory drawn from comparing old receipts is re
2026-07-14 Policy-Supported Successful-Sibling Universal Curriculum
This version cannot run its second step. Across 624 first attempts it found 227 failures, but counting and routing had none and selection had only two; the plan required four failures in every skill. It therefore stopped
2026-07-14 Search-Scaffold Universal Curriculum
No. On 26 fresh paired cases, the staged-search model tied replay at 16 correct and trailed its parent at 18. All three parsed 23 answers and hit the limit three times; the candidate solved none of four execute/induct ca
2026-07-14 On-Policy Failure-Prefix Universal Curriculum
No. The unmodified parent solved 16 of 26 fresh tasks and equal-compute replay solved 18, while training corrections after the model's own real failures solved only 15. The repair model got none of the six targeted execu
2026-07-14 Failure-Selected Counterfactual Restart Curriculum
The data-selection stage passed: from 624 fresh attempts, the frozen rule selected 52 clean restarts, exactly four for each of 13 skills. Forty correct accuracy or cap failures and twelve correct-but-too-long cases are n
2026-07-13 Validation-policy counterexample curriculum
The test could not answer the question. Before the newly trained model was allowed to run, the starting model and the equal-size old-lesson control each fixed all 48 practice cases. The required gains were therefore impo
2026-07-13 Replay-Anchored Universal Curriculum Continuation
No at this dose. The designed mix passed its fresh synthetic screen but scored about 42% overall, below both the 44% mature policy and the 49% replay-only comparison. It also fell below base on one family. Replay-only wa
2026-07-13 Mid-Density Token-Matched Universal Curriculum
The 160-lesson stream improved fresh local accuracy from 17 to 19 of 26 cases, raised readable answers from 18 to 23, and cut answer-limit contacts from nine to three. It still missed the readability and length gates by
2026-07-13 Low-Density Token-Matched Universal Curriculum
No. Neither 40 nor 80 abstract lessons passed the fresh local screen. The stronger 80-lesson stream matched the inherited anchor at 54% accuracy and parsed 62% of answers, but still hit the answer cap 10 times; broad eva
2026-07-13 Qwen3.5-4B: Installing Universal Features via Designed Synthetic Curricula
Not with either tested training geometry. Continuing from the strong policy raised fresh synthetic accuracy from 50% to 69% and beat the untrained base overall, but it scored 31% aggregate versus 45% for the strong start
2026-07-13 State-Formation Capacity Adjudication
Not run yet. The reviewed test starts with three compact-update training runs. If any required state check misses, six matched controls become mandatory; the final three runs open only if the full-size version also misse
2026-07-13 Semantic-policy headroom tournament
No rule produced a stable training target after a failed test: every one of 72 cases reached a correct patch. But before seeing test output, the model got zero of 54 ambiguous first patches fully right and never opened t
2026-07-13 Qwen3.5-4B Jacobian Counterfactual Branching
No. Across all three allowed strengths, the meaningful nudge made its assigned answer win only 4 of 48 times—exactly the one-in-twelve chance rate and identical to a generic nudge. The probabilities barely moved even at
2026-07-13 Qwen3.5-4B Early Text Hypothesis Forking
The experiment design and model-free checks now pass, but the model has not run. Review expanded the first draft from twelve operation names to twenty-four fully specified operations, added two fair late-hint comparisons
2026-07-13 Counterfactual evidence-acquisition curriculum
The run stopped at its first checkpoint-compatibility test. Across 48 frozen contexts, the starting checkpoint differed from the comparison anchor by 0.110735 centered logits, above the fixed 0.100000 limit. Entropy stay
2026-07-13 Qwen3.5-4B: The Compute-Optimal Confidence Policy (select + abstain + escalate)
Partly. Picking the answer with the highest single-token P(True) confidence is the best grader-free selector at every compute level, and abstaining below a confidence threshold trades coverage for accuracy cleanly — both
2026-07-12 Transaction-invariant recovery curriculum
Partly. Focused training lifted practice repairs from 52% to 82% and preserved the test-and-revise loop. But on different transaction APIs it reached 72% versus 70% before training, far below the required ten-point gain.
2026-07-12 Verifier-conditioned recovery banking curriculum
Yes — replaying real failures taught recovery, lifting success from about half of cases to more than nine in ten. But the best version also added a little "explain your fix" text, and that supposedly tiny slice actually
2026-07-12 Qwen3.5-4B Same-Prefix Advantage Routing
Only when it replicates, and here one helper didn't. The test actually ran: the deep specialist's local edge held up on a fresh, separate batch, but the quick specialist looked strong on one batch of stuck points and the
2026-07-12 Repository search-compress-bank coding curriculum
No. It became flawless on the six codebase types it trained on, solving every task in exactly four clean steps. But on four never-seen codebase types its success collapsed from 68% to 35% — worse than just running the un
2026-07-12 Locality-first recovery-reason interpolation
Yes on skill, no on shipping. One dial setting recovered from broken code 97% of the time, about 12 points above the act-only version and 15 above a matched-training baseline, while barely moving unrelated behavior. But
2026-07-12 Payload-capable recovery agent harness
No. It led on the first unseen task block at 71%, but confirmed at 69%, exactly tied with the action-only model and short of the required lead. The external benchmark stayed sealed.
2026-07-12 Qwen3.5-4B Pareto Policy Integration
No. Retested on fresh, uncontaminated tasks, the version built for quick work actually lost at quick work by about two points, while the version built for long work won at both quick AND long tasks. One version quietly d
2026-07-12 Qwen3.5-4B Context-Local Jacobian Clamp
Yes, but only when the edit lands on the earlier token that first stores the word. There the model looked up the swapped word's digit on all 48 fresh puzzles, up from zero without the edit, and a wrong-word swap produced
2026-07-11 Qwen3.5-4B Specialist Policy Integration
No — we never found out. Each of four required skills had to gain ten points before merging, but scores top out at 100. One tool-use skill already sat at 99.4%, so its target of 109.4% was impossible. The experiment stop
2026-07-11 Interactive policy curriculum: oracle DAgger to execution-reward RL
No — it backfired badly. Success on the practice tasks fell from about 60% to 35%, and on tasks never touched in training from about 69% to 35%. It kept editing fluently but stopped running its checks: the verify-and-com
2026-07-07 Qwen3.5-4B: Does the Confidence Toolkit Survive on Real Code?
Yes, but not the obvious way. Averaging the model's certainty across every token of a program barely beats a plain majority vote among the tries. The real winner: make the model write the code, then answer one yes/no que
2026-07-05 Qwen3.5-4B Latent Decomposition: be your own tool-search
No, not out of the box. One step from the goal it picks the right operation about eight times better than chance, but on the opening move three steps out it ranks correctly only at chance — recognition, not planning. As
2026-07-04 Qwen3.5-4B Tool-Seeded Banking: does tool-search + banking cross the depth-3 wall?
Partly. A brute-force search found working three-step programs the model never produces itself, and retraining on them lifted three-step success from a hard zero to 5 of 40 fresh, never-seen tasks when it can reason acro
2026-07-02 Qwen3.5-4B Context Composition
Only when its answer survives. The fine-tuned skill is genuinely the sharpest — 95% correct when the model replies in the required form, beating the untrained model's 83% under the same step-by-step procedure. But the tr
2026-06-30 Qwen3.5-4B Thinking-Budget Scaling
Yes, but with a sweet spot. Turning on the model's built-in reasoning lifted the accuracy of the answer you would actually ship from 76% to 91% on basic Python tasks — a genuine capability gain, not just a wider pool of
~2026-06-30 Qwen3.5-4B Thinking-Budget Controller
For cost, yes; for accuracy, no. Drafting cheaply and thinking harder only when the draft fails the one visible example matches a generous fixed budget's accuracy (88%) using about a fifth of the thinking (113 versus 507
2026-06-28 Qwen Oracle-Distilled Acquisition Policy
No. The trained picker does read real signal: it beats revealing nothing (50 to 57 percent of tasks fully solved) and crushes a version fed scrambled answers (27 percent). But it lost to simply grabbing a varied spread o
2026-06-28 Qwen3.5-4B Tool State Policy LoRA
Surprisingly, yes, and with almost no learning. Trusting the program only when it passes a worked example and disagrees with the quick answer lifted accuracy from 56% to 66%, exactly matching the best any picker could re
2026-06-28 Qwen3.5-4B Live Tool DAgger
Only its cost, not its accuracy. Answering directly solved none of twelve unseen tasks; letting the model write and run code recovered two — about one in six — and spoiled nothing it already had right. But even a flawles
2026-06-28 Qwen3.5-4B Adaptive Tool Controller
Partly. One structural cue — does the direct answer have fewer columns than the raw data implies? — safely flags the reshaping tasks where a program helps, lifting accuracy from 42% to 50% with zero broken tasks. But it
2026-06-26 Qwen3.5-4B Pass@k Coverage RL
No. Retraining the model to spread its tries across different problems actually lowered coverage: it solved about 44% of unseen coding problems within four tries, versus about 69% from simply raising the randomness dial
2026-06-26 Qwen3.5-4B OPSD Pressure Locality Audit
No. At the exact spots where correct code diverges from code that passes surface tests but is secretly wrong, the reference hint adds essentially nothing — scoring no better than a scrambled, meaningless hint. The hint o
2026-06-25 → 26 Qwen3.5-4B Sampler Portfolio Scheduler
In theory yes, in practice no. A flawless chooser that knew which style would crack each problem solved 75% of coding tasks at the cheapest run's cost, while just doubling tries from one style reached 67% at twice the co
2026-06-24 → 26 Qwen3.5-4B Oracle Probe Synthesis MDP
It's the menu. Just curating which eight test inputs the model chose from raised success from about 43% to 49% — a bigger jump than any training gave. Supervised coaching added a bit more (48% to 51%); preference- and re
2026-06-25 Qwen3.5-4B Trained vs Frozen Repair MDP Report
Retry from scratch. Given the same compute budget, a few more fresh attempts rescued 5 of 57 stuck problems, while a fixer trained to patch broken code rescued only 2 — and its repairs secretly failed the hidden tests ne
2026-06-25 Qwen3.5-4B Strategy Token Diversity LoRA
No. Feeding the model labeled coding tactics recovered 37.5% of the problems it first failed, slightly below plain extra sampling at the same cost, which recovered 41.7%. Tellingly, scrambling the labels so they meant no
2026-06-24 Qwen Recurrent VM Repair Policy
Yes, but only partway. Letting the model run its program, read the output, and fix one line at a time roughly tripled accuracy, from about 11% to 34% on the main test and 16% to 40% on reworded prompts. But a perfect edi
2026-06-24 Qwen In-Policy VM-ECHO Distillation
Only partly. It became excellent at predicting mechanical outcomes, like how deep a program runs (about 92% right), but stayed no better than a coin flip at judging which program is actually correct. So it could not rank
2026-06-24 Qwen Fuyu VM GRPO-ECHO
No. Copying worked solutions alone solved about 10% of tasks; one round of reward-based self-coaching dropped that to 7-8%. The coaching produced encouraging signals — it could rank good fixes over bad ones 80% of the ti
2026-06-24 Qwen Action-Conditioned VM-ECHO Policy Iteration
Barely. Learning to grade drafts by their run results nudged picking accuracy only from about 10% to 11%, far short of the 37% reachable by always choosing the best available draft. It reliably picked programs that ran,
2026-06-24 Qwen3.5-4B Oracle Process GRPO
Yes, but with sharp limits. Teaching the model to copy a flawless test-picker lifted its success from about a third of puzzles to 43%, versus roughly 31% for picking blindly. Layering on preference-based and reward-based
2026-06-24 Qwen3.5-4B Learned Active Trace Policy
It depends. On the main test set a simple even-splitting rule beat the trained picker after one extra input — 91 percent of programs fully correct versus 87 — and the best-possible choice reached 97 percent. The picker d
2026-06-24 Qwen3.5-4B Deployable Information Ceiling Sweep
It's the clues. On the hard hidden rule, the plain strategy got just 4% right when starved of tests and clues, far below the 74% of a rival who secretly knew the answer. But allowing more tests lifted it to 86%, and more
2026-06-24 Qwen3.5-4B Bucket-Belief Probe Ranker
Barely. After training, the 4-billion-parameter model solved 50% of puzzles within three tests, versus 49% for a simple no-learning rule that just picks the cleanest-splitting test, essentially a tie. Even limited to tho
2026-06-24 Qwen3.5-4B Adaptive Evidence Budget Policy
Yes. Trained just for this single decision, the model matched the accuracy of running all ten checks — solving about 92 of every 100 tasks — while using only about five checks on average, near a perfect-hindsight stopper
2026-06-23 Qwen On-Policy Repair-to-Compiler
Yes, but the self-correction isn't the secret ingredient. On brand-new requests the model climbed from about 29% correct programs to 99% after training on its own corrected drafts. The catch: training on plain correct pr
2026-06-23 Qwen Iterative Repair Policy
Yes. The frozen model alone got about 30% of programs exactly right; editing one step at a time lifted that to 53% on fresh problems, closing roughly 38% of the distance to the best a perfect fixer could reach (89%). The
2026-06-23 Qwen Hidden VM On-Policy Canonical Repair
No. Training the model on automatically corrected recipes reached 61% on new tasks, versus 59% for plain training — a 2-point gap that is basically noise, and it left longer tasks no better. The corrections are genuinely