Research roadmap

Rendered from knowledge/research_roadmap.md

This roadmap is organized around programs, not imported tracks. It should help future agents decide what to do next and when to open new lines.

Portfolio Priority 1: Build Selection Science

Program: Evidence-Conditioned Selection

Why: candidate pools often contain hidden-correct outputs, but weak visible evidence causes wrong commits.

Next work:

• visible-only selector benchmarks with abstention,
• generated counterexample evidence without hidden expected answers,
• independent implementation consensus,
• deployable-gap scorecards for every oracle-ceiling result.

Portfolio Priority 2: Compare Structured Representations

Program: Structured Execution And Compilers

Why: structured execution is promising, but the causal contribution of typed slots, latent state, bytecode, curricula, and state supervision is still underexplored.

Next work:

• one shared suite comparing direct text programs, typed bytecode, latent slots, and differentiable executors,
• multi-seed replication of strongest compiler results,
• harder length, paraphrase, and family shifts.

Portfolio Priority 3: Make Memory Useful

Programs:

• Algorithmic Memory And Retrieval
• Operator And Skill Inventories

Why: memory and inventories can add coverage, but naive prompt memory can hurt.

Next work:

• memory as constraints/tests/candidates instead of only context,
• operator-card and skill-card schemas,
• shortlisters for larger banks,
• active disambiguation for type-colliding primitives.

Portfolio Priority 4: Couple Evidence Acquisition To Decisions

Programs:

• Active Evidence Acquisition
• Process Control And Tool Use

Why: evidence gathering should optimize downstream decisions under budget.

Next work:

• STOP/MORE and commit/repair controllers,
• expected-output-free probes,
• family-aware acquisition policies,
• tool-state transfer across substrates.

Portfolio Priority 5: Make The Repository Better At Research

Program: Collective Experimentation Infrastructure

Why: the project should produce many future lines without losing prior lessons.

Next work:

• claim/evidence ledgers,
• program-level scorecards,
• stronger metadata schemas,
• issue/backlog templates for hypotheses,
• validation gates for hidden-label boundaries and artifact manifests.