This reference defines Kiln's implementation-neutral artifact, verification, live replay, protocol-error, capacity, and conformance boundaries for OpenEnv training. The runtime contract is OpenEnv discovery plus a stateful WebSocket session. It does not depend on any particular environment implementation.
For the normal workflow, start with the OpenEnv training guide.
The machine-readable record types live in
contracts/kiln-openenv-v1.schema.json.
Artifact bundle
Every successful collection publishes three files:
openenv.rollouts.jsonlis the canonicalAgenticGroupcorpus consumed by native GRPO. Each scored rollout carries fail-closedkiln.openenv-rollout.v1provenance.openenv.replay.jsonis akiln.openenv-replay.v1manifest. It records each group's exact reset payload and reset observation, then every candidate's ordered action/result exchanges and final state. It links to the canonical JSONL digest instead of creating a second training representation.openenv.rollout-summary.jsonis the collection receipt. Summary v5 binds the other files plus the ordered, seed-free per-environment reset plan by SHA-256 and records schemas, controls, statistics, any immutable admitted training contract, and any submission. Offline verification still accepts legacy v2 and v3 receipts.
Move, retain, or archive the three files together. If the receipt's paths no
longer resolve, provide --dataset and --replay overrides to verification or
live replay.
The summary records every discovered identity and typed schema; behavior
adapter; seed, sampling, concurrency, recovery, capacity, and reset-plan
controls; group, rollout, step, model-token, and latency counts; returns,
outcomes, artifact identities, the exact kiln.openenv-training-contract.v1,
and the response from openenv train.
Server-owned runs publish the same three representations below
<adapter_dir>/.openenv/runs/<run-id>/ and expose them as dataset, replay,
and summary artifact links in GET /v1/openenv/runs/<run-id>. The adjacent
run.json is the durable orchestration record, not a fourth training
representation. It records lifecycle progress, failure or cancellation, the
immutable admitted training contract, and the linked native training job. An interrupted active run is marked failed
after restart because a stateful environment session cannot be assumed
resumable.
The CLI can own this complete persisted lifecycle without reconstructing HTTP requests or artifact paths:
kiln openenv start --request openenv-run.json \
--idempotency-key experiment:counter:17 --follow
kiln openenv artifact <run-id> summary --output openenv.rollout-summary.json
kiln openenv artifact <run-id> environment_eval_receipt \
--output evidence/environment-evaluation/receipt.json
start reads exactly one regular, non-symlink JSON object up to 1 MiB and lets
the server validate the authoritative OpenEnvRunRequest. artifact first
reads current run status and accepts only an exact kind and relative URL from
its manifest. Redirects are disabled, so the Kiln origin cannot be exchanged
for another host. The client requires manifest-matching Content-Length and
strong ETag, requests identity encoding, requires private, no-store and
nosniff, then independently caps, counts, and SHA-256 hashes the streamed
body. A same-length mutation still
fails. Bytes are staged beside the destination, synced, and atomically
published only after all checks pass. Existing files are preserved unless
--force explicitly requests replacement; failures leave no partial output.
--json emits a kiln.openenv-artifact-download.v1 local receipt without
embedding the configured Kiln base URL or its possible credentials.
Retry-safe workflow creation
OpenEnvRunRequest.idempotency_key is an optional 1..=128-byte opaque token
using ASCII letters, digits, ., _, :, or -. A new request returns HTTP
202. While that run remains retained, an exact retry returns its current or
terminal status with HTTP 200 and the original run ID; it creates no second
queue entry, environment session, model request, trainer, or evaluation. The
binding and normalized request survive restart. Concurrent identical POSTs are
atomic, so exactly one creates work.
Normalization occurs through the typed request: input aliases and omitted
defaults therefore compare by effective meaning rather than raw JSON bytes.
Reusing a key with any changed field returns HTTP 409
openenv_run_idempotency_conflict. Omitting the key deliberately creates a new
run. Bind one key to one experiment attempt, and never place a credential or
private payload in it because the key persists in run.json and public status.
kiln openenv start --idempotency-key <key> inserts the key into its bounded
request file and refuses to disagree with an existing body field. The dashboard
generates a UUID and reuses it while an identical submission remains unresolved,
so a lost response can recover the accepted workflow. Idempotency retention is
bounded by the same terminal TTL and max_tracked_runs history policy; once a
record is no longer retained, that key can create new work. The fixed-cardinality
kiln_openenv_runs_total{status="idempotent_replay"} counter measures recovered
submissions without exposing keys as labels.
A pathname is not publication. The download route accepts only artifact kinds
present in that run's persisted artifacts manifest, so staged or partially
written bundles remain unreachable. Publication hashes each regular,
non-symlink file with bounded streaming reads and cross-checks dataset and
replay bytes against the summary. Every download repeats the exact byte-count
and SHA-256 check on one opened descriptor before streaming that same
descriptor. Valid responses carry the manifest byte count as Content-Length,
the quoted digest as a strong ETag, private, no-store, and nosniff.
Missing, replaced, oversized, symlinked, truncated, or modified files fail with
409 openenv_artifact_integrity_failed; Kiln never serves the drifted bytes.
Restore the original bundle or recollect instead of editing retained artifacts.
After the last episode and before deriving artifacts, collection re-reads every
stable discovery surface for every endpoint and requires exact equality with
its initial inspection. Persisted status calls this phase revalidating.
The canonical complete-discovery digest binds raw metadata, advertised names,
OpenAPI, and schema documents—including unknown extension fields—independent of
whitespace and object-key order. Authentication method, URLs, typed metadata,
schema digest, and action/observation/state schemas are also compared. A mismatch
publishes no bundle. The summary therefore proves equal discovery identity at
the collection boundaries; it cannot prove that an implementation behind the
same URL remained unchanged between those observations or after publication.
Pin an environment image or binary and retain its deployment identity for
serious experiments.
Operational rollout telemetry
Once the three-file bundle is published, its OpenEnvRolloutStats projection
is stored in run.json and returned as rollout_stats. It includes return
mean/range, all four terminal outcomes, recoverable errors, capacity retries,
environment steps, model tokens, and mean model latency. Paired held-out status
uses the same type as baseline_stats and candidate_stats. CLI follow and the
dashboard therefore show the same statistics that offline verification binds.
Prometheus accumulates published collections as
kiln_openenv_episode_terminations_total{termination="done"|"max_steps"|"invalid_model_action"|"protocol_error"}
plus recoverable-error, capacity-retry, environment-step, model-token, and
model-latency totals. These are closed global dimensions: environment URLs,
names, adapters, credentials, and run IDs never become labels. Per-run return
values remain in bounded status and receipts rather than creating unsafe metric
cardinality.
Authentication boundary
Environment identity records whether collection used none or bearer
authentication. It never records the bearer value, API/dashboard credential
handle, or environment-variable name. Server-owned workflows resolve handles
from exact-origin configuration:
[openenv]
allow_remote_environments = true
[openenv.credentials.production-arcade]
origin = "https://arcade.example.com"
bearer_token_env = "ARCADE_OPENENV_TOKEN"
Set ARCADE_OPENENV_TOKEN before starting Kiln. API and dashboard requests
align an opaque handle with each environment; use null for a public slot:
{
"environment_urls": ["https://arcade.example.com/openenv"],
"credential_ids": ["production-arcade"]
}
Direct commands resolve an environment-variable name immediately before client construction:
ARCADE_OPENENV_TOKEN=... kiln openenv inspect \
--environment https://arcade.example.com/openenv \
--credential-env ARCADE_OPENENV_TOKEN
kiln openenv train \
--environment https://arcade.example.com/openenv \
--credential-env ARCADE_OPENENV_TOKEN \
--output-adapter arcade-agent
Repeat --credential-env in environment order and use - for a public slot.
The sensitive header is applied uniformly to /health, discovery, Task API, and the
WebSocket upgrade. Remote credentials require HTTPS/WSS; loopback HTTP remains
available for local development. The
kiln_openenv_authenticated_operations_total{operation="inspect"|"task_catalog"|"run"} metric
counts protected operations without credential, origin, or token labels.
Authenticated response bodies and episode frames are checked for direct token
reflection before parsing so a peer cannot place the configured credential in
training data or artifacts.
Live replay of an authenticated bundle must provide a credential source in the same environment position. Replay checks that the resulting authentication method matches captured identity, but intentionally permits token rotation. Offline verification needs no credential because secrets are outside the content-addressed artifact boundary.
kiln openenv replay --summary protected.rollout-summary.json \
--credential-env ARCADE_OPENENV_TOKEN
Task API catalogs are discovery, not reset
Dataset-backed OpenEnv servers may expose GET /{environment}/splits plus
POST /{environment}/{tasks|num_tasks|task|task_range}. HTTP 501 on a
provider-backed route is a conforming declaration that the environment has no
TaskProvider; it does not disable seeded rollout or training.
# List split names and their provider-authored types.
kiln openenv tasks --environment http://127.0.0.1:8990
# Read a bounded page without fetching the provider's complete task list.
kiln openenv tasks --environment http://127.0.0.1:8990 \
--split train --start 100 --limit 25 --json
The dashboard exposes the same catalog beside protocol inspection. Server
clients use POST /v1/openenv/tasks with environment_urls, optional aligned
credential_ids, optional environment_name and split, and start/limit.
The response schema is kiln.openenv-task-catalog.v1.
Task rows are arbitrary, untrusted JSON. Kiln preserves open split types and
provider fields, percent-encodes advertised environment names as one URL
segment, limits discovery bodies to 2 MiB, limits raw client collections to
16,384 items, and limits CLI/API/dashboard pages to 200 rows. The server metric
kiln_openenv_task_catalog_inspections_total{status="started"|"completed"|"failed"}
has fixed-cardinality labels.
OpenEnv defines no operation that selects a Task API row for a WebSocket
session, and a provider's reset need not consult its task catalog. Kiln
therefore never copies a row into reset data, run records, replay manifests, or
training receipts. Portable scheduling remains the explicit reset object plus
Kiln's deterministic group seed. If an environment defines its own row/reset
convention, bind it explicitly through reset options; that resulting reset
payload is then hashed and replayed under the normal artifact contract.
Paired held-out evaluation bundle
A server train run with environment_eval adds:
environment-evaluation/
baseline/{rollouts.jsonl,replay.json,summary.json}
candidate/{rollouts.jsonl,replay.json,summary.json}
receipt.json
Both sides use the production collector with identical URL rotation, reset payloads, held-out seeds, candidate indices, generation seeds, and bounds. Before comparison, Kiln requires identical discovered environment identities, reset payloads and observations, group/seed identities, and candidate indices. Drift fails the run instead of producing an unpaired estimate.
receipt.json is kiln.openenv-environment-evaluation.v1. It binds exact
baseline and candidate adapter content revisions, execution provenance, both
summary SHA-256 values, mean returns, improved/regressed/tied counts, the
two-sided exact sign-test result, fixed policy version, decision, and promotion
outcome. The run status publishes every file with its own content digest.
Run kiln openenv verify against each summary to validate its underlying
dataset/replay bundle independently.
Evaluation seeds cannot overlap the training interval. A promotion gate needs
at least 20 paired seed groups and significant per-seed mean-return improvement
at p < 0.05, plus configured point thresholds. Replications within one seed
do not inflate significance. Rejection or inconclusive evidence leaves the
candidate unserved; auto_load=false records a passed candidate as kept.
Offline verification
kiln openenv verify --summary openenv.rollout-summary.json
Verification is network-free and bounded. It:
- checks the byte count and SHA-256 digest of the JSONL and replay manifest;
- parses every canonical GRPO group and fail-closed OpenEnv provenance record;
- validates the replay manifest and its link to the canonical dataset;
- recomputes each episode return from tagged rewards and configured protocol error penalties; and
- cross-checks group, candidate, step, environment, seed, termination, and artifact totals against the summary.
An unsupported schema, malformed hash, non-finite return, inconsistent done,
invalid protocol-error state, mismatched transcript, or inconsistent receipt
causes verification to fail. A successful check emits a
kiln.openenv-verification.v1 report; it does not contact the environment or
claim that its current behavior is unchanged.
Exact live replay
kiln openenv replay --summary openenv.rollout-summary.json
Live replay first performs the complete offline verification. It then:
- discovers every live target and requires the complete captured discovery digest, identity, and action/observation/state schemas;
- opens fresh capacity-aware sessions;
- sends the exact effective reset objects and captured actions; and
- compares reset observations, observations, tagged rewards,
done, protocol errors, and captured final state exactly.
The result is a kiln.openenv-replay-run.v1 report. Any mismatch is observable
environment drift. Equal schema identity alone is insufficient because task
data, randomness, or implementation behavior may have changed without a schema
change.
If collection ended because the model produced malformed JSON, only the environment prefix can be replayed: that malformed policy generation was never sent on the OpenEnv wire. The replay report counts such prefix-only candidates explicitly.
Reset options can include arbitrary task payloads. Kiln hashes them into each rollout and retains the exact effective object once per replay group because exact replay requires it. Treat replay files as sensitive when reset tasks contain private data.
Protocol-error outcomes
The OpenEnv recoverable error codes are:
INVALID_JSONUNKNOWN_TYPEVALIDATION_ERROREXECUTION_ERROR
Kiln turns a recoverable error into a complete observation feedback turn on the
same WebSocket. The policy can correct its next action without losing episode
state. Each error contributes --protocol-error-reward (default -1) to the
episode return. A candidate may use at most --max-recoverable-errors
(default 3); the next recoverable error terminates it as protocol_error.
The terminal error codes are:
CAPACITY_REACHEDFACTORY_ERRORSESSION_ERROR
FACTORY_ERROR and SESSION_ERROR end the candidate. Invalid model JSON and a
JSON object that fails the advertised action schema are recorded separately
from environment errors as invalid_model_action, while exhaustion of the
horizon is max_steps; neither is mislabeled as environment done. Kiln
compiles the schema during inspection without HTTP/filesystem reference
resolution and validates before step, so ACTION_SCHEMA_VALIDATION_FAILED
never sends the rejected action to the environment. Its retained diagnostic is
bounded to closed keywords and JSON Pointer paths rather than model values or
environment-authored schema content.
OpenEnv has no request IDs or episode resume. Kiln therefore pumps only
Ping/Pong control frames plus periodic read-only state exchanges while policy inference is pending and rejects any
unsolicited application message. A timeout, socket failure, binary, malformed
or oversized response, credential reflection, or wrong response type
permanently poisons the session. The action is not resent and a late response
can never be consumed as the answer to a later action.
Capacity acquisition
Kiln's persisted control plane first applies its own bounded FIFO admission.
At most openenv.max_active_runs complete workflows execute; additional valid
runs remain cancellable and position-visible until a slot opens, with total
active, queued, and retained records exported as fixed-cardinality metrics.
The queue consumes no OpenEnv session or model capacity. max_tracked_runs is
the hard combined bound for active, queued, and retained terminal records.
Operators can alert on kiln_openenv_runs_active,
kiln_openenv_runs_queued, and kiln_openenv_runs_tracked; cumulative
admission and restart counts use
kiln_openenv_runs_total{status="admitted"|"resumed"}, and
kiln_openenv_run_queue_wait_seconds_total records aggregate queue delay.
The v5 admission sequence is stable; on restart, entries that never acquired a
slot resume in that exact FIFO order. A pristine queued v4 entry is first
materialized once into a v5 contract; it is never recomputed after that
migration. Any admitted non-terminal workflow fails
explicitly because an external
episode, trainer, or evaluator cannot be assumed resumable.
OpenEnv servers may cap active sessions and can send CAPACITY_REACHED as the
first WebSocket application frame. That result is terminal for the attempted
socket, not necessarily for the collection. Kiln closes it, waits with bounded
backoff, and opens a fresh session until capacity becomes available or
--capacity-wait-seconds expires.
This permits a rollout group to request more concurrent candidates than a small server admits without silently dropping candidates. Kiln continues to bound environment count, group count, candidate count, steps, action tokens, active sessions, discovery response bodies, WebSocket messages, data and replay artifacts, the summary, and the inline training corpus.
Those independent limits are not deferred until serialization. One 512 MiB aggregate retained-representation budget is charged when reset data and each action, observation, error, and final state arrive. Completed candidates are moved—not cloned—into their group, replay, and receipt projections, and the budget is reconciled after each compaction. Reset-option files are rejected from metadata before a large read, JSONL hashing streams into SHA-256, and the replay encoder refuses the write that would cross its 256 MiB artifact cap. Budget exhaustion aborts collection before dataset, replay, or summary is published; reduce group size, concurrency, steps, or environment payload size.
Live replay uses the same capacity acquisition rule. Capacity retries are reported separately from environment transitions and never become fabricated training observations.
Persisted failure semantics
Every newly failed server-owned run retains a bounded
kiln.openenv-run-failure.v1 object in OpenEnvRunStatus.failure. It is the
automation contract; the adjacent error string remains only as a backwards-
compatible projection for older clients and historical records. The failure
contains a closed stage, closed code, retryable, message, actionable
hint, and occurrence time. When the environment actually supplied evidence,
it also retains the exact uppercase OpenEnv protocol_code or upstream
http_status. Credentials, URLs with secrets, run IDs, and unbounded peer
payloads never become metric labels.
The stable codes are:
| Code | Meaning |
|---|---|
run_admission_failed |
Kiln could not acquire or persist workflow admission. |
run_interrupted |
A restart interrupted a non-resumable environment, trainer, or evaluator owner. |
persisted_contract_invalid |
A retained request or immutable training contract cannot execute safely. |
environment_unavailable |
Discovery or the stateful transport could not reach a usable peer. |
environment_capacity_exhausted |
Fresh-session acquisition exhausted the bounded capacity wait. |
environment_protocol_error |
The peer violated or terminally rejected the advertised protocol. |
environment_identity_changed |
Final discovery revalidation found a mid-collection deployment change; no artifacts were published. |
collection_failed |
Episode collection or policy execution failed before a complete corpus existed. |
artifact_publication_failed |
A complete collection could not publish its fail-closed artifact bundle. |
training_submission_failed |
Native GRPO rejected or lost the training handoff. |
training_failed |
The authoritative native trainer failed. |
training_evidence_invalid |
Trainer receipt, adapter manifest, corpus digest, or OpenEnv lineage failed validation. |
post_evaluation_failed |
A requested static evaluation failed or never published. |
environment_evaluation_failed |
Paired held-out collection, evidence, or promotion failed. |
internal_error |
No narrower safe category applies. |
retryable=true means an operator may submit a new run after correcting the
hinted condition. It never promises in-place resume and never changes the
idempotency rule: retrying the same retained key returns the original terminal
run, so a deliberate new attempt needs a new key. The CLI and dashboard render
the typed diagnosis directly. Prometheus exports
kiln_openenv_run_failures_total{stage="...",retryable="true|false"} over a
fixed stage × retryability matrix; endpoint, message, code payload, and run ID
cardinality cannot leak into the metric.
Identity drift uses stage identity_verification. It is retryable only as a
new attempt after the environment deployment is stable or pinned; the completed
episode sockets and mixed candidate set are never resumed or salvaged.
Training admission
For every training request, Kiln materializes the exact native-GRPO
configuration before persistence or direct environment contact.
It overrides rollout-owned behavior_policy, base_adapter, output_name,
and auto_load, then validates the environment-token loss and policy contract,
LoRA scale, checkpoint interval, behavior-adapter layout, installed
post_eval suite, serving profile, backend GRPO workload, optimizer, and model
rank ceiling. It also seals the exact base-model, inference-runtime, and
optional adapter content revision that must generate the corpus and initialize
the trainer. Persisted kind=train does this before creating its run
directory and atomically stores the result as
kiln.openenv-training-contract.v1 in v5 status. Collection, restart, trainer
submission, static evaluation, CLI, and dashboard all consume that exact
contract rather than reapplying current defaults. Direct kiln openenv train first calls
POST /v1/openenv/training/preflight, validates the v1 receipt, and later
submits its exact effective_config, behavior-policy identity, and returned optional post_eval;
rejection contacts no environment and writes no artifacts. Both paths increment
kiln_openenv_training_preflights_total{status="accepted"|"rejected"};
persisted rejection also increments
kiln_openenv_runs_total{status="training_preflight_rejected"}. Rollout-only
requests reject output_adapter, training_config, post_eval, and
environment_eval instead of ignoring them.
Summary v5 embeds the same contract and exact policy identity before the dataset/replay/summary bundle is first published. It therefore remains auditable if final native queue or memory admission rejects after collection. A summary containing a training submission without its contract fails schema and offline verification; legacy v2/v3 receipts may not claim the new field.
Direct preflight returns the exact current queue/tracked-job snapshot with
capacity_reserved: false. The final native GRPO admission repeats immutable
checks after collection and adds time-varying queue and live-memory capacity.
This second gate remains authoritative because capacity can change while
episodes run.
Current collection requests exact rollout provenance for each separately sampled action. It rejects a policy revision change within an episode or across the corpus. Native no-importance-correction admission then requires that same identity to match the selected base policy. If behavior used a LoRA, execution revalidates and copies those exact bytes into the private trainer staging root while holding the adapter mutation barrier; later same-name disk rewrites cannot change the optimizer's starting weights.
Retained trainer evidence
A completed persisted train run is self-contained. Its
training.training_data.openenv field carries the admitted
kiln.openenv-training-data.v1 lineage: exact corpus and ordered task-plan
digests, endpoints and schema identities, seed range, steps, rewards, and
termination counts. Clients do not need to join the run to /v1/train/jobs to
identify what trained the adapter.
Completion also adds train_receipt and adapter_manifest to the run's
ordinary manifest-gated artifacts array. Kiln validates the native receipt,
requires successful status and the requested adapter name, binds both files to
the admission corpus and semantic OpenEnv lineage, verifies the manifest's
receipt hash, then copies the exact bytes atomically into the run directory.
Each source file has a 4 MiB limit and must be a regular non-symlink file. The
run-owned evidence therefore survives adapter lifecycle operations and every
download keeps the same length, ETag, SHA-256, and same-origin checks:
kiln openenv artifact <run-id> train_receipt --output evidence/train_receipt.json
kiln openenv artifact <run-id> adapter_manifest --output evidence/adapter_manifest.json
The dashboard's Prove it after training control emits ordinary
post_eval. The named suite must already be installed; Kiln follows adapter
and optional baseline jobs to terminal outcomes. train-set-eval is diagnostic
and cannot set min_accuracy. Static evaluation can accompany paired
environment_eval, but a workflow has one automatic promotion owner:
post_eval.min_accuracy and environment_eval.gate are mutually exclusive.
Protocol conformance oracle
Run the byte-real interoperability gate with:
CARGO_BIN="$(command -v cargo)" scripts/check_miniopenenv_interop.sh
The script rebuilds whichever oracle checkout OPENENV_INTEROP_ROOT names and
derives its complete games, synthesis-task, and exact-text inventories from
that checkout's published build graph. It does not duplicate environment names
inside Kiln. Pull-request and push CI select a fixed revision for reproducible
gating; a weekly and manually dispatched edge lane selects the oracle's current
main, so upstream protocol or environment drift becomes visible without a
Kiln commit. The current pin publishes a C99 counter plus twenty-two
text-profiled servers—fourteen arcade/synthesis environments and eight
text-first math families—then tests:
- health-gated connection, canonical complete-discovery identity, typed schema identity, and close;
- the optional
input_textprofile across every environment in the matrix, without making that downstream convention a protocol requirement; - thinking-on action collection where separated reasoning is retained in the trainable trajectory, the prompt-owned opener is excluded from the action mask, and only the final schema-valid answer reaches the environment;
- schema-discovered answer strings, deterministic seeded prompts, recoverable
wrong-type actions, exact integer rewards, and frozen post-
doneobservations across all eight one-step math environments; - integer and floating-point rewards;
- object, integer, and string action shapes;
- dynamic legal actions and procedural seeded state;
- recoverable
EXECUTION_ERRORfollowed by correction on the same socket; and - unsolicited terminal
CAPACITY_REACHEDfollowed by fresh-session reacquisition; and - paired held-out baseline/candidate collection whose environment-owned bandit returns drive the exact sign-test gate.
The end-to-end lane collects two candidates against a bandit limited to one session, forces a semantic action error and corrective turn, submits the canonical groups to a fake Kiln training API, verifies all three artifacts, and replays every environment exchange.
Miniopenenv is deliberately confined to this test-oracle boundary. The gate rejects miniopenenv-named production environment variables, and production Rust sources contain no miniopenenv-specific runtime branch. Dynamic test inventory is not runtime discovery policy: the same client and training path accepts every implementation that satisfies the OpenEnv protocol.
Troubleshooting
Collection times out waiting for capacity. Another client or a long
candidate held all sessions beyond --capacity-wait-seconds. Reduce
concurrency, increase that bounded wait, or raise the environment's session
limit.
A recoverable error ends the candidate. The candidate spent
--max-recoverable-errors, or the code was terminal. Inspect the trajectory and
receipt counters before changing the budget.
Replay reports drift. Confirm that the same environment build, task data, and reset semantics are deployed at the captured URL. Stable schema identity does not imply stable behavior.
A run fails with environment_identity_changed. The environment's stable
HTTP discovery identity—including an unprojected metadata, schema, inventory,
or OpenAPI extension—changed between initial inspection and the post-episode
revalidating phase. Pin or stabilize the deployment, inspect it again, and
submit a new run with a new idempotency key. Kiln deliberately publishes none
of the completed episodes from that attempt.
A protected environment returns 401 or rejects the WebSocket upgrade.
Confirm that the configured credential origin exactly matches the URL's
scheme/host/port, the named environment variable is present and non-empty in
the Kiln process, and replay received one aligned --credential-env. Remote
bearer credentials require HTTPS/WSS; redirects are never followed.
Candidates in one group get different initial prompts. The environment is not deterministic for the supplied reset seed. Kiln rejects the group instead of computing misleading relative advantages.
Most outcomes are invalid_model_action. Inspect the action schema and the
retained harness code. INVALID_MODEL_ACTION means the reply was absent or was
not one JSON object; ACTION_SCHEMA_VALIDATION_FAILED means its shape or values
missed the advertised schema. Increase --max-action-tokens when a reasoning
trajectory is truncated, and consider an SFT bootstrap for the JSON final-action
format. Disable thinking only when you intentionally want to train
final-action-only trajectories.
Rewards have no variance. GRPO has no within-group signal. Use harder tasks, more policy sampling, a more informative reward, or an SFT/OPD bootstrap.
Training is rejected. Persisted and direct workflows preflight immutable
failures before collection; correct the returned config, adapter, suite,
backend, or optimizer error. A direct CLI collection can still encounter a
time-varying queue or memory rejection at final native admission because the
preflight snapshot is not a reservation; its rollout artifacts remain valid
and can be resubmitted with kiln train grpo.
A remote environment redirects. Redirects are rejected at the trust boundary. Pass the canonical base URL directly.