Qualification case-result contract

Closed command-to-runner evidence for one qualification case execution. JSON Schema validates the wire shape; scripts/qualification/run.py also enforces finite numbers, canonical ordering, declared metric ownership, the selected variant's effective configuration, file safety, and size limits. The runner normalizes this record before aggregation, and the resulting qualification receipt—not this file—is the run verdict.

Schema identity

Property	Value
Title	Kiln qualification command case result
`$id`	`https://kiln.local/schemas/qualification-case-result-v1.json`
Dialect	`https://json-schema.org/draft/2020-12/schema`
Root type	`object`
Root object	closed

Root fields

Field	Required	Type	Constraints and default	Description
`schema_version`	yes	`any`	const 1	Command case-result contract version. The runner accepts only version 1.
`case_id`	yes	`string`	pattern ^[a-z0-9][a-z0-9._-]{2,127}$	Case identifier copied exactly from the selected workload case. A syntactically valid but different identifier is rejected.
`status`	yes	`any`	enum "passed", "failed", "skipped"	Command-reported case outcome. The runner can downgrade this status when execution, output, evidence, source, model, or infrastructure checks fail; it never treats this field alone as the qualification verdict.
`duration_seconds`	yes	`number`	minimum 0	Non-negative finite duration reported by the command. The normalized evidence and receipt use the runner's monotonic wall-clock duration instead.
`effective_config`	yes	`configObject`	-	Configuration the command says it actually used. It must exactly equal the selected workload variant's effective_config; a mismatch fails the case and prevents the receipt from claiming that configuration.
`metrics`	yes	`array<metric>`	-	Measurements emitted by the command, sorted by unique name. Their names must exactly match result_protocol.declared_metrics for this case.
`tolerances`	yes	`array<tolerance>`	-	Optional, command-reported tolerances for metrics in this same file, sorted by unique metric name. The runner preserves them only in per-execution case-result artifacts; workload comparison_policy remains the authority for receipt comparison.
`details`	yes	`string \| null`	-	Optional diagnostic text from the command. The runner may append execution or contract failures and caps the normalized text at 2,048 characters.

Definitions

configValueOne finite JSON scalar, null, or recursively nested configuration object. Arrays are not valid configuration values.0 fields

Type: string | number | boolean | null | configObject.

Composition and conditional rules

The following JSON is copied exactly from this schema node.

{
  "oneOf": [
    {
      "type": [
        "string",
        "number",
        "boolean",
        "null"
      ]
    },
    {
      "$ref": "#/$defs/configObject"
    }
  ]
}

configObjectClosed-shape configuration namespace whose lowercase keys can be addressed as dot-separated comparison paths. Values may be finite scalars, null, or nested objects; list values are excluded.0 fields

Type: object.

Composition and conditional rules

The following JSON is copied exactly from this schema node.

{
  "propertyNames": {
    "pattern": "^[a-z][a-z0-9_-]*$"
  },
  "additionalProperties": {
    "$ref": "#/$defs/configValue"
  }
}

metricOne finite command-owned measurement with the definition needed to interpret and aggregate it. The runner checks this definition against any matching committed comparison rule.5 fields

Type: object. Constraints: closed object.

Field	Required	Type	Constraints and default	Description
`name`	yes	`string`	pattern ^[a-z][a-z0-9_.-]{0,127}$	Stable metric identifier declared by the workload case. Metric entries must be unique and sorted by this field.
`value`	yes	`number`	-	Finite observed value. JSON Schema accepts a number; the runner explicitly rejects NaN and infinity.
`unit`	yes	`string`	minimum length 1	Non-empty unit label, such as s, bytes, tokens_per_second, or bool. This text is part of the metric definition and must match committed policy when the metric is compared.
`aggregation`	yes	`string`	minimum length 1	Non-empty description of how the command produced this value. Across repeated executions, the runner requires the same definition and emits a mean_of__ result.
`lower_is_better`	yes	`boolean`	-	Metric direction supplied by the command. This does not choose the comparison operator; any committed workload metric rule must declare the same direction.

toleranceAdvisory tolerance attached to one reported metric. It is retained in normalized case-result artifacts but is not copied into qualification receipt results and does not override workload comparison policy.3 fields

Type: object. Constraints: closed object.

Field	Required	Type	Constraints and default	Description
`metric`	yes	`string`	pattern ^[a-z][a-z0-9_.-]{0,127}$	Name of a metric present in this result. Tolerance entries must be unique and sorted by this field.
`absolute_tolerance`	yes	`number`	minimum 0	Non-negative finite absolute allowance reported by the command, expressed in the named metric's unit.
`relative_tolerance`	yes	`number`	minimum 0	Non-negative finite proportional allowance reported by the command, where 0.05 represents five percent.