The gaps — where the protocol turns out not to exist
GENERATED by tools/gendocs.sh. Do not edit this file; edit the markers in
the source and run make docs.
A gap is a question the protocol does not answer. Not an oversight we are reporting — a place where two implementations can both be right, behave differently, and have no way to discover that about each other. Every one is annotated at the line that had to choose, in a fixed three-line form:
| Line | Says |
|---|---|
unspecified: | the question, and the evidence that nothing answers it |
we chose: | what this implementation does |
a different implementer might: | a defensible other answer, and what it would cost |
The third line is the one that makes a gap different from a design note. If nothing plausible could go in it, the question was not open.
This file is separate from WARTS.md deliberately. A wart is a
fact about the reference that a compatible server must reproduce. A gap is a
fact about the specification: there is nothing to reproduce, and two conforming
servers may differ. Merging them would teach a reader that those are the same
kind of thing, and they are the two most different things in this repository.
A HOLE IN THE NUMBERING IS AN ANSWER, NOT A QUESTION. Every number from 1 to the highest in use has a section below, and an empty one says whether it was retired and why.
GAP-
GAP-01
Declared at src/server/session.c:50 ↗:
GAP-01: whether `reset` must precede `step`.
unspecified: no document says. The reference has no phase to check and no
place to check it, so the question was never asked upstream.
we chose: it must not. A `step` on a fresh connection runs against the
zeroed state block and answers with an observation, which is
what corpus case 59 records the reference doing and what the
side-by-side probe in docs/evidence/OBSERVED-WIRE.md shows the
two servers doing byte for byte.
a different implementer might: refuse a `step` before a `reset` with
VALIDATION_ERROR, and would then reject traffic a conforming
server accepts — including traffic in the captured corpus.
Also referred to from: src/server/mcp.c:221
GAP-02
Declared at src/server/session.c:62 ↗:
GAP-02: whether `done: true` fences further steps.
unspecified: `done` is a field on an observation and there is nothing else
in the protocol that reads it. No frame ends an episode, and
there is no episode object for one to end.
we chose: it fences nothing. An environment that keeps mutating after it
said `done` keeps being stepped, which is the reference's
behaviour: two further steps past `done:true` reach `total` 4
and 5 with `done` still true, on the reference and on this
server identically. docs/evidence/OBSERVED-WIRE.md has both
transcripts.
a different implementer might: refuse a `step` after `done` until a `reset`
arrives. Nothing on the wire would distinguish that server from
this one until a client did it, at which point the two behave
differently with no way for the client to have known.
GAP-03
Declared at src/proto/encode.c:60 ↗:
GAP-03: the order of the four keys.
unspecified: nothing in the reference states that object key order is
contractual. It falls out of pydantic's declaration order, and
a JSON object is an unordered collection in RFC 8259 section 4.
we chose: insertion order, everywhere, in both the tree and the writer —
which reproduces the captured bytes exactly and is the only
choice under which a byte-for-byte replay is meaningful at all.
a different implementer might: sort keys, or emit them in table order, and
no client would break — but every fixture in
tests/fixtures/reference-corpus.json would stop matching, so
the replay could no longer be a byte diff.
Also referred to from: src/proto/jsonrpc.c:498 src/wire/json.h:130 src/wire/json.h:322
GAP-04
Declared at src/proto/encode.c:95 ↗:
GAP-04: whether an environment can tell "no metadata" from "empty metadata".
unspecified: the reference declares `metadata: Optional[dict] = {}` and then
tests it for truth, which collapses None and {} into one case
before either reaches an encoder. Nothing states whether that
collapse is protocol or an artefact of one `if`.
we chose: the same collapse. oe_json_len answers 0 for a NULL pointer and
for an empty object, so oe_obs.metadata has no third state and
an author cannot accidentally create one.
a different implementer might: keep them distinct and emit `"metadata":{}`
on HTTP for an explicitly empty object while reserving
`"metadata":null` for an absent one. No captured exchange
distinguishes the two, so nothing would catch it.
GAP-05
Declared at include/openenv/openenv.h:339 ↗:
GAP-05: how binary data crosses an OpenEnv wire.
unspecified: there are six JSON Schema `type` keywords here and none of them
is bytes, because JSON has no binary scalar. Nothing upstream
says what an environment whose observation is an image or an
audio buffer should publish, and MUST 8.B.4 forbids the obvious
alternative outright: a client sends TEXT frames only, never
binary, so the transport cannot carry it either.
we chose: nothing. There is no bytes kind, and an author who needs one
base64s into an OE_F_STR field and says so in its
`description`. That keeps the schema honest — the wire really
does carry a string — at the cost of a third more bytes.
a different implementer might: publish `{"type":"string","format":"binary"}`
or JSON Schema 2020-12's `contentEncoding: "base64"`, which is
the keyword that exists for exactly this. Either is legal and
neither is agreed, so two servers that both "support images"
would need a client written for one of them.
GAP-06
Declared at src/proto/decode.c:804 ↗:
GAP-06: timeout_s is validated and then has no effect.
unspecified: nothing says what a server does when a step exceeds it,
because no server has ever measured one — the field is read
into the model, filtered into step()'s kwargs if the
signature happens to declare it, and never consulted.
we chose: reproduce both halves. `timeout_s:0` is a 422 (corpus 24)
and a valid one changes nothing.
a different implementer might: enforce it, and be the only OpenEnv server
that does — which would make a client's timeout behaviour
depend on which server it reached.
Also referred to from: src/proto/decode.h:221
GAP-07
Declared at src/proto/decode.c:822 ↗:
GAP-07: request_id is accepted and never echoed.
unspecified: the field exists on the request model and appears in no
response, no log line and no error item.
we chose: validate its length, forward nothing, echo nothing.
a different implementer might: return it in the response, which would be
a correlation id — a thing OpenEnv does not have, and one no
client would read.
GAP-08
Declared at src/server/conn.c:827 ↗:
GAP-08: how long may a server hold an upgraded connection that says nothing?
unspecified: nothing in OpenEnv says, and the reference's own reaper is
configured off by default — its `session_timeout` is None, so
the sweep it runs never destroys anything. What DOES protect
the reference is one layer down and easy to miss: `websockets`
pings every 20 s and drops a peer that has not ponged by 40 s.
we chose: an idle deadline of OE_WS_IDLE_MS, re-armed by every inbound
frame, and the shorter OE_WS_ASSEMBLY_MS while a message is
half-delivered. We send no Ping of our own, so this is the
whole mechanism rather than half of one, and a client that
means to keep its socket has to speak on it.
a different implementer might: send Pings and reap on a missing Pong, which
is what the reference's library does — a timer per connection,
bought for a liveness check that survives a peer which is
genuinely idle rather than gone.
WE CHOSE THAT AFTER GETTING IT WRONG, WHICH IS THE PART WORTH READING. The
first version disarmed the deadline here, on the reasoning that an idle
WebSocket is a client between calls. That is true of /ws — capped by
max_sessions, eight of them — and false of /mcp, which returns from
answer_upgrade before oe_session_open is ever called and therefore consumes a
CONNECTION slot while consuming no SESSION slot. The session cap, the only
other limit in the program, protected nothing at all.
Measured against envs/echo: 64 plain `GET /mcp` upgrades from one client,
about 150 bytes each, nothing sent afterwards — and GET /health answered
ConnectionReset for the life of the process. A cap that does not bound the
resource is not a cap, and a justification that covers one of two routes is
not a justification.
/mcp is deliberately still NOT counted against max_sessions. It holds no
session, and a server that reported a connection as a session in a
CAPACITY_REACHED frame would be lying about itself on the wire — which
serve.c's own banner gives as the reason this program does not fork. What
bounds /mcp is OE_MAX_CONNECTIONS plus the clock above, which is what bounds
every other unauthenticated socket here.
Also referred to from: src/server/conn.h:287
GAP-09
Declared at src/proto/errors.c:519 ↗:
GAP-09: the relative order of `ctx` and `url`.
unspecified: no captured item carries both — corpus 17 and 24 have `ctx` and
no `url`, corpus 20 and 22 have `url` and no `ctx` — so the
corpus cannot decide which comes first.
we chose: `ctx` then `url`, which is the order pydantic's own error
dictionaries are built in.
a different implementer might: emit `url` first, and every captured case
would still match byte for byte, which is exactly why this is
recorded rather than asserted.
GAP-10
Declared at src/proto/errors.c:214 ↗:
GAP-10: the message of a session-fatal error.
unspecified: the reference's text here is a Python AttributeError string
— corpus case 64 pins "'int' object has no attribute
'get'" for a payload of `5` — which is an implementation
detail of CPython's dict, not a protocol message.
we chose: the caller supplies it, so a byte-exact replay can pass the
reference's string and a running server can say something
true instead.
a different implementer might: emit a fixed string, and no client would
notice — they branch on `code` (MUST 8.B.11).
Also referred to from: src/proto/decode.c:188 src/proto/decode.c:385
GAP-11
Declared at src/proto/value.c:408 ↗:
GAP-11: what a NUL inside a JSON string means to a program written in C.
unspecified: a string holding one NUL byte is well-formed JSON (RFC 8259
section 7) and a perfectly ordinary Python `str`, so the
reference has no rule about it and no reason to want one. Every
C API that returns a `const char *` needs one.
we chose: length-carrying `oe_str` end to end, and this accessor REFUSES:
a value with an interior NUL yields the caller's fallback, so
the C string an author reads is never a shortened version of the
string the validator checked. `oe_as_slice` is the faithful form
and is exported beside it.
a different implementer might: truncate at the NUL, which is what a
`strdup`-shaped API does by construction; or reject the document
at parse time. The first is the one that turns a validated value
into a different value between two lines of the same function.
Also referred to from: src/proto/jsonrpc.c:755 src/proto/value.c:30 src/wire/json.h:335
GAP-12
Declared at src/wire/json_parse.c:943 ↗:
GAP-12: unpaired surrogates.
unspecified: nothing says what an implementation owes for "\ud800".
we chose: refuse the document, at the escape that caused it.
a different implementer might: keep the lone surrogate, as CPython does,
and fail later in the encoder — or substitute U+FFFD.
GAP-13
Declared at src/proto/schema.c:181 ↗:
GAP-13: the model title — the only place a TYPE name reaches the wire.
unspecified: the reference puts the Python class name here
("CounterAction"), and C has no class names. `oe_env` has one
name, the environment's, and nothing that names an action.
we chose: strip a trailing "Environment" from `oe_env.name` and append
"Action" or "Observation" — which is the naming convention every
environment in the reference repository follows, so
"CounterEnvironment" yields the captured "CounterAction" and
"CounterObservation" exactly.
a different implementer might: add two more `const char *` members to oe_env,
or emit the environment name unchanged. Nothing on the wire
depends on this string — no client parses it, the conformance
validator checks only that the three sub-objects are objects —
which is why a derivation is enough and why an author who needs
a specific name has `action_schema_json` instead.
Also referred to from: src/proto/schema.h:80
GAP-14
Declared at src/proto/schema.c:460 ↗:
GAP-14: what a container field's schema says about its MEMBERS.
unspecified: no captured environment declares an object- or array-typed
field, so nothing pins the shape of one.
we chose: the shape pydantic produces for the only two types `oe_field`
can describe. OE_F_OBJ carries `"additionalProperties":true`,
which is what `metadata` has and what `Dict[str, Any]` means;
OE_F_ARR carries `"items":{}`, the empty schema, which is what
`list[Any]` produces — and `list[Any]` is the only array shape
a table with no element kind can be describing. Both verified
by generating the same model through pydantic 2.13.4 and
diffing the documents.
a different implementer might: omit `additionalProperties` (pydantic does
for a bare `dict` rather than a `Dict[str, Any]`), which is a
one-key difference nothing reads. An author who needs a TYPED
element schema has `action_schema_json` and this generator will
never grow one, because oe_field carries no element kind to
build it from.
GAP-15
RETIRED — a type that is not a string is now named the way Python's str() names it — src/proto/pyrepr.c, src/proto/decode.c
GAP-16
Declared at src/wire/json.c:183 ↗:
GAP-16: which member wins when a document names one key twice.
unspecified: RFC 8259 section 4 says the names in an object SHOULD be unique
and that the behaviour of an implementation given a repeated
name is unpredictable. Nothing upstream states a rule either,
because nothing upstream had to: a dict comprehension has one.
we chose: the last member wins for a lookup, and every member stays in the
tree in the order it arrived — so a lookup agrees with the
upstream reader and a document that is read and written back is
unchanged. tests/t_json_parse.c pins both halves.
a different implementer might: refuse the document outright, which several
hardened parsers do; or keep the FIRST member, which is what a
streaming parser that ignores a key it already has would do.
Both are conforming, and each turns a request one of the others
accepts into a different value or a refusal.
Also referred to from: src/wire/json.h:322
GAP-17
RETIRED — id is validated as the two-arm union it is declared as, and coerced the way pydantic coerces it — src/proto/jsonrpc.c
GAP-18
Declared at src/proto/pyrepr.c:150 ↗:
GAP-18: a non-ASCII byte is passed through untouched.
unspecified: nothing in the protocol says how a peer's string is spelled
inside a validation message; this whole sentence is a Python
exception that leaked onto the wire.
we chose: ASCII exactly as CPython prints it, and bytes >= 0x80 verbatim.
That is right for every PRINTABLE character, which is what a
field name or a method name is, and differs from CPython for the
unprintable ones, where it consults a Unicode category table
this program does not carry and does not want.
a different implementer might: carry the table, or escape everything above
ASCII, and no conformance criterion would notice either.
GAP-19
Declared at src/server/mcp.c:51 ↗:
GAP-19: creating a session from a transport that holds none.
unspecified: the reference answers `openenv/session/create` by constructing
an environment, storing it under a fresh UUID and returning the
id — so the answer is a function of a table, a random source and
a clock, and this seam has none of the three. WART-17 in
PROTOCOL.md records that those sessions then consume the /ws
capacity budget and leak until closed, which is the reference's
own hazard and not a thing to reproduce blind.
we chose: -32000, the reference's own server-error slot for a create that
did not happen, with `data:null`. The two `data` shapes it uses
there — {"active_sessions":N,"max_sessions":M} for capacity and
{"factory_name":"..."} for a failed factory — both describe
something that was attempted, and nothing was attempted here, so
sending either would be a false report in a machine-readable
field.
a different implementer might: hold the table in the host and pass it in, at
the cost of a seam that is no longer a pure function of its
arguments. `oe_serve` is where that table belongs, and this is
the answer until a caller supplies one.
Also referred to from: src/proto/jsonrpc.c:963
GAP-20
RETIRED — the exact spelling of a 501 body is captured now — corpus cases 91-95, the five task 501s against the registered counter_env, confirm the chosen method names byte for byte — src/server/http_const.c
GAP-21
Declared at src/server/mcp.c:143 ↗:
GAP-21: whether `arguments` is validated against the published `inputSchema`.
unspecified: no document says, and the reference's two branches DISAGREE:
the FastMCP branch validates through pydantic, and the branch
whose tool shape we emit does not check the schema at all.
Measured on 8908 against a tool declaring `amount: integer`:
{"name":"bump","arguments":{"amount":"seven"}}
-> {"jsonrpc":"2.0","id":5,"result":"sevenseven"}
A declared integer accepted a string and the tool RAN. The TYPE
is never checked, and that half of the gap is STILL OPEN here:
`amount` is a declared key, so the gate below passes it, and our
`int` getter's fallback then answers 2. KEY SET ONLY means exactly
that — we gate on WHICH arguments a tool declares, never on their
types.
WHAT THE BRANCH DOES ENFORCE IS ITS SIGNATURE, through
`f(**arguments)`: CPython's calling convention rejects two input
classes before the body runs — `arguments` that is not a mapping,
and any key the tool does not declare. Four rows, live against
8906 (interop/mcp_srvonly_env.py, checked in). The `ours` column
is what THIS AGENT DOOR answers now that it reproduces that gate
(the block just below tool_call):
arguments reference 8906 ours
-------------------- -------------------------- --------------
null -32603 "…argument after ** -32603 "Tool
must be a mapping, not arguments must
NoneType" be an object"
[1] -32603 "…must be a -32603 (same
mapping, not list" sentence)
{"nope":1} -32603 "_bump() got an -32603 "Tool
unexpected keyword called with an
argument 'nope'" undeclared
argument"
cat {"x":1} -32603 "_cat() got an -32603 (same
unexpected keyword undeclared
argument 'x'" sentence)
The DECISION now matches on all four — an error where we used to
answer a result — but the SENTENCE does not and cannot: the
reference's is a leaked CPython TypeError naming a Python function
and an interpreter internal. So corpus case 90 ({"nope":1}) STAYS
a divergence, reclassified from "we answer {"result":2}" to
UNREPRODUCIBLE — same code, our own prose
(oe__rpc_fail_args_not_object / oe__rpc_fail_arg_undeclared,
jsonrpc.c). The key set we gate on is the PUBLISHED SCHEMA's
(oe__tool_arg_undeclared), not the handler's actual reads: the
approximation that stands in for the signature C does not have,
and it agrees with the signature for every environment here.
THE TRAINER DOOR IS A DIFFERENT STORY AND STILL DIVERGES. It does
not go through `**` at all: 8902 validates the call with pydantic
and answers `error_type:"invalid_args"` where WE STILL answer a
result, because the step-door `call_tool` (src/proto/decode.c,
src/server/session.c) hands `arguments` to the handler unvalidated
once its field set passes.
{"type":"call_tool","tool_name":"echo_message",
"arguments":{"message":"x","z":1}}
8902 -> …"error":{"error_type":"invalid_args","message":
"1 validation error for call[echo_message]\nz\n
Unexpected keyword argument …"}
ours -> …"result":"x","error":null
OE_TOOLERR_INVALID_ARGS exists because the reference emits it;
this framework never raises it, and an author who wants those
bytes calls oe_tool_fail(out, OE_TOOLERR_INVALID_ARGS, …).
we chose: on THIS agent door, the `**` KEY SET half and nothing more:
refuse a non-mapping `arguments` and any undeclared key with
-32603, no type check and no JSON Schema validator (the reference
runs neither on this branch). When the gate passes,
`call->arguments` still reaches the handler exactly as the peer
sent it — NULL for an absent key, `params.get("arguments", {})`
one language over — so a declared key with a surprising VALUE
costs an author nothing: every getter in section 3 of
<openenv/openenv.h> takes a fallback, and `{"amount":"seven"}`
reaches `oe_obj_get_int(args,"amount",1)` and gets the 1. On the
TRAINER door and the step-door `call_tool`, no argument gate at
all — that divergence stands and is named above.
a different implementer might: also check the TYPES against the schema they
published, and would then reject traffic a conforming server
accepts — the GAP-01 failure mode in a new place, and the reason
we stopped at the key set. Or gate the TRAINER door too, at the
cost of a second refusal path whose sentence is FastMCP's and not
the interpreter's. An earlier revision of this comment said we
took NONE of the `**` half, "because the key set a C tool declares
is whatever its handler reads, and nothing in struct oe_tool knows
it"; we now take it on the strength of the PUBLISHED schema, and
the gap between advertised keys and read keys is the divergence
that remains rather than the one that stopped us.
Also referred to from: envs/echo_tools.c:199 include/openenv/openenv.h:852 include/openenv/openenv.h:977 src/proto/decode.c:1037 src/proto/decode.h:261 src/proto/decode.h:264 src/proto/decode.h:266 src/proto/jsonrpc.h:369 src/proto/jsonrpc.h:384 src/proto/tool.h:200 src/proto/tool.h:218 src/server/http_stateless.c:231 src/server/mcp.c:269 src/server/session.c:356 src/server/session.c:364
GAP-22
Declared at src/proto/decode.c:928 ↗:
GAP-22: one declaration gates both doors.
unspecified: nothing states what makes an environment tool-capable. The
reference uses TWO tests that are not the same test and can
disagree — `POST /mcp` asks whether the environment has an
`mcp_client` or an `mcp_server` attribute
(http_server.py:897, :945), while the step door asks whether
`action_cls` is the base `Action` or one of the two MCP
action classes (serialization.py:29-38) and then relies on
the environment's own `step` to recognise what it is handed.
I ran an environment that satisfies the first and not the
second, on 127.0.0.1:8908: `POST /mcp` `tools/list` returned
two tools, and `POST /step {"action":{"type":"list_tools"}}`
on the same process returned its ordinary observation with no
`tools` key in it. Two doors, one environment, two answers to
"do you have tools".
we chose: `env->tools != NULL`, for both doors, and nothing else. One
declaration, so the two doors cannot disagree, and the C API
has one thing to teach instead of two.
a different implementer might: gate the step door on the action shape
alone and let any environment answer `list_tools`. Cost of
OUR choice, and it has two halves:
(a) An environment that declares a tool table AND its own
`type`-keyed action loses two values of `type` on the step
door — `PROTOCOL.md` says `type` is not globally reserved,
so that is a real, if narrow, collision, and it is why the
gate is the table rather than the two strings.
(b) THE STEP DOOR ACCEPTS TOOL ACTIONS THAT A GENERATED
`GET /schema` DOES NOT DESCRIBE. This sniff runs BEFORE
validate.c, so for any environment with `tools != NULL` the
two tool shapes bypass the published action model entirely —
extra keys and all. The SAME BINARY answers
`{"type":"list_tools"}` for envs/counter.c, which has no tool
table, with `422 extra_forbidden on 'type'`: the document is
contradicted only for tool-bearing environments, the
surprising direction, since those are the ones whose schema a
consumer most needs.
WHAT THE REFERENCE PUBLISHES, AND WHY WE NOW PUBLISH IT TOO.
8902's `GET /schema` is 3061 bytes and its action model is the
framework's `CallToolAction`, carrying `type`, `tool_name`
and `arguments` — a `const`/`anyOf`/`$defs` shape no field
table generates. Corpus case 88 CAPTURED it, so closing the
gap did not mean inventing protocol (which is what an earlier
version of this comment claimed): the bytes were in the corpus
the whole time. envs/echo_tools.c hands the captured
`CallToolAction`/`CallToolObservation` over through the
`action_schema_json` escape hatch (item 7), so its document
now reproduces 8902's byte for byte and REPLAYS in
tests/t_replay.c. Live, from build/rel/bin/echo_tools:
GET /schema -> 3061 bytes, action.title "CallToolAction",
properties type/tool_name/arguments (corpus case 88)
POST /step {"action":{"type":"list_tools"}}
-> 200, the full tool table
POST /step {"action":{"type":"list_tools","extra":1}}
-> 422 {"detail":[{"type":"extra_forbidden","loc":
["extra"],"msg":"Extra inputs are not permitted",
"input":1,"url":"https://errors.pydantic.dev/2.13/v/
extra_forbidden"}]} (corpus case 89, byte for byte)
Item 8(a) closed the extra-key half: the sniff now validates
the shape against a FIXED FIELD SET — {metadata,type} for a
list, {metadata,type,tool_name,arguments} for a call — and a
key the shape does not declare is `extra_forbidden` exactly as
the `extra="forbid"` model would refuse it — see
check_tool_action_extras below, and decode.h section 3. Case
89 REPLAYS. So the residual
cost is narrower still: what the step door STILL accepts is a
`call_tool` with a MISSING `tool_name`, which the published
model marks required — item 8 scoped the sniff to the field
set, not to required fields. And the document is escaped only
by QUOTING — a tool environment that leans on the generator
(its own action fields, or none) still publishes one that does
not name the tool actions.
THE EVIDENCE THAT THE TITLE ALONE DECIDES THIS. verifiers'
shipped OpenEnv taskset (v1/tasksets/openenv/taskset.py:96-100)
reads ONE field to decide whether an environment has tools —
`action_schema["title"] in {"Action","CallToolAction",
"ListToolsAction"}` — and only then does it
`step({"type":"list_tools"})` to discover them. Our old
generated title was `EchoAction`, in no such set, so against us
that branch never fired: a real trainer saw a tool environment
as a toolless one, and our own examples/oe_agent — schema-
driven, reading the same title — sent `{}` at echo_tools for
the same reason. `CallToolAction` is in the set, so discovery
fires now, exactly as against the reference. The interop oracle
in tools/interop.sh asserts that outcome on our live server;
docs/NOT-THE-PROTOCOL.md divergence 8 is the write-up.
Also referred to from: envs/echo_tools.c:286 envs/echo_tools.c:85 src/proto/decode.c:1046 src/proto/decode.c:849 src/proto/decode.c:855 src/proto/decode.h:255 src/proto/decode.h:312 src/proto/encode.h:204 src/server/http_stateless.c:316 src/server/http_stateless.c:318 src/server/session.c:424 src/server/session.c:430
GAP-23
Declared at src/wire/http.h:17 ↗:
GAP-23: connection reuse, and WHEN a connection ends — the blind spot the byte
replay cannot see by construction.
unspecified: nothing above this layer says whether a client may send a second
request on one socket, nor how long a server may hold that socket
after the last byte of a response. The corpus cannot answer
either, because all 78 captured HTTP exchanges forced
`Connection: close` — so not one of them records a reuse or a
timing, and the whole class is invisible to a byte diff BY
CONSTRUCTION: a fossil has no second request and no clock. It is
why a client once waited ~1001 ms to EOF for a 0.14 ms answer and
all 103 cases stayed green.
we chose: one exchange per connection (the smuggling posture below), and a
half-close the instant the response is flushed and the request
drained — so time-to-EOF is the compute time and OE_CONN_LINGER_MS
is the DRAIN bound, not the answer's delay. See
oe_net_shutdown_write in wire/net.h and maybe_half_close in
src/server/conn.c.
a different implementer might: keep the socket open for a second request — and
the reference does exactly that, honouring keep-alive whenever a
client does not force close. Measured live: a second request on
one socket gets a second answer at the reference and a clean EOF
here. The corpus cannot tell the two apart, because its capture
client always sent `Connection: close`, so tools/interop.sh
measures the divergence live instead.
WHY THE PARSER TOUCHES NO SOCKET AND NO ARENA.
`oe_http_parse` is a pure function from a byte range to a struct of slices.
It allocates nothing, so it cannot fail for want of memory; it blocks on
nothing, so a peer that sends one byte and stalls cannot park the server.
The connection layer reads non-blocking into a buffer and calls this only
when the buffer might hold a whole request; when it does not, the answer is
OE_EAGAIN and the loop comes back. There is not one partial-read case in
this file.
The price is one invariant the caller owes us, and it is stated at the top
of the connection layer too: THE BUFFER MUST NOT MOVE OR CHANGE while an
oe_http_req derived from it is alive. Every slice in that struct points
into the caller's bytes. Nothing is copied.
THE SMUGGLING RULES.
Request smuggling is what happens when two programs reading the same bytes
disagree about where one request ends and the next begins. It needs a
disagreement and it needs a reused connection. This file removes the
disagreement by refusing every construct that has ever produced one, and
the server removes the reuse by closing after a single exchange.
duplicate Content-Length 400, even when the two values agree
Content-Length that is not 1*DIGIT 400 ("-1", "+5", "0x10", "5, 5", "")
Content-Length over the body cap 413
ANY Transfer-Encoding 501, before the value is even read
bare CR or bare LF as a terminator 400
whitespace before a header colon 400 ("Content-Length : 5")
obs-fold (a continuation line) 400
NUL anywhere in the header block 400
a target that is not origin-form 400 (no "http://host/x", no "*")
a repeated Host, or none on 1.1 400
header block over the byte cap 431
more headers than the count cap 431
target over the target cap 414
Note what is NOT in that list: there is no rule that prefers Content-Length
to Transfer-Encoding, or the first Content-Length to the second. A parser
with a preference has an opinion an attacker can exploit. This one has no
opinion, because ambiguity is answered with a status code and a closed
socket.
THE PERCENT SIGN IS SOMEBODY ELSE'S PROBLEM. `req.path` is the raw bytes the
peer sent, undecoded. Whether `/%77idget` should reach the `/widget` handler
is a routing decision and routing is not in this layer.
Also referred to from: src/server/conn.c:500 src/server/conn.h:270 src/wire/net.h:200
GAP-24
Declared at src/server/http_const.c:445 ↗:
GAP-24: whether to implement the Task API's five provider-backed routes.
unspecified: the six routes are mandatory to REGISTER (CONF-20), but a 501 is
a conforming answer for a provider-less environment and every
environment this project ships is provider-less. Whether to grow
the oe_env callbacks that would answer them with real task data —
list_splits, list_tasks, num_tasks, get_task, get_task_range, and
the reference's _normalize_split fold — is not the protocol's to
decide.
we chose: decline, on evidence. No upstream client consumes these routes:
grepping trl/, verifiers/, prime-rl/ and torchforge/ for the paths
and for the EnvClient task methods found nothing that calls them,
so the callbacks would be dead code graded only by a test we wrote.
interop/task_env.py stands a reference env WITH a provider, and the
corpus holds the divergence as REFUSED (cases 97-102): the
reference answers real dataset rows and a 400 for a bad index, and
we answer 501. docs/PROTOCOL.md section 2 documents every shape.
a different implementer might: implement them. That is the NAMED RE-OPEN
CONDITION — the day an upstream client consumes these routes,
oe_env grows the five callbacks and the _normalize_split fold, and
corpus cases 97-102 flip from REFUSED to REPLAY.