Provenance — where this implementation's knowledge came from
No upstream code is vendored in this repository. Not a line.
That is the first thing to establish, because it is the thing that makes everything else here worth reading. The protocol was re-derived by running the reference — huggingface/OpenEnv, the Python original — and recording what came out of the socket. This file is the record of how.
And the claim is stated as an audit rather than a slogan, because the slogan
version was not quite true. git ls-files '*.py' returns twenty-seven .py
files: twenty-four under interop/, plus tools/site.py, tools/mutate.py and
tools/synthlab_parity.py.
Of the twenty-seven, fourteen import the reference (import openenv), one
imports the official mcp SDK and nothing of the reference's, and twelve import
nothing of the reference's at all: eleven of those import nothing but the
standard library or a training framework, and the twelfth imports the owner's own
task generator out of the gitignored
reference/synthlm, which is a specification this repository grades itself
against and is not upstream code. That is the whole population: 14 + 1 + 12 = 27.
The string BSD-3-Clause appears in exactly one file that is not prose about it.
All twenty-seven are named below — an audit that summarised a directory would be
the slogan again:
What git ls-files '*.py' finds | What it is |
|---|---|
interop/interop_client.py, interop_rewards.py, interop_bigframe.py | ours. Harnesses that import openenv.core.env_client and drive our server with it |
interop/interop_tools.py | ours. The same job on the agent door: imports openenv.core.mcp_client and drives envs/echo_tools.c with it |
interop/probe_env.py, reward_env.py, mcp_srvonly_env.py, task_env.py | ours. Environments defined against the reference's Environment base class, so the corpus is captured from a real reference server — task_env.py the one declaring a TaskProvider, so the Task API's success shapes exist as observed bytes |
interop/prod_env.py | ours. probe_env.CounterEnvironment re-registered through the reference's HTTPEnvServer.register_routes in ServerMode.PRODUCTION — the six corpus cases captured on port 8903 |
interop/docker_launch.py | ours. The Docker-launcher oracle: imports openenv.core.generic_client.GenericEnvClient and drives miniopenenv:counter the way the launcher a trainer calls does |
interop/interop_specmcp.py | ours. Points the official mcp SDK — not the reference — at both servers in turn and asserts they refuse it with the same sentence |
interop/bench.py | ours. The throughput bench. Standard library only — socket, subprocess, time — it drives both servers over a socket and imports neither the reference nor mcp |
interop/drift.py | ours. The make drift classifier. Standard library only — json, subprocess — it reads the corpus manifest and the reference's file paths as strings, importing nothing |
interop/mcp_minimal_server.py | ours. The minimality twin. Standard library only: no openenv, no fastapi, no fastmcp |
interop/recapture.py | ours. The capture tool that produced tests/fixtures/reference-corpus.json. Standard library only — it speaks to the reference over a socket, never in-process |
interop/interop_verifiers_schema.py | ours. Oracle 6: runs verifiers' own tool-discovery predicate against our GET /schema. Standard library only — ast, json, urllib — importing neither the reference nor mcp |
interop/train_bandit.py | ours. Oracle 8: imports openenv.core.env_client and runs epsilon-greedy against arcade/bandit.c — the oracle whose verdict is a learning curve rather than a diff |
interop/bandit_env.py | ours. arcade/bandit.c ported onto the reference's Environment base and create_app, so make record-arcade can time one game on two server implementations. Imports the reference; a port, never a copy — there is no upstream bandit to copy from |
interop/openenv_text.py | ours. The framework-free core the three training adapters share: it loads the manifest make arcade-up writes, describes each server off its own GET /schema, coerces a model's reply onto whichever action field that schema named, and plays one episode over an RFC 6455 client of its own. Standard library only — the same forty lines recapture.py carries, for the same reason |
interop/train_trl.py | ours. The TRL adapter: imports the reference (openenv.GenericEnvClient and openenv.core.harness) and hands one session factory to TRL's async-GRPO harness worker. trl, datasets and transformers are imported inside main(), so the session half loads with no trainer installed |
interop/train_prime.py | ours. The prime-rl config writer: it describes every manifest entry over the wire and prints a run config. Standard library only, and short, because prime-rl needs no adapter from us — the verifiers OpenEnv taskset already reads the schema itself |
interop/train_tinker.py | ours. The Tinker adapter: an Env, an EnvGroupBuilder and an RLDataset written against tinker_cookbook's own protocol. Imports chz, tinker and tinker_cookbook — a training framework, and nothing of the reference's |
interop/adapters_smoke.py | ours. What make adapters runs: four legs that drive the four files above against the real arcade with no GPU and no model. Leg A imports the reference and puts upstream's own MCPHarnessAdapter in front of our session |
interop/exploit_battery.py | ours. The cheap-heuristic battery: named cheap solvers run over the four arcade task servers' fresh tasks over a real WebSocket, and over the saved control rows under the gitignored reference/synthlm-rows. Standard library only — its RFC 6455 client is the forty lines recapture.py carries, for the same reason |
tools/site.py | ours. The static-site generator, standard library only, and one of the three .py outside interop/ |
tools/mutate.py | ours. The make mutate driver — plants one edit per rebuild and grades it on the SAN leg. Standard library only, and the second .py outside interop/ |
tools/synthlab_parity.py | ours. The synth lab's grammar oracle: parses the four arcade task servers' input_text and the owner's published dataset rows under one grammar per family, and under --prove imports synthlm — the owner's own generator, out of the gitignored reference/synthlm — to recompute each answer independently. That import is a specification, not upstream code: nothing of huggingface/OpenEnv is in it. Standard library otherwise, and the third .py outside interop/ |
BSD-3-Clause in reference-corpus.json | not source. It is inside the captured body of GET /openapi.json — recorded evidence, byte for byte, the same as every other case |
The qualifier "that is not prose about it" is doing real work, and the reason is
the same lesson as everything else here. An earlier draft said the string appears
once in the tree, which is a checkable claim, and
git ls-files -z | xargs -0 grep -o BSD-3-Clause | wc -l refutes it. This
paragraph and the README's licence section each add hits of their own, and so
does any future paragraph about the audit. Writing a raw count down is what broke
it. Scoped to source and fixtures, the claim survives being written down, which
is the only kind of claim worth making.
No file in this tree was copied from, or derived from, an upstream file.
The two licences, since a careful reader asks. Upstream is BSD-3-Clause;
this project is Apache-2.0 (see ../LICENSE ↗). Nothing transfers,
because nothing was taken. The audit above is the evidence, and a protocol —
route names, key order, status codes — is not copyrightable subject matter in
the first place. Apache-2.0 was chosen over BSD-3-Clause for its explicit patent
grant and its explicit statement about contributions, both of which matter more
for a library people are meant to embed than the three-clause form does.
The second thing: C is not an accident of taste. It is impossible to
accidentally paraphrase Python in C. There is no dataclass to copy, no decorator
to keep, no model_dump() whose behaviour you can adopt without understanding.
Every line had to be re-derived from bytes, and where the bytes did not answer
the question, GAPS.md says so at the line that had to choose anyway.
What the reference was, exactly
| project | huggingface/OpenEnv ↗ |
| revision | 60ecc7b7d6ecf830448aa9fbf330e3c7cecbc542 |
| CPython | 3.13.7 |
fastapi | 0.140.0 |
starlette | 1.3.1 |
pydantic | 2.13.4 |
uvicorn | 0.51.0 |
mcp | 1.28.1 |
fastmcp | 3.4.4 |
websockets | 16.1.1 — the one row captured_with does not record |
The revision, the Python version and every row above except websockets are
recorded inside the corpus itself, under captured_with. So a fixture file can
never drift from the claim about where it came from, and make interop refuses
a venv, or an interop/requirements.txt, that does not match them. websockets
produced RFC 6455 framing through uvicorn during the capture, and
interop/recapture.py did not write it into the manifest. So that one row rests
on this table alone; recording it belongs to the next capture.
The library versions matter more than they look. pydantic owns the exact
wording of eighteen validation messages this project reproduces. fastmcp
owns the shape of a tools/list reply that this project refuses to reproduce
(see NOT-THE-PROTOCOL.md); the six MCP rows below are
its serialisations, which is why it is pinned.
The reference tree is not part of this repository — .gitignore has
/reference/ in it. tools/interop.sh looks for it at reference/OpenEnv and
skips cleanly when it is not there, so a clone with no Python on it still
runs everything in make check.
The corpus — 103 byte-exact cases
tests/fixtures/reference-corpus.json ↗
is the fossil record: 78 HTTP exchanges and 25 WebSocket sessions, captured from
a running reference server, byte for byte. HTTP entries carry the status line
and every header. WebSocket entries carry the RFC 6455 handshake, the expected
Sec-WebSocket-Accept token, and every frame in both directions.
interop/recapture.py ↗ is what produced it.
Re-capturing is a deliberate act with a version bump attached, not something
that happens by accident on somebody's laptop. That is why the file is checked
in rather than produced by the test.
It is not a one-command regeneration, and the script's own docstring says so
rather than leaving it to be discovered. Six servers must already be
listening. Five are ours: interop/probe_env.py on 8901, interop/prod_env.py
on 8903, interop/reward_env.py on 8905, interop/mcp_srvonly_env.py on 8906
and interop/task_env.py on 8907, each with its exact uvicorn command
written down. The sixth is the reference's own echo_env on 8902, for which
this repository ships no launcher, because it is the reference's environment
and not ours. That last one holds nine of the one hundred and three cases, six
of the fifteen exchanges tests/t_replay.c prints as divergent, and every byte
of FastMCP's serialisation in the corpus.
How the corpus is used is the part that makes it more than a directory of
.json files. tests/t_replay.c pushes every case through the four pure
functions in ../include/openenv/embed.h ↗ —
no socket, no port, no Python, no clock — and diffs the bytes.
$ ./build/rel/tests/t_replay
103 cases from tests/fixtures/reference-corpus.json
captured from 60ecc7b7d6ecf830448aa9fbf330e3c7cecbc542
...
REPLAYED 103 exchanges, byte for byte
EXPECTED-DIVERGENT 15 exchanges
REFUSED 14 bytes we will not emit, on evidence
UNREPRODUCIBLE 1 a Python library's artefact, not ours to forge
NOT-YET 0 implementable, and not yet built
OUT-OF-SCOPE 0 exchanges
103 cases -> 118 exchanges: 88 replay whole, 14 do not, 1 split
The fifteen that do not replay are printed by name with the reason, and the test asserts every count. So an exchange the classifier ever quietly dropped is a test failure, not a shrinking number nobody watches.
The verdict is per EXCHANGE and not per case, which is the difference between this table and the one it replaced. A WebSocket case is a CONNECTION of up to eight frames, and while a case stood or fell whole, one frame could take three others out of the corpus with it. Corpus case 74 sat entirely outside the replay while three of its four frames were OpenEnv's own shapes carrying OpenEnv's own values. Hence the split row: exactly one case is partly reproduced and partly diverged, and rounding it to either whole verdict would be the arithmetic that hid it.
The fifteen fall into four groups:
- two
/openapi.jsondocuments this project answers minimally — one of them the production-mode document, ten paths not thirteen; - six MCP rows where the reference's answer is a Python library's
serialisation rather than OpenEnv's:
mcp.types.Tool.model_dump()'s nine keys on each of the two agent doors,fastmcp.client.client.CallToolResultin tworesultvalues, the session extension, and one error code where the reference's own two branches disagree with each other; - one tool-door row: the undeclared keyword argument CPython refuses for a tool, which our agent door now refuses too but cannot spell in the interpreter's own words;
- and six task-API rows, corpus cases 97–102:
interop/task_env.pydeclares a task provider, so the reference answers real dataset rows where we answer501— a conforming answer for a provider-less env, and a refusal on evidence, because no upstream client consumes these routes (GAP-24is the ledger entry, with the named re-open condition).
(Three that used to be on this list — the step door's extra key,
GET /schema itself, and the three production-mode 404s — became replays as
items 7, 8 and 9 landed.) Each names the field and the type in the
classifier's own words, on every run.
Each also carries one of three verdicts, subtotalled under the banner and asserted like every other count:
- refused — on evidence, where we could emit the bytes and decline to: the
FastMCP shapes, the session extension, the minimal OpenAPI document, the
task-API
501s; - unreproducible — the one row whose sentence is CPython's own;
- not-yet — now zero, the conformance arc having closed it.
That last subtotal is the one watched to zero, and it has arrived.
A not-yet that comes back — a future capture we could match but have not —
fails a line in t_replay.c. OUT-OF-SCOPE is zero too. Both empty
buckets are printed rather than deleted, so that setting a future capture
aside, or parking a new to-do, still costs somebody a line.
A fossil cannot be asked a new question
A corpus proves you still agree with what the reference said. It cannot prove you agree with what the reference does. So fifteen more oracles grade this implementation, and the judge in every one of them is somebody else's code:
Fourteen, in seven families — the count is 1 + 5 + 4 + 1 + 1 + 1 + 1, because
family 2 drives four different servers with five scripts and family 3 grades the
client boundary from both chairs:
| # | Oracle | What only it can prove |
|---|---|---|
| 1 | openenv validate --url | the reference project's own conformance suite, run against our server. Six required criteria. It is not an argument for tool support: _validation.py:340-378 posts {} to /mcp and checks only that the answer is a 200 carrying "jsonrpc": "2.0", which the -32600 body already satisfied. The oracle exists to prove the tool vocabulary did not BREAK it |
| 2a–2d | the genuine upstream Python EnvClient completing an episode against our server — against envs/counter.c, against envs/rewards.c, at the frame cap, and against tour/01_minimal.c, which links no protocol library at all | that a real client, with none of our code in it, gets what it expects |
| 2e | the genuine upstream MCPToolClient — a different client, the reference's only consumer of a tool table — listing and calling two tools on envs/echo_tools.c | the agent half of the dual API boundary. It reads the table into a pydantic model with extra="forbid" and three fields, so a fourth key on the wire fails before any assertion of ours runs |
| 3a, 3b | our client completing an episode against the genuine upstream Python server, and the same client's bytes diffed across both servers | the reverse direction. A client tested only against its own server is a mirror, not a measurement |
| 3c, 3d | tour/03_client.c — the client protocol hand-written with src/wire/ and nothing from src/proto/ — driving the same genuine Python server to done, and its client frames diffed byte for byte against examples/oe_agent's | that the counted client half (make lines-tour) is the real protocol and not a paraphrase: the hand-written chair and the library chair emit identical bytes, and a server we did not write accepts both |
| 4 | the official Model Context Protocol SDK driven at the reference and at both of our servers, all three dying on McpError: Method not found: initialize | that "we are not an MCP server" is a reproduction and not a shortfall. Neither is the reference, and this is the byte-level proof of it |
| 5 | interop/mcp_minimal_server.py — a stdlib-only Python server the same MCPToolClient completes against on both doors with identical tool tables | how small a tool surface can be and still be real. A C implementation cannot demonstrate that alone: a reader has no way to tell a small protocol from a large one implemented tersely |
| 6 | verifiers' own OpenEnv taskset predicate — upstream's literal, read out of its source at run time — deciding from GET /schema alone that envs/echo_tools.c HAS tools | that the code a real trainer ships takes the tool branch against us. It reads one field, action.title, and our old generated title silently failed the test while every byte-diff stayed green |
| 7 | a socket-level probe timing time-to-EOF and attempting connection reuse at both servers | the connection lifetime the corpus cannot see: its capture forced Connection: close on every exchange. Ours answers in sub-millisecond and closes (one exchange per connection, the GAP-23 posture); the reference keeps the socket alive — a real divergence, asserted rather than hidden |
They are make interop rather than part of make check, because each needs a
second implementation running. sh interop/setup.sh is what builds that
second implementation. It clones the reference at the pinned revision, makes
the venv, installs the reference package's own dependency set, and then
installs interop/requirements.txt over the top so the seven pinned versions
win. Without it every oracle prints SKIP, which is honest and is not evidence.
The pin is read from the corpus, and the whole of it is checked.
tools/interop.sh reads captured_with.packages out of
../tests/fixtures/reference-corpus.json ↗
— the six versions in the table above, written there by interop/recapture.py
at capture time. It refuses a venv that differs from any of them rather than
grading against it, because a byte diff produced by a different library is
upstream's change reported as our regression. It refuses interop/requirements.txt
on the same comparison, so the pins file is a checked copy of the fossil and not
a second opinion. It used to check pydantic alone, against a number scraped out
of requirements.txt, while fastmcp — which owns six of the fifteen divergence
rows — was not pinned at all.
Real output, this machine, this tree:
$ PORT_PYTHON=8950 sh tools/interop.sh
interop.sh — /home/ericflo/Development/miniopenenv
pinned peer: fastapi 0.140.0, fastmcp 3.4.4, mcp 1.28.1, pydantic 2.13.4, starlette 1.3.1, uvicorn 0.51.0
ok oracle 1: reference validator — 6/6 required criteria
ok oracle 2a: upstream EnvClient drove envs/counter.c to done
ok oracle 2b: upstream EnvClient read bool/int/float rewards back as themselves
ok oracle 2c: our frame cap costs the real client a 300 KB action, exactly as limits.h says
ok oracle 2d: upstream EnvClient drove tour/01_minimal.c to done — the two-endpoint claim
ok oracle 2e: the genuine MCPToolClient listed and called two tools on envs/echo_tools.c
ok oracle 3a: examples/oe_agent drove the real Python server to done
ok oracle 3b: same client, both servers — 5 replies identical byte for byte
ok oracle 3c: tour/03_client.c drove the real Python server to done — the hand-written client
ok oracle 3d: examples/oe_agent (library) and tour/03_client (wire) sent 3 identical client frames byte for byte
ok oracle 4: the official mcp SDK dies the same way on all three servers — McpError: Method not found: initialize
ok oracle 5: 230 lines of stdlib Python satisfy the genuine MCPToolClient on both doors
ok oracle 6: verifiers's taskset predicate decides envs/echo_tools.c HAS tools
ok oracle 7: time-to-EOF 0.112 ms here / 0.313 ms at the reference (both sub-linger; ours was ~1001 ms before the half-close) — one socket carries one exchange here, a second at the reference (keep-alive): the lifetime divergence the corpus never captured
interop.sh: every oracle passed
Oracle 2d is the guard on milestone 1, and it did not exist until Phase 4.
tour/01_minimal.c is sixty-four statements of C with the JSON written as
string literals and no protocol library. The claim that the genuine upstream
client drives it to done is the sharpest pedagogical claim in this
repository, and for three milestones nothing checked it. Every other oracle drives envs/counter.c,
which is the whole four-thousand-line library, so a regression in the
hand-written literal path would have survived all of them.
Two of those rows are worth a sentence each.
Oracle 2c measures a cost rather than a success. OE_WS_MAX_FRAME is
256 KiB where the reference accepts 16 MiB, and the genuine upstream client
does not fragment. So a single action over our cap loses the message and the
connection. That oracle sends a 300 KB action to both servers and asserts the
reference takes it and we refuse it. A limit that nothing measures is a limit
nobody can argue with; SECURITY.md is where the argument is.
Oracle 3b runs one client against both servers and diffs. Five replies, identical byte for byte. That is the strongest statement in this repository that does not involve the word "corpus".
Oracle 4 is the only one here where PASSING means a peer failed, and that
inversion is the point. initialize is the single measured blocker — not the
_meta explanation this project's own notes once carried, which is wrong.
mcp.types.JSONRPCRequest.model_fields is ['method','params','jsonrpc','id'],
and _meta travels inside params, which OpenEnv accepts and ignores. The
script unwraps the SDK's two nested ExceptionGroups before comparing. A
comparison of the outermost message would have matched "unhandled errors in a
TaskGroup" against itself, and passed whatever either server said.
Where each kind of claim comes from
Every factual claim in this codebase is one of four kinds, and the source tells you which:
| Kind | Where it appears | How to check it |
|---|---|---|
| captured | corpus case N in a comment | open the fixture; the bytes are there |
| probed | [probed] beside a citation | a question the corpus does not answer, put to a running reference server once. Recorded in docs/evidence/ |
| read | a file.py:line citation | read that line in the reference tree |
| ours | a GAP-nn marker | nothing upstream answers it; the marker says what we chose and what somebody else might |
The distinction is not pedantry. A wart backed by captured bytes is a fact
about the protocol. A wart backed by a reading of Python is a fact about one
implementation of it. The two deserve different confidence from anybody
writing a third implementation. WARTS.md prints the observed:
line of every marker verbatim so the reader can tell which they are looking at.
There is one row in the divergence table in src/proto/encode.c — how a float
is spelled — that says outright: NOTHING pins this. Every captured reward is
in the range where the HTTP and WebSocket float formatters agree. So that row
rests on unit tests rather than on captured traffic, and saying so is more useful
than a citation that does not hold.
The one thing the corpus made us do that a specification would not
The reference's /step validator accepts the JSON string "42" as an integer,
because pydantic's default mode is lax. PROTOCOL.md's first draft suggested
skipping that — strict validation is simpler and more defensible.
It is also unimplementable. Two of the captured cases carry the error slug
int_parsing — case 20 (POST /step bad type) and case 66
(ws err: VALIDATION_ERROR int_parsing) — and int_parsing is the report of a
conversion that was attempted. A strict validator cannot emit it, because it
never attempts one. Skipping lax coercion would have made the primary acceptance
test unpassable and added two more cases to the exception list above. So it is
implemented, in about twenty-five lines, and the two int_parsing cases replay
rather than joining the nine.
That is the shape of the whole project: the bytes won every argument they were in.
Reading on
PROTOCOL.md— what was derived: our statement of the protocolWARTS.md— the reference behaviours a compatible server must reproduceGAPS.md— the questions the derivation could not answerNOT-THE-PROTOCOL.md— what we refuse to implement, and why each refusal is safeevidence/OBSERVED-WIRE.md,evidence/REFERENCE-NOTES.md— the probe notes the source cites