miniopenenv
The OpenEnv protocol, rebuilt from observed bytes in dependency-free C99 — small enough to read in an afternoon, faithful enough that Hugging Face's own client cannot tell it from the reference.
C99 · zero dependencies · Apache-2.0 · ericflo.github.io/miniopenenv ↗
New to OpenEnv? It is Hugging Face's standard for running reinforcement-learning
environments as services. The environment runs in a container and speaks HTTP and
WebSocket. The loop it speaks is the classic Gym one: reset(), step(action),
observation, reward. So a trainer in one process can drive a game, a browser, a
codebase or a robot sim in another. One run of the loop — a reset, then steps,
until done — is an episode. The reference implementation,
huggingface/OpenEnv ↗, is 752 Python files
and 164,342 lines at the revision this repository captured (60ecc7b7).
The protocol buried inside those lines is a JSON envelope with four keys, seven
WebSocket message types, and sixteen routes. This repository is that protocol,
alone. It was reimplemented from captured wire traffic, separated from the
framework around it — and the separation is enforced, not described. src/wire/
may not contain a single OpenEnv word. src/proto/ may not contain a single system
call. make check fails the build if either stops being true.
It is written in C not because C is a good language for RL infrastructure, but because you cannot accidentally paraphrase Python in it. Every line had to be re-derived from bytes observed on a socket.
Pick a door:
- You want to understand OpenEnv — read What the protocol is, then take the tour: sixteen steps, about an hour. At the end you will know the protocol rather than a codebase.
- You are implementing OpenEnv in another language — three files are the spec
that does not exist upstream.
docs/PROTOCOL.mdis the numbered requirements.tests/fixtures/reference-corpus.json↗ is 103 byte-exact captured cases to test against.docs/GAPS.mdlists the places where the protocol is underspecified and you will have to choose. - You are debugging any OpenEnv setup, even a pure-Python one —
examples/oe_probe.c↗ connects to any OpenEnv server and annotates every byte in both directions: the handshake, the frame headers, the rule each field obeys.make && build/rel/bin/oe_probe http://your-server --episode. - You want the smallest possible OpenEnv server — a complete conforming environment is 45 statements of C and ships as a 211 kB container. The Python base image is 467 MB.
- You want to train something against it, today —
arcade/has a recorded gallery of fourteen environment servers built on this protocol. Ten are games — a bandit, cart-pole, snake, 2048, Wordle, blackjack, a maze, pong, connect four and Kuhn poker — and one trainer learns to play them over a real WebSocket. Four are task generators: reset one with a seed and a difficulty and it mints a problem no file has ever held, computes its own answer, and pays exact match, which is the verifiable-reward shape a GRPO-style trainer consumes.make train-cartpoleprints a learning curve of 2000 episodes over a real WebSocket in a few seconds;make task-dynamints a thousand distinct tasks and measures the rate. All fourteen also print the whole decision into oneinput_textfield, so a language model can play them with no prompt written per game. Beside them, the math room adds eight text-first environments for quantities, algebra, functions, transformations, discrete structures, probability, data, and checked reasoning.make math-measureindependently solves and attacks 300 generated tasks;make math-upleaves all eight ready for a training run.make arcade-upstarts the recorded fourteen and writes the manifest that three shipped training adapters read, andmake adaptersdrives those adapters against the running servers with no GPU and no model weights. - You found a byte we got wrong — file it. A byte-level counterexample is the
most valuable issue this project can receive.
docs/GAPS.mdis the roadmap of decisions most worth arguing with.
Contents: Sixty seconds · The protocol, entire · Three lessons from the wire · Proof · Size · What OpenEnv is not · Write an environment · Embed & ship · Limitations · Going deeper · Provenance
Sixty seconds
No dependencies. No autotools, no CMake, no submodules. Any C99 compiler on a POSIX
system: Linux, macOS, the BSDs. (Elsewhere, see docs/PORTING.md;
the port is one file.)
$ git clone https://github.com/ericflo/miniopenenv && cd miniopenenv
$ make
built: rel lib=build/rel/libopenenv.a programs=59
$ ./build/rel/bin/echo --port 8010 &
openenv info listening on: 0.0.0.0:8010 max_sessions=8 max_connections=64
$ curl -s localhost:8010/health
{"status":"healthy"}
$ curl -s -X POST localhost:8010/step -H 'content-type: application/json' \
-d '{"action":{"message":"hello"}}'
{"observation":{"metadata":{},"echoed":"hello"},"reward":null,"done":false,"metadata":null}
$ kill %1
That is a complete, conforming OpenEnv server: all sixteen routes, /schema,
WebSocket sessions, the lot. Hugging Face's own openenv validate gives it 6/6.
(The default port is 8000, the container contract; 8010 here only because this
machine had 8000 taken.)
The shape of what you just built:
┌────────────────────────────────────────────────────────────────┐
│ your trainer · your agent · curl — anything that speaks HTTP │
└──────────────────────────────┬─────────────────────────────────┘
│
┌─ this repository ────────────▼─────────────────────────────────┐
│ │
│ src/server/ · src/client/ the I/O hosts │
│ a poll() loop and a dialer — the only code that │
│ owns a socket │
│ ──────────────────────────────────────────────────────── │
│ src/proto/ THE PROTOCOL │
│ 16 routes · 7 message types · one 4-key envelope · │
│ 7 error codes — four pure functions: no socket, │
│ no clock, no global state │
│ ──────────────────────────────────────────────────────── │
│ src/wire/ plumbing (not OpenEnv) │
│ HTTP/1.1 · RFC 6455 · JSON · UTF-8 · SHA-1 · arena — │
│ forbidden to contain a single OpenEnv word │
│ │
│ make check fails the build if either boundary breaks │
└────────────────────────────────────────────────────────────────┘
That middle layer is the entire answer to "what is OpenEnv, exactly?" It is four pure functions, so you can embed it with no sockets at all. And the test suite can replay recorded reference traffic through it byte-for-byte, in about a millisecond.
The protocol, entire
A conforming OpenEnv server is an HTTP/1.1 server that answers sixteen routes. Every
status code below was produced by envs/counter.c while writing this table.
| # | Method | Path | Answers |
|---|---|---|---|
| 1 | GET | /openapi.json | 200 — info.version selects the conformance profile |
| 2 | GET | /list_environments | 200 ["counter_env"] |
| 3 | GET | /{env}/splits | 501 — the Task API; every shipped env declines it |
| 4–7 | POST | /{env}/tasks, …/num_tasks, …/task, …/task_range | 501, or 422 on a missing field |
| 8 | WS | /mcp | bare JSON-RPC, no envelope, spaced separators (WART-13) |
| 9 | POST | /reset | 200 — stateless: builds an environment, runs it, destroys it |
| 10 | POST | /step | 200 — stateless too; the action arrives wrapped under "action" |
| 11 | GET | /state | 200 — truncated to the two base fields (WART-05) |
| 12 | GET | /metadata | 200 — six keys, fixed order |
| 13 | GET | /health | 200 {"status":"healthy"} — the container readiness contract |
| 14 | GET | /schema | 200 — JSON Schema for action and observation; the only typing channel |
| 15 | POST | /mcp | 200 JSON-RPC 2.0, four methods — the agent door: tools/list, tools/call, and two session methods we decline |
| 16 | WS | /ws | the episode — the only route where one can exist |
Over /ws: text frames, one JSON object each, strictly lock-step. One response per
request, no correlation ids, nothing server-initiated.
// CLIENT -> SERVER (five) // SERVER -> CLIENT (four)
{"type":"reset" [,"data":{…}]} {"type":"observation","data":{…}}
{"type":"step" , "data":{…}} {"type":"state" ,"data":{…}}
{"type":"state"} {"type":"error" ,"data":{…}}
{"type":"close"} {"type":"mcp" ,"data":{…}}
{"type":"mcp" , "data":{…}}
Every observation is this envelope, and it is the whole data model:
{"observation": {…env-specific…}, "reward": 3, "done": false, "metadata": {…}}
reward is bool | int | float | null. There is exactly one done, and no
Gymnasium terminated/truncated pair. Errors are seven strings, a closed set:
INVALID_JSON UNKNOWN_TYPE VALIDATION_ERROR EXECUTION_ERROR <- socket stays open
CAPACITY_REACHED FACTORY_ERROR SESSION_ERROR <- socket closes
That is the protocol. All of it. The route table is
src/proto/routes.c ↗; the message and error vocabulary is
src/proto/types.h ↗, a file that is 80% comment because the
comment is the specification. Our full prose statement, with twenty-three numbered
server requirements and twenty client ones, is docs/PROTOCOL.md.
Three lessons from the wire
The stateless surprise
The route table marked /reset and /step stateless, and that word is doing
more work than it appears to. make demo-stateless — real output, trimmed of its
commentary and cut to the two steps:
$ curl -s -X POST http://127.0.0.1:8990/step -H 'content-type: application/json' -d '{"action":{"amount":2}}'
{"observation":{"metadata":{"steps":1},"total":2,"last_note":""},"reward":2.0,"done":false,"metadata":{"steps":1}}
$ curl -s -X POST http://127.0.0.1:8990/step -H 'content-type: application/json' -d '{"action":{"amount":2}}'
{"observation":{"metadata":{"steps":1},"total":2,"last_note":""},"reward":2.0,"done":false,"metadata":{"steps":1}}
A counter, stepped twice: 2, then 2 again. Nothing is broken. Each HTTP call constructed a fresh environment, ran one step, and destroyed it; there is nowhere for the second step to accumulate. Now the same two steps over the WebSocket:
$ build/rel/bin/oe_probe http://127.0.0.1:8990 --episode --steps 2 --action '{"amount":2}'
>> {"type":"step","data":{"amount":2}}
<< {"type":"observation","data":{"observation":{"metadata":{"steps":1},"total":2,…},"reward":2.0,"done":false,…}}
>> {"type":"step","data":{"amount":2}}
<< {"type":"observation","data":{"observation":{"metadata":{"steps":2},"total":4,…},"reward":4.0,"done":true,…}}
2, then 4. One connection is one session is one environment instance; WS /ws
is the only transport in OpenEnv where an episode can exist. From inside the Python
API that fact is invisible. For anyone writing a non-Python client, it is the first
thing to know. And the demo asserts its own punchline: the two HTTP bodies must be
byte-identical and the two WS observations must differ, or
tools/demo.sh ↗ exits 1. A demo that cannot fail is a screenshot.
(Two more lessons hide in those transcripts: the action is wrapped under "action" on
HTTP but bare on WS, and HTTP emits "metadata":null where WS omits the key. Same
environment, same values, two encodings — each marked at the line of
src/proto/encode.c ↗ that honours it.)
The two doors, and what only one of them counts
OpenEnv's architecture claims a dual API boundary: a trainer door where an
episode happens, and an agent door where tools are called. Both reach the same
environment instance, and they disagree about what a step is. One WebSocket, one
instance of envs/echo_tools.c ↗, trimmed to the five frames that
carry the point:
state (fresh) <- {"episode_id":"ep-fixed","step_count":0}
step call_tool <- {"type":"observation","data":{"observation":{"metadata":{},"tool_name":…
state after step call_tool <- {"episode_id":"ep-fixed","step_count":1}
mcp envelope tools/call <- {"type":"mcp","data":{"jsonrpc":"2.0","id":9,"result":"Hello!"}}
state after mcp call <- {"episode_id":"ep-fixed","step_count":1} <- unchanged
Same tool, same instance, same socket. Through the trainer door a tool call is a
step: the counter moves and the answer is an observation with somewhere to put
reward and done. Through the agent door it is not: the answer is a bare JSON-RPC
result with nowhere to put either. A handler reads one bool, out->stepwise, to know
which door it arrived at, and tests/t_replay.c asserts both halves — the counter
that moved exactly once, and the bool that flipped. Drive the full eleven-frame
version against the reference's echo_env. Its five state readings are
0, 1, 2, 2, 2; ours are 0, 0, 1, 1, 1. There, asking what tools exist is
itself a step
(divergence 7)
— the one place the tool vocabulary makes a trainer's number differ.
The doors differ in one more way, and it is a correctness rule rather than a style
choice: a tool's argument schema is input_schema on the trainer door and
inputSchema on the agent door, because the reference's own agent-side converter
reads only the second spelling and tells a model "no arguments" when it is fed the
first.
Every byte, annotated
examples/oe_probe.c ↗ prints the wire itself — frame headers in
hex, then the JSON, then the rule each field obeys. It links none of our protocol
code, so what scrolls past is the wire and not our model of it. Two excerpts from a
real run (~ lines are annotation, never bytes):
>> Sec-WebSocket-Key: E0HQzuZW5B2a1ioFXy3tdQ==
<< HTTP/1.1 101 Switching Protocols
<< Sec-WebSocket-Accept: Odi9htUlIQLqoQ5iDQR8xeXl+G0=
~ we computed base64(SHA1(key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"))
~ = Odi9htUlIQLqoQ5iDQR8xeXl+G0=
~ they match. From the next byte on, this socket carries frames.
~ this proves the peer read the specification and nothing else. It
~ authenticates nobody: the key is a nonce, not a secret.
--- frame 1 -------------------------------------------------
>> 81 9a 45 4f a8 63 (6 bytes: the frame header)
~ byte 0 = 0x81 FIN=1 RSV=000 opcode=0x1 (TEXT)
~ byte 1 = 0x9a MASK=1 length field = 26
~ a client MUST mask every frame it sends and a server MUST NOT mask any
~ (RFC 6455 §5.1). Same protocol, opposite obligations, decided entirely
~ by which end of the socket you are.
Point it at your own server — including a Python one you are debugging. It also has
--health, --schema and --metadata modes, and it ends every episode by watching
the closing handshake.
Does it actually interoperate?
An oracle, here, is an independent judge — usually upstream's own code. Sixteen oracles grade this implementation: one recorded corpus and fifteen live checks. The judge in every live one is somebody else's code.
Oracle zero is the corpus: 103 cases (118 exchanges) recorded off six real reference servers and checked in. The replay drives them through the four pure functions with no socket, no clock and no Python:
$ ./build/rel/tests/t_replay
REPLAYED 103 exchanges, byte for byte
EXPECTED-DIVERGENT 15 exchanges
REFUSED 14 bytes we will not emit, on evidence
UNREPRODUCIBLE 1 a Python library's artefact, not ours to forge
NOT-YET 0 implementable, and not yet built
OUT-OF-SCOPE 0 exchanges
103 cases -> 118 exchanges: 88 replay whole, 14 do not, 1 split
ok t_replay: 437 checks, 0 failed
Every non-replayed exchange is printed by name with its reason, and each reason names the exact field. The test asserts all the counts, so an exchange the classifier ever quietly dropped is a failure rather than a shrinking number nobody watches. The verdict is per EXCHANGE and not per case, which is why one case is split: three of corpus case 74's four WebSocket frames replay byte for byte and the fourth prints the name of the Python dataclass it declines to be.
The fifteen exchanges above — the 14 REFUSED plus the 1 UNREPRODUCIBLE — fall
into four groups. Two are /openapi.json documents we deliberately answer
minimally, one of them the production-mode document with ten paths, not
thirteen. Six are MCP rows where the reference's answer is a Python library's
serialisation rather than OpenEnv's. One is a tool-door row: the undeclared
keyword argument CPython refuses for a tool, which our agent door now refuses
too but cannot spell in the interpreter's own words. And six are task-API rows
where the reference env declares a task provider and we answer 501, the
conforming answer for an env without one — declined on evidence, because no
upstream client consumes those routes (GAP-24). Three earlier entries — the
step door's extra key, the tool action model our GET /schema publishes, and
the three production-mode 404s — all REPLAY now, as items 7, 8 and 9 landed.
Each divergence also lands in one of three verdicts — refused on evidence,
unreproducible without being a Python library, or not yet built — the three
subtotals under the banner above. The not-yet count is now zero, and asserted
so: the conformance arc that closed it means every divergence that remains is
a decision on evidence or a Python library's own bytes, not a to-do parked
where nobody watches it.
But a corpus can only answer the questions it was asked at capture time. So we also
ask live — sh interop/setup.sh once to build the reference peer, then:
$ make interop
ok oracle 1: reference validator — 6/6 required criteria
ok oracle 2a: upstream EnvClient drove envs/counter.c to done
ok oracle 2b: upstream EnvClient read bool/int/float rewards back as themselves
ok oracle 2c: our frame cap costs the real client a 300 KB action, exactly as limits.h says
ok oracle 2d: upstream EnvClient drove tour/01_minimal.c to done — the two-endpoint claim
ok oracle 2e: the genuine MCPToolClient listed and called two tools on envs/echo_tools.c
ok oracle 3a: examples/oe_agent drove the real Python server to done
ok oracle 3b: same client, both servers — 5 replies identical byte for byte
ok oracle 3c: tour/03_client.c drove the real Python server to done — the hand-written client
ok oracle 3d: examples/oe_agent (library) and tour/03_client (wire) sent 3 identical client frames byte for byte
ok oracle 4: the official mcp SDK dies the same way on all three servers — McpError: Method not found: initialize
ok oracle 5: 230 lines of stdlib Python satisfy the genuine MCPToolClient on both doors
ok oracle 6: verifiers's taskset predicate decides envs/echo_tools.c HAS tools
ok oracle 7: time-to-EOF 0.112 ms here / 0.313 ms at the reference (both sub-linger; ours was ~1001 ms before the half-close) — one socket carries one exchange here, a second at the reference (keep-alive): the lifetime divergence the corpus never captured
interop.sh: every oracle passed
Oracle 1 is openenv validate --url, the reference project's own conformance suite:
the only judge here we did not write a line of. Oracle 2 is the genuine
openenv.core.env_client.EnvClient, with its own WebSocket library and its own
pydantic models, driving our C server to done. Oracle 2e is a different upstream
client, MCPToolClient: it reads a tool table out of a C environment into a pydantic
model that forbids a fourth key. Oracle 4 is the only one here where passing means a
peer FAILED. The official Model Context Protocol SDK cannot handshake with the
reference implementation either, and the two servers refuse it with the same sentence.
Oracle 3 is the direction that matters most: our client in front of a server we did not write. A client tested only against its own server is a mirror, not a measurement. Its strongest form points the same C client at the Python server and at ours, and diffs every payload:
$ diff wire-python.txt wire-c.txt
$ echo $?
0
An empty diff. Two servers sharing no line of code, indistinguishable to a client that reads bytes: the only definition of protocol compatibility that means anything.
And each oracle was made to fail on purpose
Three deliberate mutations, each the obvious simplification a maintainer would reach for in six months, each caught by a different oracle:
| The mutation | Caught by | What it reported |
|---|---|---|
| Make WS coerce rewards to float, as HTTP does | the genuine Python client | FAIL bool_true: got 1.0 (float), want True (bool) |
Make WS always emit "metadata", as HTTP does | the byte-diff | one line differed — and every live client stayed green |
Answer /health with {"status":"ok"} | the reference validator | 5/6, health_endpoint failed — and every live client stayed green |
The last two changes were each visible to exactly one oracle, and real clients felt neither. No single judge sees everything; that is why there are fifteen live oracles, and the corpus underneath them.
How big is it, really?
Don't take a paragraph's word for it. The measurement is a build target, and
make check fails if this transcript drifts more than ten percent from what the
target actually prints:
$ make lines
GROUP / DIRECTORY CODE TOTAL
-----------------------------------------------
proto (THE PROTOCOL) 4516 10990
-----------------------------------------------
wire (RFC plumbing) 4771 9681
server (I/O) 2777 5300
client (I/O) 944 1651
-----------------------------------------------
envs (YOURS) 294 756
tour 815 1407
of which: THE PROTOCOL 64
of which: the socket 177
of which: neither half 13
the Python: PROTOCOL 78
the Python: the socket 0*
03_client: THE PROTOCOL 73
03_client: the socket 391
03_client: neither half 20
02_conformant: WHOLE FILE 77
examples 1651 2274
tests 16143 25273
include (public API) 271 1219
OpenEnv's own test suite contains the proof that OpenEnv is small. It is only
hard to notice from inside 164,342 lines. The proof is the tour block above.
tour/01_minimal.c ↗ serves a real episode to the genuine Python
client, with the JSON written as string literals and no protocol library at all.
The file is fenced into a protocol half and a socket half, so the target can count
them. 64 statements of protocol, 177 of socket. The 78 beside it is the
reference's own minimal server, 109 lines of Python whose docstring says it
"implements exactly what GenericEnvClient (simulation mode) needs". Counted
symmetrically, the two sides are nearly a dead heat. The claim that survives:
the protocol is tiny in both languages, and everything else is plumbing somebody
wrote once. The 4,771 lines of wire/ are that plumbing (RFC 6455, RFC 8259,
HTTP/1.1, UTF-8): work that import fastapi hides, and none of it OpenEnv.
tour/02_conformant.c ↗ closes the loop: the same environment
written with the library is 77 statements, more than the 64 hand-written ones.
A library does not make the two-endpoint case smaller; it makes the other fourteen
routes free.
tour/03_client.c ↗ measures the other chair. 01_minimal.c is
the protocol from the server's side; this is the client's — the twenty 8.B MUSTs a
program driving an environment must keep — hand-written with wire/ and nothing from
src/proto/, fenced and counted the same way: 73 statements of client protocol
beside the server's 64. It drives the genuine Python reference to done and sends
byte-for-byte the same client frames the library client does (make interop
oracles 3c and 3d). Now the boundary is measured from both chairs, not one.
(Counting rules, for the wary: CODE excludes comments and blanks, TOTAL does not;
a * marks a figure recorded rather than measured; on a fresh clone, before
sh interop/setup.sh fetches the reference, both Python rows carry one.)
What OpenEnv is not
Half the lesson, and the half you cannot get from inside a Python API. Every claim
below was established by reading captured traffic —
docs/NOT-THE-PROTOCOL.md has the observed bytes for each.
The strongest single piece of evidence is a header census: across all
78 captured HTTP responses, seven distinct header names appear, and that is the
complete set: allow, connection, content-length, content-type, date,
location, server.
- No authentication, no CORS, no TLS. Isolation is the container's job.
- No protocol version on the wire. Nothing in any header or payload names one.
- No action/observation spaces, no legal-move masks, no turn-taking, no player
identity. Connect4's
legal_actionsupstream is an ordinary environment field; nothing reads it. - No episode state machine.
stepbeforeresetis accepted.stepafterdone:trueis accepted, indefinitely. Sequencing is client-side convention. - No session ids, no correlation ids, no streaming, no server-initiated frames.
- Rubrics, evals, harnesses, trainers, the CLI,
openenv.yaml, Hub integration, container providers, the web UI — zero wire footprint, all of it. By volume that is most of the reference repository, and none of it reaches the socket.
Where the reference's behaviour is real but plainly accidental, we reproduce it
faithfully and mark the line with a WART comment: twenty-eight of them, each in a
fixed claim / reference / observed / honoured-by form, indexed in
docs/WARTS.md, which is generated from those markers so it
cannot drift from the code. The flagship three: HTTP and WS encode the same
observation differently; GET /state silently drops your subclass fields; HTTP
/reset and /step are stateless.
And where the protocol turns out not to exist at all (nobody specified it; two
implementations can disagree and both be right), that is docs/GAPS.md,
also generated, and the most useful document here for anyone building a third
implementation. Twenty-two numbered entries, twenty open, two retired with their numbers
kept as holes so citations never silently change meaning.
Write an environment (45 statements)
This is envs/echo.c ↗ with comments removed — 45 statements in a
113-line file, where the rest of the file is explanation:
#include <openenv/openenv.h>
static const oe_field ECHO_ACTION[] = {
{ "message", OE_F_STR, NULL, "The message to echo back" },
OE_FIELDS_END
};
static const oe_field ECHO_OBS[] = {
{ "echoed", OE_F_STR, "\"\"", NULL },
OE_FIELDS_END
};
typedef struct { int64_t n; } echo;
static oe_err echo_reset(void *self, const oe_json *params, oe_obs *out)
{
(void) params;
((echo *) self)->n = 0;
oe_obj_set_str(out->fields, "echoed", "");
return OE_OK;
}
static oe_err echo_step(void *self, const oe_json *action, oe_obs *out)
{
((echo *) self)->n++;
oe_obj_set_str(out->fields, "echoed", oe_obj_get_str(action, "message", ""));
return OE_OK;
}
static oe_err echo_state(void *self, oe_json *out)
{
oe_obj_set_null(out, "episode_id");
oe_obj_set_int(out, "step_count", ((echo *) self)->n);
return OE_OK;
}
static const oe_env ECHO = {
.name = "EchoEnvironment",
.description = "EchoEnvironment environment",
.action_fields = ECHO_ACTION,
.observation_fields = ECHO_OBS,
.state_size = sizeof(echo),
.reset = echo_reset,
.step = echo_step,
.state = echo_state
};
int main(int argc, char **argv) { return oe_main(argc, argv, &ECHO); }
No allocator. No ownership question. No error branch. No null check. The server
allocates and zeroes state_size bytes per session and hands them to you as self;
oe_obj_set_* cannot fail from your side; every getter takes a fallback. And that is
not a style aspiration: make check greps this file for the arena type and the two C
allocator functions and fails the build if any appears.
Two field tables, one struct, three callbacks, one main. It buys all sixteen
routes, /schema, /openapi.json, eight concurrent sessions, RFC 6455, and a 6/6
from the reference's own validator. envs/README.md walks the two
bigger examples: counter.c adds reward and done; rewards.c makes the
HTTP-vs-WS reward divergence visible from a shell.
Embedding it, and shipping it
The protocol with no sockets in the binary
oe_err oe_session_frame(oe_session *s, const char *in_json, size_t in_len,
oe_buf *out, bool *should_close);
oe_err oe_ws_mcp_frame(const oe_env *env, const char *in_json, size_t in_len,
oe_buf *out);
oe_err oe_http_handle(const oe_env *env, const char *req, size_t req_len,
oe_arena *msg_arena, oe_buf *out);
Those three functions — the episode, the bare JSON-RPC of WS /mcp, and every HTTP
route — contain every byte OpenEnv will ever put on a wire, and they open no socket,
read no clock, own no global. Checkable, not asserted: make check-embed
builds a program that includes only include/openenv/embed.h ↗,
links the whole archive, and comes out with fifteen undefined symbols — malloc,
memcpy, snprintf, strtod and friends. No socket, no poll, no signal,
no clock. That is the entire operating-system dependency of the OpenEnv protocol,
and it is why the corpus replay needs no network and finishes in two milliseconds.
make check-embed also proves the other two consumption routes: pkg-config against
a real make install, and vendoring the sources under a deliberately hostile flag set
(-std=gnu17 -Werror -D_GNU_SOURCE -funsigned-char …) — then runs both and diffs
protocol bytes. docs/PACKAGING.md has all three in full.
The container contract is two sentences
Listen on port 8000, bound
0.0.0.0. AnswerGET /health.
make docker builds a two-stage image (Alpine builder, FROM scratch final) and
then proves the contract against the running container:
$ make docker
ok GET /health answers from outside — so it bound 0.0.0.0, not loopback
ok HEALTHCHECK reports healthy — the binary probed itself, no curl in the image
ok docker stop returned in 1s (the grace period is 10)
miniopenenv:counter 210632 bytes
ghcr.io/huggingface/openenv-base:latest 467705200 bytes
(base image e96f47a4f0b9, pulled July 2026)
ratio: 2220x
The honest version of that comparison: openenv-base is a base image carrying
CPython and every environment's shared dependencies, ours is a finished server with
one environment compiled in. Nobody did anything wrong. The claim is only this:
the smallest image that can answer /health and run an episode is 211 kB, and
468 MB is where you start when the answer is written in Python.
Limitations
Stated plainly, because a teaching repository that overstates itself teaches the wrong lesson twice.
- Single-threaded. One
poll()loop; a slowstep()blocks every session and/healthwith it. Why workers would make the server lie about itself on the wire is argued insrc/server/README.md. - No TLS, server or client. It is refused loudly at the call that names
https://, never silently downgraded, andoe_client_url_verdicttells a caller "TLS unsupported" apart from "server down". Put a reverse proxy in front, which is exactly what the reference's own Spaces provider does: the two one-liners, a transcript of a whole episode crossing TLS, and the test that pins the seam aredocs/PACKAGING.mdsection 5. - MCP tool tables are implemented, one shape per door, and that is all they
are:
name,description, and an argument schema spelledinput_schemaon the trainer door andinputSchemaon the agent door — the reference's own FastMCP-free branches, three keys on both. What we decline is one library's exhaust: a nine-keyTooldump whosemetashould be_meta, aCallToolResultenvelope no client reads, and the MCP specification's lifecycle.docs/NOT-THE-PROTOCOL.mdlists every refused key with the line that produces it. - We are not an MCP server. No
initialize, no session, no SSE, no resources or prompts. Neither is the reference: three genuine MCP clients — including fastmcp's own — cannot complete a handshake against it, both servers refuse them with the same sentence, andinterop/has that as a test. - Production mode is served as a parameter; simulation is the default.
oe_serve_opts.production(and theproductionargument ofoe_http_handle) selects the reference'sServerMode.PRODUCTION, which removes/reset,/stepand/state— a request to one is the generic 404, answered before the method check, and/openapi.jsonomits the three. It is a parameter and never a field onoe_envor a global, so two servers in one process may differ. WS/wsis untouched. Corpus cases 43-45 (port 8903) replay byte for byte in this mode; nothing in the reference's shipped entry points reaches it, but the behaviour is real when you ask for it. GET /openapi.jsonis deliberately minimal where the reference emits 17 kB of FastAPI prose; the validator accepts both.- Close-frame reason strings are our prose, not the reference's; close codes match everywhere but one documented case. Byte-exactness claims are scoped to observation, state and error payloads, never blanket.
- A client reads at most 512 KiB per message. The cap is derived, not guessed: a
parse tree is not the size of its source text, the worst measured JSON shape costs
128.23 bytes of arena per byte of input, and three static assertions keep the
constants from drifting apart.
atari_env's 403 kB observation arrives with a 23% margin. The full story, including how the first measurement was wrong, is indocs/SECURITY.mdand beside the constants insrc/wire/limits.h↗. - A message's arena budget is a flat ceiling, not a function of the message. The
obvious refinement was built, measured, and rejected: the same arena holds the
reply, and a 41-byte step can be answered by a 400 kB observation.
docs/MEMORY.mdhas the numbers. - 216 of 517 Autobahn cases need permessage-deflate, which we do not negotiate
(RFC 7692, not 6455):
258 OK, 259 expected exceptions, 0 unexpected— and an unexpected pass fails the run just as an unexpected failure does.
The fuzzers found two real bugs during development; both are fixed and kept as
regression inputs. docs/FUZZING.md has them, plus the three
places the Autobahn oracle was wrong and the code was right.
Going deeper
| You want | Read |
|---|---|
| the whole thing, guided | docs/TOUR.md — sixteen steps, about an hour |
| the protocol as prose | docs/PROTOCOL.md — 23 server + 20 client requirements |
| what is deliberately absent | docs/NOT-THE-PROTOCOL.md |
| where the spec runs out | docs/GAPS.md (generated) |
| the reference's accidents, honoured | docs/WARTS.md (generated) |
| the protocol/library boundary, per symbol | docs/BOUNDARY.md (generated) |
| the memory model in one rule | docs/MEMORY.md |
| threat model and every cap | docs/SECURITY.md |
| install, vendor, embed, containerize | docs/PACKAGING.md |
| porting off POSIX | docs/PORTING.md — rewrite one file |
| games, synthesis tasks, and eight math environments | arcade/README.md — built on the protocol, not part of the teaching core |
| training a small model against them | docs/TRAINING.md — three walkthroughs: TRL, prime-rl, Tinker, each one an adapter this repository ships |
The five-minute version: tour/01_minimal.c ↗ →
include/openenv/openenv.h ↗ → envs/echo.c ↗
→ src/proto/types.h ↗ → src/proto/encode.c ↗,
where the two observation encoders sit side by side with the divergence table between
them. And cat src/*/README.md is a sixty-second architecture tour on its own — each
opens with a claim you can check.
Build targets, each of which knows how to fail:
make / make test | library + 59 programs; 11,200 checks, run twice — as built, then under ASan+UBSan — and make test asserts that number against this line |
make check | 52 structural rules: layer boundaries, banned functions, doc staleness, link rot |
make demo-stateless | the four-curl lesson above, asserted |
make demo-two-readers | one recorded observation read the right way and the plausible-but-wrong way, plus the corpus census under it (MUST 8.B.12) |
make interop | the fifteen live oracles (sh interop/setup.sh first; skips loudly without it) |
make lines / make docs | the size ledger; regenerate the three generated docs |
make arcade / make train-snake / make task-dyna | all twenty-two arcade servers and their three clients; the train- and task- targets drive the recorded fourteen-server gallery over a real WebSocket. make record-arcade re-runs that gallery into arcade/recordings.txt |
make arcade-up / make adapters | start all fourteen and leave them running, with a manifest naming where each one is; then drive the shipped training adapters against them with no GPU and no model. make arcade-down stops exactly those |
make math / make math-measure / make record-math | eight text-first math servers and mathlab, the independent client that solves from the prompt, rejects a wrong answer, repeats the seed, and checks terminal freezing |
make math-up / make math-down | leave the eight math servers running with their own manifest and pidfile, or stop exactly that group |
make fuzz-run / make autobahn | the fuzzers; 517 RFC 6455 conformance cases |
make install / make check-embed / make docker | docs/PACKAGING.md |
Provenance
This is a cleanroom implementation. No upstream code is vendored here.
The twenty-four .py files under interop/ ↗ are ours. Fourteen
import the reference rather than copying it, one imports the official mcp
SDK, and nine import neither — eight of those nothing but the standard library,
the ninth a training framework. tools/mutate.py, tools/site.py and
tools/synthlab_parity.py import neither as well, and the last of the three is
the twenty-seventh .py in the tree.
The protocol was re-derived by booting the reference implementation
and recording what came out of the socket: 103 byte-exact cases — every HTTP header,
every WebSocket frame, the handshake included — captured in July 2026 at reference
revision 60ecc7b7 and checked in at
tests/fixtures/reference-corpus.json ↗, with
the revision recorded inside the corpus so a fixture can never drift from the claim
about where it came from.
Every factual claim in this codebase is one of four kinds, and the source tells you
which: captured (corpus case N), probed (recorded in
docs/evidence/), read (a file.py:line
citation into the reference), or ours (a GAP-nn marker at a line that had to
choose). A wart backed by captured bytes is a fact about the protocol; one backed by
reading Python is a fact about a single implementation, and a third implementer
deserves to know which is which. There is one row of the divergence table in
src/proto/encode.c ↗ that says outright NOTHING pins this,
because no captured byte ever has.
docs/PROVENANCE.md is the full story, including what it took
to keep "no upstream code" a checkable claim rather than a slogan.
The project is complete against that reference revision. The gaps ledger is the roadmap the protocol itself suggests; issues and byte-level counterexamples are the most valuable thing you can send.
License
Apache-2.0. See LICENSE ↗.
OpenEnv is a project of Hugging Face and its contributors, licensed BSD-3-Clause; this repository is independent, unaffiliated, and reimplements the protocol from observed behaviour. If you want the ecosystem — environments on the Hub, trainers, rubrics, the CLI — you want the reference ↗. If you want to understand what all of that is built on, you are in the right place.