One episode, annotated
One recorded session with an OpenEnv server, explained line by line. First the handshake that opens the connection, then one complete episode frame by frame, then the close. Nothing here is a diagram of the protocol — it is the bytes.
OpenEnv is Hugging Face's standard for running reinforcement-learning
environments as services. A client resets an environment, then steps it with
actions; each step returns an observation and a reward. An episode
is one run of that loop: a reset, then steps, until done.
OpenEnv speaks two transports: single HTTP calls, and a WebSocket connection
that stays open and carries a whole episode. Below, the
reference means
huggingface/OpenEnv ↗,
the Python implementation this repository was captured from.
What follows is a single session against one small environment, printed by
examples/oe_probe.c ↗
and reproduced here unedited. The probe links none of this repository's
protocol code: only sockets, HTTP/1.1, RFC 6455 (the WebSocket
specification) and JSON. So what scrolls past is the wire: the
bytes actually exchanged on the socket, not our model of them. The probe
speaks to any OpenEnv server, including a pure-Python one you are
debugging: build/rel/bin/oe_probe http://your-server --episode.
Nine plates follow, in session order. A plate is a numbered
block of transcript, quoted verbatim from the one recording behind this
page,
site/wire.txt ↗.
The quoting is enforced: make site exits nonzero if a line
printed here is not a line of that file. The ~ annotation lines
are held to the same rule, because they are the probe's words and not this
page's. The prose between the plates can be wrong; the bytes cannot be
invented.
Run the two commands in plate 1 yourself and you will get the same session back. Only the random values will differ — the key in the handshake, the accept hash computed from it, and the masking key on each client frame, each explained where it first appears. Nothing else will.
Three marks, printed by the probe itself, carry the whole transcript:
- >>
- bytes this program sent
- <<
- bytes the server sent
- ~
- annotation: the probe's prose, never bytes
The session
Plate 1 is the setup. The first command starts
envs/counter.c ↗
on port 8990: a deliberately small environment whose every step adds
amount to a running total. The second command points the probe
at it and asks for one episode of two steps. Both programs write to the same
terminal, so their lines interleave: the server's session
opened lands in the middle of the probe's banner. Two processes, one
terminal, and no editing afterwards.
$ build/rel/bin/counter --port 8990 &
openenv info listening on: 0.0.0.0:8990 max_sessions=8 max_connections=64
$ build/rel/bin/oe_probe http://127.0.0.1:8990 --episode --steps 2 --action '{"amount":2}'
openenv info session opened
oe_probe — http://127.0.0.1:8990
>> is what this program sent. << is what the server sent.
~ is annotation: our prose, never bytes.
It uses no protocol code of ours — only sockets, HTTP, RFC 6455 and
JSON — so what scrolls past is the wire itself.
The opening handshake
A WebSocket connection begins as an ordinary HTTP request. The client
asks the server to switch this socket away from HTTP; the server agrees with
status 101. That request and reply are the opening
handshake, and they are the last HTTP this socket will speak. Four
request headers do all of it. The reply that matters is one line and one
header: the status, and Sec-WebSocket-Accept, a hash of the
client's key that proves the server read the same specification. After the
reply's blank line, every byte on this socket is WebSocket.
=== WS /ws — the opening handshake =========================================
>> GET /ws HTTP/1.1
>> Host: 127.0.0.1:8990
>> Upgrade: websocket
>> Connection: Upgrade
>> Sec-WebSocket-Key: OjHPcY9sdu0yulKgj4hevQ==
>> Sec-WebSocket-Version: 13
>>
~ an HTTP request whose purpose is to stop speaking HTTP. Four headers do
~ all of it: Upgrade, Connection, Sec-WebSocket-Key, -Version: 13.
~ Sec-WebSocket-Key is 16 bytes from the kernel, base64-encoded. Not a
~ password, not a session id, and never reused.
~ No Sec-WebSocket-Protocol and no Sec-WebSocket-Extensions are offered.
~ Nothing is negotiated, so nothing can be negotiated away from under
~ the frame parser — which is why this program links no zlib.
<< HTTP/1.1 101 Switching Protocols
<< Upgrade: websocket
<< Connection: Upgrade
<< Sec-WebSocket-Accept: eUGvrMg/HJ6oWYLS4pILy0daWS8=
<<
~ we computed base64(SHA1(key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"))
~ = eUGvrMg/HJ6oWYLS4pILy0daWS8=
~ the server said
~ = eUGvrMg/HJ6oWYLS4pILy0daWS8=
~ they match. From the next byte on, this socket carries frames.
~ this proves the peer read the specification and nothing else. It
~ authenticates nobody: the key is a nonce, not a secret.
Sec-WebSocket-Accept is
base64(SHA1(key + GUID)) — a proof that the peer read RFC 6455
and nothing more. Our SHA-1 is
src/wire/sha1.c ↗,
and it exists for this one line.
— site/wire.txtFrame 1 — reset
From here on, every message travels in a frame: a short binary header, then the payload, which is the message itself. The header says what kind of payload follows (the opcode), how long it is, and whether it is masked. Frame 1 spends six bytes on the header and twenty-six on the payload, and the annotations read the header back a bit at a time.
Masking is the one rule that differs by direction. A client XORs
every payload byte with four fresh random bytes called the masking key,
carried in the header. A server sends its payload bare. RFC 6455 requires
both, and that is why a captured client frame looks like noise while a
captured server frame is readable text. The message inside this frame is
reset: start an episode, and send back the first
observation.
=== WS /ws — the episode ===================================================
~ strictly lock-step from here: one frame out, exactly one frame back.
~ Nothing is pipelined and nothing arrives unbidden.
--- frame 1 -------------------------------------------------
>> 81 9a 02 44 34 97 (6 bytes: the frame header)
~ byte 0 = 0x81 FIN=1 RSV=000 opcode=0x1 (TEXT)
~ byte 1 = 0x9a MASK=1 length field = 26
~ payload = 26 bytes, carried in the length field itself
~ the 7-bit form: a length under 126 is the length field itself, and no
~ further bytes were needed to say so.
~ masking key = 02 44 34 97
~ the four mask bytes: every payload byte on the wire is XORed with them,
~ cycling every 4. That is why a capture of a client frame looks like
~ noise and a capture of a server frame is readable text.
~ a client MUST mask every frame it sends and a server MUST NOT mask any
~ (RFC 6455 §5.1). Same protocol, opposite obligations, decided entirely
~ by which end of the socket you are. The key is fresh for every frame.
>> {"type":"reset","data":{}}
~ the payload as we wrote it. On the wire every byte of it is XORed with
~ the four-byte key above, which is why a capture looks like noise.
~ `data` is optional on reset and its contents are NOT validated: the
~ reference forwards them as keyword arguments and silently drops whatever
~ does not fit reset()'s signature. Over HTTP the same object is a 422.
<< 81 76 (2 bytes: the frame header)
~ byte 0 = 0x81 FIN=1 RSV=000 opcode=0x1 (TEXT)
~ byte 1 = 0x76 MASK=0 length field = 118
~ payload = 118 bytes, carried in the length field itself
~ MASK=0, as a server frame must be. A masked frame from a server is a
~ protocol error and closes the socket with 1002.
<< {"type":"observation","data":{"observation":{"metadata":{},"total":0,"last_note":"reset"},"reward":null,"done":false}}
~ two keys, always in this order: `type`, then `data`. There is no id, no
~ sequence number and no correlation field — the protocol is lock-step,
~ so the answer to your last frame is the next frame you read.
~ the four-key envelope, and it is the whole payload: observation, reward,
~ done, metadata. No terminated/truncated pair, no info dict, no episode
~ object, no protocol version. The negative space teaches as much as the
~ fields do.
~ reward is null, and HTTP does not change that: POST /reset and
~ POST /step both send "reward":null too. WART-03 is about bool and
~ int, which HTTP coerces to floats; null is the one value the two
~ transports agree on.
~ `done` is at the TOP level of `data`, beside `observation` and not inside
~ it — and it is the ONLY terminal signal this protocol has: no
~ terminated/truncated pair, no info dict. `reward` sits beside it for the
~ same reason. Run --schema to see the trap the schema sets around both.
~ no top-level `metadata` key: it was empty, and this transport omits an
~ empty one. HTTP sends `"metadata":null` instead — same value, two
~ encodings. WART-04.
~ `metadata` also appears INSIDE observation, unconditionally. It is emitted
~ twice on purpose, and the nested copy is the one always present.
observation, reward, done,
metadata. Note what is absent: no id, no sequence
number, no protocol version, no terminated/truncated pair. The missing
top-level metadata is
WART-04, a reference accident this
implementation reproduces: HTTP would have sent
"metadata":null here. — site/wire.txtFrames 2 and 3 — the two steps
A step is one move of the loop: the client sends an action, and
the environment answers with an observation and a reward. The probe sends
the same action twice, {"amount":2}, and the total climbs — 2,
then 4. That climb is state surviving from one step to the next, and this
transport is the only one that has it.
Over HTTP there is no climb: each POST /step call builds an
environment, runs one step and destroys it, so the counter reads 2 both
times. Here one connection is one session is one environment instance. The
action's shape differs too. POST /step wraps the identical
object under an "action" key; on this transport the action is
bare.
--- frame 2 -------------------------------------------------
>> 81 a3 f1 5c 2e 3e (6 bytes: the frame header)
~ byte 0 = 0x81 FIN=1 RSV=000 opcode=0x1 (TEXT)
~ byte 1 = 0xa3 MASK=1 length field = 35
~ payload = 35 bytes, carried in the length field itself
~ masking key = f1 5c 2e 3e
>> {"type":"step","data":{"amount":2}}
~ the action is the BARE value of `data`. POST /step wraps the identical
~ object under an "action" key; this transport does not. That asymmetry is
~ the single thing a hand-written client gets wrong first.
<< 81 7e 00 90 (4 bytes: the frame header)
~ byte 0 = 0x81 FIN=1 RSV=000 opcode=0x1 (TEXT)
~ byte 1 = 0x7e MASK=0 length field = 126
~ payload = 144 bytes; 126 is the escape, so the real length is the next 2 bytes big-endian
~ 126 and 127 are escapes, not lengths: they say the real length follows in
~ two or eight big-endian bytes. RFC 6455 also requires the SHORTEST form
~ that fits, which is why wire/ws.c refuses a padded one with 1002.
<< {"type":"observation","data":{"observation":{"metadata":{"steps":1},"total":2,"last_note":""},"reward":2.0,"done":false,"metadata":{"steps":1}}}
~ reward is a float. On this transport that means the environment
~ produced a float; over HTTP it would mean nothing at all, because
~ HTTP coerces every reward to one.
src/wire/ws.c ↗
closes a padded one with 1002. — site/wire.txt--- frame 3 -------------------------------------------------
>> 81 a3 26 47 65 f2 (6 bytes: the frame header)
~ byte 0 = 0x81 FIN=1 RSV=000 opcode=0x1 (TEXT)
~ byte 1 = 0xa3 MASK=1 length field = 35
~ payload = 35 bytes, carried in the length field itself
~ masking key = 26 47 65 f2
>> {"type":"step","data":{"amount":2}}
<< 81 7e 00 8f (4 bytes: the frame header)
~ byte 0 = 0x81 FIN=1 RSV=000 opcode=0x1 (TEXT)
~ byte 1 = 0x7e MASK=0 length field = 126
~ payload = 143 bytes; 126 is the escape, so the real length is the next 2 bytes big-endian
<< {"type":"observation","data":{"observation":{"metadata":{"steps":2},"total":4,"last_note":""},"reward":4.0,"done":true,"metadata":{"steps":2}}}
~ done is true. There is no terminated/truncated pair in OpenEnv and no
~ episode object: this one boolean is the whole terminal signal.
tools/demo.sh ↗
exits 1 unless both facts hold. WS /ws is the only transport in
OpenEnv where an episode can exist.
— make demo-statelessFrame 4 — state
state asks the environment for its internal bookkeeping: the
fields it tracks that are not part of any observation. At sixteen bytes of
payload it is the shortest legal message in the protocol, because it carries
no data key at all. Sending an empty one is a
VALIDATION_ERROR.
--- frame 4 -------------------------------------------------
>> 81 90 77 70 ed 4b (6 bytes: the frame header)
~ byte 0 = 0x81 FIN=1 RSV=000 opcode=0x1 (TEXT)
~ byte 1 = 0x90 MASK=1 length field = 16
~ payload = 16 bytes, carried in the length field itself
~ masking key = 77 70 ed 4b
>> {"type":"state"}
~ `state` carries no `data` key at all. Sending "data":{} is a
~ VALIDATION_ERROR at loc ["data"] — the envelope models forbid extras.
<< 81 4a (2 bytes: the frame header)
~ byte 0 = 0x81 FIN=1 RSV=000 opcode=0x1 (TEXT)
~ byte 1 = 0x4a MASK=0 length field = 74
~ payload = 74 bytes, carried in the length field itself
<< {"type":"state","data":{"episode_id":"ep-fixed","step_count":2,"total":4}}
~ the FULL state dump, every field the environment wrote. GET /state
~ would have shown you exactly two keys — episode_id and step_count —
~ because it truncates to the base State model. WART-05: this is the
~ one place a non-Python client genuinely cannot see everything.
episode_id, step_count, total. Over
GET /state you would see two: the reference serialises the base
State model and your subclass fields are dropped without a word. That is
WART-05, the one place a non-Python
client genuinely cannot see everything.
— site/wire.txtFrame 5 — close
close ends the session, and it is the one message answered
with silence: no observation comes back. The server breaks its loop and
starts the closing handshake instead. Each side sends a small CLOSE
frame, a goodbye, so the socket ends by agreement rather than by dropping.
The probe's half of that goodbye was written by its frame reader, not by any
caller.
--- frame 5 -------------------------------------------------
>> 81 90 59 4b b8 21 (6 bytes: the frame header)
~ byte 0 = 0x81 FIN=1 RSV=000 opcode=0x1 (TEXT)
~ byte 1 = 0x90 MASK=1 length field = 16
~ payload = 16 bytes, carried in the length field itself
~ masking key = 59 4b b8 21
>> {"type":"close"}
~ the one message answered with SILENCE. No frame comes back; the server
~ breaks its loop and performs the CLOSING HANDSHAKE instead, which is what
~ you are about to watch.
<< 88 02 (2 bytes: the frame header)
~ byte 0 = 0x88 FIN=1 RSV=000 opcode=0x8 (CLOSE)
~ byte 1 = 0x02 MASK=0 length field = 2
~ payload = 2 bytes, carried in the length field itself
~ a control frame: it may arrive between the fragments of a message, it is
~ never fragmented itself, and its payload is at most 125 bytes.
>> 88 82 ad b3 f5 ee ae 5b (8 bytes: a whole frame, header and payload)
~ the reader wrote that reply itself. Three of them are not optional in
~ RFC 6455 — a Pong for every Ping, an echo for every Close, and the
~ Close that carries a framing error — and a caller that has to remember
~ them will one day not.
~ close code 1000 — 1000 is `the work is finished`.
~ the reader has already echoed this Close for us. Both sides have now
~ said goodbye, which is the difference between a closed socket and a
~ dropped one: a drop loses whatever the peer was still saying.
oe_probe: every exchange completed.
0x8, two
bytes of payload, close code 1000. Both sides said goodbye, which is the
difference between a closed socket and a dropped one — a drop loses whatever
the peer was still saying. — site/wire.txtGET /schema — the typing channel
One question is still open: how does a client know what an action looks
like? The probe never asked; {"amount":2} came from its command
line. Nothing on the WebSocket publishes the shape. GET /schema
is the only place a non-Python client can learn what an environment accepts.
The answer is three JSON Schema objects in 1,468 bytes, on one line — much
the longest on this page. Every plate here soft-wraps the way a terminal
does; where the line breaks is the window's business and not the
protocol's.
$ build/rel/bin/oe_probe http://127.0.0.1:8990 --schema
oe_probe — http://127.0.0.1:8990
>> is what this program sent. << is what the server sent.
~ is annotation: our prose, never bytes.
It uses no protocol code of ours — only sockets, HTTP, RFC 6455 and
JSON — so what scrolls past is the wire itself.
=== GET /schema — the only machine-readable typing channel =================
>> GET /schema HTTP/1.1
>> Host: 127.0.0.1:8990
>> Accept: application/json
>> Connection: close
>>
~ every line above ends CRLF, and the blank line ends the head.
~ Connection: close — one exchange per connection. This program does not
~ pipeline, and neither does the server: it says `connection: close` on
~ every response and means it.
<< HTTP/1.1 200 OK
<< content-length: 1468
<< content-type: application/json
<< connection: close
<<
<< {"action":{"additionalProperties":false,"properties":{"metadata":{"additionalProperties":true,"description":"Additional metadata for the action","title":"Metadata","type":"object"},"amount":{"default":1,"title":"Amount","type":"integer"},"note":{"default":"","title":"Note","type":"string"}},"title":"CounterAction","type":"object"},"observation":{"additionalProperties":false,"properties":{"done":{"default":false,"description":"Whether the episode has terminated","title":"Done","type":"boolean"},"reward":{"anyOf":[{"type":"boolean"},{"type":"integer"},{"type":"number"},{"type":"null"}],"default":null,"description":"Reward signal from the last action","title":"Reward"},"metadata":{"additionalProperties":true,"description":"Additional metadata for the observation","title":"Metadata","type":"object"},"total":{"default":0,"title":"Total","type":"integer"},"last_note":{"default":"","title":"Last Note","type":"string"}},"title":"CounterObservation","type":"object"},"state":{"additionalProperties":true,"description":"Base class for environment state.\n\nRepresents internal environment state, separate from observations.","properties":{"episode_id":{"anyOf":[{"type":"string"},{"type":"null"}],"default":null,"description":"Unique identifier for the current episode","title":"Episode Id"},"step_count":{"default":0,"description":"Number of steps taken in the current episode","minimum":0,"title":"Step Count","type":"integer"}},"title":"State","type":"object"}}
~ three JSON Schema objects: action, observation, state. This is the whole
~ of what a non-Python client can learn about an environment's shapes.
~ action: 3 properties — metadata amount note
~ observation: 5 properties — done reward metadata total last_note
~ state: 2 properties — episode_id step_count
~ `metadata` in the action, and `done`/`reward`/`metadata` in the
~ observation, are the framework's, not the author's: every OpenEnv
~ environment publishes them whether or not it declared them.
~ AND TWO OF THEM ARE A TRAP. `done` and `reward` are declared on the
~ observation model here, and NEITHER IS EVER SERIALISED INSIDE
~ `observation` on any transport — the reference dumps the inner model
~ with exclude={"reward","done"}. Read both from the TOP level of
~ `data`. Two shipped upstream environments read them from the wrong
~ place and see the class default forever.
~ `required` is derived from which properties have no `default`.
~ examples/oe_agent.c turns exactly this document into an action.
oe_probe: every exchange completed.
done and reward are
declared on the observation model, and neither is ever serialised
inside observation on any transport. Read both from
the top level of data, as plates 3 to 5 do.
examples/oe_agent.c ↗
turns exactly this document into an action.
— site/wire.txt$ kill %1
openenv info stopping
What to read next
That was one environment on one transport. The
tour is sixteen steps and about an hour, and at the end you know the
protocol rather than a codebase.
docs/PROTOCOL.md states the
same protocol as twenty-three numbered server requirements and twenty client
ones. The probe that printed this page is
examples/oe_probe.c ↗
— one file, with none of this repository's protocol code linked, and with
--health, --metadata and
--schema beside --episode. The captured reference
traffic all of this was derived from is
docs/evidence/OBSERVED-WIRE.md.
Point the probe at something you did not write:
build/rel/bin/oe_probe http://your-server --episode. If it prints
a byte this repository gets wrong,
that is the most
valuable issue this project can receive ↗.