miniopenenv

arcade/README.md → html

arcade/ — games, synthesis tasks, and text-first mathematics

make arcade            # twenty-two servers and the three clients that drive them
make train-cartpole    # start it, train against it over a WebSocket, print the curve
make task-dyna         # start one task server, mint tasks against it, print the sweep
make math-measure      # independently solve and attack 300 generated math tasks
make record-arcade     # the recorded fourteen-server gallery
make record-math       # all eight math servers into arcade/math-recordings.txt

The recorded gallery has fourteen environment servers: ten games and four synthesis tasks. The math room adds eight more, with its own client, recording, and launcher. The second command above runs 2000 episodes over a real WebSocket in a few seconds and prints a learning curve made of hashes. Nothing in this directory is mocked, stubbed or replayed: the trainer opens a real socket to a real server and plays until the numbers move. While it plays it says where it has got to, on stderr, so a twelve-second run is not twelve seconds of silence.

This directory is the ecosystem side of this repository's own boundary. ../envs/ teaches the protocol and is corpus-coupled — its bytes are diffed against captured recordings, so those files may not change. These are built on the protocol and are free. Delete the directory and no corpus case, oracle verdict or generated ledger changes.

That is why every file here opens with the same seven-line banner, and why the banner is the first thing a reader sees — tools/check.sh cmps those seven lines against CHARTER.md ↗'s copy in every arcade source, so the marking cannot drift file by file:

ARCADE — built ON the OpenEnv protocol, not part of the teaching core. envs/ is the other side of this boundary: corpus-coupled oracles that may not change, because tests diff their bytes against recordings. Delete this directory and no corpus case, oracle verdict or generated ledger in this repository changes: the one oracle that names a file here skips without it. These are ordinary downstream environments, written with the same public API any stranger gets: <openenv/openenv.h>, oe_main, and nothing else.

What deleting it does cost, exactly. The sentence above says proof and means it, because the experiment has been run — mv arcade /tmp && sh tools/check.sh — and it is not free. Two of make check's document rules point here, both named by the line they print:

make site stops too, because this file is a registered page and recordings.txt ↗ is the arcade page's quoted source. Remove the directory and those lines move with it. What does not move: make test prints the same 11,200 checks and t_replay the same 103 cases, because no corpus case, no oracle verdict and no conformance figure touches this directory. make interop's eighth oracle is the one place the arcade appears in a suite, and it skips when the binary is absent.

(Rules are cited by name and never by number. Their ordinals shift the moment anybody adds a rule above them — which is what adding 19d and 19e did — and a number that drifts silently is the whole defect this repository is built against.)

§ 1 · The ten games, and the port each one is recorded on

The ten games, and the port each one is recorded on

FileEnvironmentPortActionThe interesting part
bandit.c ↗BanditEnvironment8801{"arm": 0..9}ten Bernoulli arms; the smallest complete learning loop there is
cartpole.c ↗CartPoleEnvironment8802{"push": 0/1}the Barto-Sutton-Anderson equations, Euler-integrated
snake.c ↗SnakeEnvironment8803{"turn": 0/1/2}a grid as text, a growing tail, and food
g2048.c ↗Game2048Environment8804{"dir": 0..3}the real merge rules, one merge per tile per move
wordle.c ↗WordleEnvironment8805{"guess": "crane"}the two-pass colouring, so duplicate letters behave
blackjack.c ↗BlackjackEnvironment8806{"hit": 0/1}a reward you can only sample, and a dealer with no choices
maze.c ↗MazeEnvironment8807{"move": 0..3}the world is the seed: a maze carved fresh at every reset
pong.c ↗PongEnvironment8808{"paddle": 0/1/2}where the ball meets the paddle changes where it goes
connect4.c ↗Connect4Environment8809{"col": 0..6}the opponent lives inside the env and moves after you do
kuhn.c ↗KuhnPokerEnvironment8810{"bet": 0/1}private information on a public wire, and one session playing both seats

words.h ↗ is the one header an arcade file may include: 1466 common five-letter words typed into this repository — not downloaded, not scraped, not lifted from another program's list — so the table ships under this repository's own licence and building the arcade needs no network and no data file. The environment and the trainer read the same list, because a player knows the words — the secret is the answer. make check rule 19e reads the shape back: lowercase, five letters, strictly ascending, at least eight hundred entries. The order is load-bearing, because reset {"seed": N} answers with ARCADE_WORDS[mix(N) % N] — insert one word and every seed picks a different answer, which is how the committed recordings go stale.

None of the second five is another of the first five. Blackjack's reward can only be sampled, so the method is Monte Carlo rather than a bootstrap. Maze builds its world out of the reset seed — an iterative depth-first carve over a 7x7 cell grid, which produces a spanning tree and therefore a walkable path for every 64-bit seed, a guarantee the env description states because a procedurally generated environment that cannot promise a way out is a slot machine wearing a maze costume. Pong's metric is the game: the return is the rally. Connect four holds an opponent, so the env does something between your action and your observation. And Kuhn poker is not Q-learning at all.

§ 2 · The four of the synth lab, and the port each one is recorded on

The four of the synth lab, and the port each one is recorded on

FileEnvironmentPortActionThe interesting part
dyna.c ↗DynaEnvironment8821{"answer": "B"}state tracking: the answer is in no part of the text, only in the replay
inducto.c ↗InductoEnvironment8822{"answer": "A B C"}rule induction: fifty-two hidden rules in lookalike pairs, and shots that may not pin one down
logic.c ↗LogicEnvironment8823{"answer": "1"}circuit evaluation: one queried gate, and a reward you can re-derive with a pencil
filtro.c ↗FiltroEnvironment8824{"answer": "alpha"}source separation: two interleaved graphs over disjoint vocabularies; walk one, ignore the other

These four are not games, and they are the reason the directory stopped saying "ten". Each is a task generator with a verifiable reward, and each generator is one skill: dyna is state tracking, inducto is rule induction, logic is circuit evaluation, filtro is source separation. reset {"seed": N, "difficulty": 1|2|3} mints a task no file has ever held, the env computes the answer it will grade against in the same call, and the episode is one step long — text in, answer out, reward 1 on exact match and 0 otherwise, done true — for three of the four. dyna is the exception and takes three steps, because it asks three questions of one arrangement, one question per step, with no answer in any prompt. The recorded run opens by measuring the faucet — a thousand seeds, a thousand distinct input_text values, zero repeats, judged by a 64-bit FNV-1a hash — and the same seed twice comes back byte-identical.

The difficulty knob is real. Each family implements three presets: dyna 5-7/8-12/13-20 elements over 20-30/25-40/40-60 operations, logic 3-4/4-5/5-6 inputs at cone depth 2-3/3-5/5-8 with XOR probability .10/.30/.50, filtro 5-8/8-12/10-15 nodes, inducto 4-6/2-4/1-3 shots over rule complexity {1}/{1,2}/{2,3}. {"difficulty": 3} actually moves them, and each recorded sweep prints the knob ranges beside the results, so the claim has a table under it.

Three interface decisions are deliberate and stated: a difficulty that is not 1, 2 or 3 reads as 1; the shots inducto shows are not deduplicated and may leave a rule ambiguous; and POST /reset still validates only seed >= 0, so difficulty rides the same door seed does. What a reset parameter means is the environment's business, which is legal_actions' lesson pointed the other way.

That inducto ambiguity has a number, and it is printed rather than left qualitative. The shot range at difficulty 3 is 1-3, so a one-shot prompt is real — 33.6% of difficulty-3 tasks — and on 3.1% of difficulty-3 tasks the rules still consistent with every example disagree about the query. The answer is never in doubt (the environment applies the rule it drew), and a solver that keeps every consistent rule and bets among them caps at 98.3% there. That residue is deliberate: it is the tie mass arcade/synthlab.c learns to price, eleven of every twelve of those ties come from the one-shot draws, and CHARTER.md §2d records the pre-registered alternative — a 2-3 floor — and why it was not shipped.

§ 3 · The eight in the math room

The eight in the math room

FileEnvironmentPortWhat the learner has to do
measure.c ↗MeasureEnvironment8831keep quantities and units attached
algebra.c ↗AlgebraEnvironment8832return the complete solution set
change.c ↗ChangeEnvironment8833reason about a function on its stated domain
space.c ↗SpaceEnvironment8834follow coordinates and linear transformations
discrete.c ↗DiscreteEnvironment8835evaluate recurrences, paths, and finite counts
chance.c ↗ChanceEnvironment8836build the right finite probability space
evidence.c ↗EvidenceEnvironment8837aggregate the rows using the named statistic
reason.c ↗ReasonEnvironment8838submit a checked proof or a complete countermodel

These are text environments for language-model training, but their reward is plain code. Every prompt includes approachable prose, a compact Data: line that makes its conventions explicit, and a bounded return grammar. Seven families use exact integers, fractions, sets, or coordinates. reason checks the submitted certificate step by step and accepts any proof or countermodel that satisfies the printed rules.

mathlab.c ↗ is the independent reader. It parses only input_text, computes the answer with a separate implementation, checks that a correct-looking near miss earns zero, repeats the seed, and steps past done. It also probes the schema with a wrong JSON type and sends a long bounded-input attack before the sweep. make math-<name> runs those checks against one server; make record-math runs all eight. The math-room guide explains the task design, difficulty presets, grading boundary, deferred scope, and what a real learning experiment still needs to measure.

§ 4 · What each task resists, and the battery that prices it

What each task resists, and the battery that prices it

A task is only as good as the worst shortcut that solves it, so each generator carries evidence that the cheap shortcuts pay chance. interop/exploit_battery.py runs named cheap solvers — read a fixed position, answer from one token, copy the previous answer, sum what needs no tracking — over 2,000 fresh tasks per family per difficulty. The same solvers run over a control population its header calls real rows: a hundred frozen rows per family in the same grammar, held precisely because a fixed file is where cheap readers get paid. One sentence per family, with the measurement that makes the case. make record-arcade prints all four tables and grades the charter's pre-registered bars: a named reader more than four points above the sharpest cheap baseline turns the run red.

One collision inside the inventory is kept deliberately: SWAP_PAIRS and REVERSE_CHUNKS_2 are the same function under two names, which is why a constant guess of that shape is worth more than one rule's draw mass. Dropping either name would have been dodging the task rather than hardening it.

§ 5 · Every one of them obeys the same five rules

Every one of them obeys the same five rules

  1. One file, one oe_main. No arcade file includes another, except words.h. What each program gets from the library is what any stranger gets after make install: sixteen routes, GET /schema, WebSocket sessions, request validation, the production flag.
  2. reset reads {"seed": N} and reseeds everything from it. Absent or wrong-typed means zero, so a run with no seed is still reproducible.
  3. The randomness is ours. splitmix64, six lines, one copy per file. No rand, no clock, no /dev/urandom — an environment that reads a clock cannot be replayed, and replay is the whole point.
  4. After done, a step returns the terminal observation unchanged, with reward null and done true. A trainer bug then shows up as a flat line rather than a haunted curve.
  5. A bad discrete action is an error, not a default. An out-of-range arm, push, dir or col sets the session's error string and returns OE_EENV, which is the protocol's EXECUTION_ERROR frame, exercised honestly. connect4.c adds the one case none of the others had: a col in range but full is refused too, which is why that observation publishes the legal set. This rule binds nothing in the synth lab, and the reason is that none of those actions is an integer: answer is a string, a missing one reads as empty, and an empty answer scores zero. There is no illegal answer there, only a wrong one — so those four servers have no OE_EENV path at all. A wrong-typed one is the other case and never reaches them: {"answer": 5} is a VALIDATION_ERROR raised by the framework, because a string field takes no coercion — the number 5 is not the string "5" to this protocol — so it is refused at the door with an HTTP 422 and scores nothing rather than zero. The two are worth keeping apart, and the seed sentence above is why: seed is an integer, it does coerce, and a wrong-typed one really does read as zero.

And legal_actions is not a protocol field. It is an ordinary observation member that connect4.c ↗ invented because its action space shrinks as columns fill. The wire spec has never heard of it; the schema advertises it like any other field; the trainer reads it out of the observation like any other field and masks its choice with it. Upstream's own connect4 environment publishes one because its author had the same problem, not because a standard told either of us to. The protocol carries observations — what an observation means is the environment's business, and this is the clearest example of that line in the repository.

And input_text is not a protocol field either. Every server in all three wings publishes one. It is the whole decision as text: the situation, the picture where there is one, and how to reply on the last line. It carries nothing the other fields do not carry, so a trainer can paste it into a prompt without auditing the environment for a hole card. That is the whole promise, and CHARTER.md §7a writes out all five clauses of it. GET /schema advertises the field exactly the way it advertises grid, and a client that has never heard of it reads the same observation it always did. What it buys is the one thing the training walkthroughs used to make a reader hand-write: a prompt per game. Now there is one prompt and the environment writes it.

The arcade also knows how to stay up. Every other target here starts a server and stops it again. make arcade-up is the exception. It starts all fourteen on the pinned ports above, leaves them running, and writes build/rel/arcade.json: one object from a server's name to its base URL. That file carries the name and the URL and nothing else. The action field is on the wire at GET /schema, no conformant environment publishes its episode length, and the concurrency cap is the trainer's. The manifest is written last, after every server has answered, so its existence means the arcade is up. make arcade-down reads the pidfile it wrote, signals exactly those pids, and takes both files away. build/ is gitignored, so the manifest is never a tracked file and make clean removes it.

The math room has the parallel make math-up and make math-down pair. It writes build/rel/math.json and uses a separate pidfile, so stopping one wing cannot stop the other.

make adapters is what reads that manifest. Four legs of smoke tests drive ../interop/openenv_text.py ↗ and the three shipped training adapters against the running arcade, with no GPU and no model weights. All fourteen are in reach of those adapters, the ten games included. An action field the schema calls integer is still answered in text, and the adapter takes the first run of digits out of the reply. One leg plays connect four end to end on nothing but the board render and a sentence naming a column. That target binds no port of its own, and it skips in writing when these binaries are absent — this directory is outside the proof boundary, and every target that touches it keeps that promise. ../docs/TRAINING.md is the walkthrough, and it says which parts of it have been run and which have not.

§ 6 · train.c — one client, ten learners

train.c — one client, ten learners

$ build/rel/bin/train --help

One binary, ten subcommands, one curve printer, one clock. It speaks <openenv/openenv.h> and nothing else of ours: it links no environment and includes no environment's header, so every feature it uses arrived as JSON over the socket. That is the constraint that makes the whole thing a demonstration rather than a diagram — the same binary would train an eleventh game written in Python.

Before it trains, it fetches GET /metadata and refuses a server whose environment name is not the one it was asked for. A trainer that grades the wrong game silently is worse than one that will not start.

The algorithms are deliberately small and are named on the first line of every run: epsilon-greedy for the bandit; tabular Q-learning for cart-pole, snake, 2048, the maze and pong; every-visit Monte Carlo control for blackjack; afterstate values, masked by the observation's legal_actions, for connect four; for Wordle a programmed deduction with an epsilon-greedy choice of opening word; and for Kuhn poker outcome-sampling regret matching, which is the only learner here that is not estimating a value at all. Each run prints what it learns and what it does not, and no run claims to solve its game. Wordle's finale does say solved 99.0%, and that is the game's own verb for finding an answer, reported for the programmed deduction — not a claim about the curve, which is only ever about the opening word.

The curve's header names what its bars span, because the bars are scaled between the run's own smallest and largest bucket. Forty hashes means "the top of this run", not "good": Wordle moves the return from 3.06 to 3.16 and snake from -0.76 to 8.05, and both draw the same last bar. The numbers beside them carry the size.

One curve on that page wins downward. Kuhn poker's value is exploitability — how much a best response would take off the average strategy, in chips per hand — so falling is the good direction, and a chart that let a tall bar mean the worst bucket would be lying by layout on a page where nine plates read tall as best. That game declares its value falls; its bars fill toward the run's own best bucket and its header writes the span top-first, (bars span 0.13 down to 0.02). Forty hashes keeps one meaning everywhere — the best this run reached — and the words down to name the direction.

§ 7 · synthlab.c — the other client, and why it is not a subcommand of the first

synthlab.c — the other client, and why it is not a subcommand of the first

$ build/rel/bin/synthlab --help

train.c stands at its own ceiling and the wing is a different algorithm family, so it got its own file rather than four more subcommands. No member of it keeps a table of values, none of them discounts, and three of the four learn nothing at all: the family's algorithm is parse, compute, verify. It shares nothing with train.c but the idiom, and the few dozen lines of session and printing it needs are duplicated, because duplication is this directory's price for having no plumbing.

Every subcommand prints the generation demo first — a thousand resets, hashed and counted, the same seed twice compared byte for byte, and the rate over the socket — because that is the measurement the whole wing exists to make.

Then, per family, the honest artifact and nothing more:

The ceiling is Bayes' and the claim never exceeds it — and uniform is not the ceiling, which is what this paragraph used to say. The generator does not draw uniformly at difficulty 3: it picks a complexity from {2, 3} half and half and only then a rule uniformly inside that pool, and the pools are twenty rules and twelve — ten and six base rules, each doubled by its lookalike — so a complexity-3 rule arrives 5/3 as often as a complexity-2 one. The Bayes tie-break is therefore the draw-weighted argmax over the survivors' answers, not the uniform one. So the run measures the ceiling on its own ties instead of asserting it: the finale's second line scores three oracles over the same 1890 ties — a uniform draw among survivors pays 45.3%, a plurality of them 46.1%, and the draw-weighted argmax 68.0%. That last is the ceiling; the curve's last bucket is 64.0%, below it, which is honest headroom and not an embarrassment. Two things are learnable, and the prior finds both from reward alone: the pool — the twenty simplest rules the hardest preset never draws, which fit a short example list anyway and lose, priced at 0.19 — and the ratio, that a complexity-3 survivor is the better bet. The oracles never touch the learner; they read the revealed answer only to score, which is the same licence the maze trainer's shortest-path line has.

§ 8 · The recording

The recording

make record-arcade starts each of the fourteen servers on its pinned port, drives one episode by hand with oe_probe, runs a client against it (train for the games, synthlab for the task servers), stops the server, and writes the whole session to recordings.txt ↗. That file is the only source of the numbers on the arcade page of the site: make site refuses to publish a transcript line the recording does not contain, refuses a block that reorders one, and refuses a figure in that page's prose the recording never printed.

The runs replay exactly — same seed, same actions, same observations — so re-recording changes only the wall-clock figures, which are measured and therefore move. The whole run takes well under a minute, and no single server out-spends 2048's twelve seconds: a plate is a demonstration, not a vigil. The four task servers cost under two seconds between them.

Four sections at the end have a judge that is not this file's. openenv validate, the reference project's own conformance suite, is run against three of the games over the network — connect four is one of them on purpose, because a server whose observation carries a field the protocol never heard of is exactly the one worth handing to upstream's judge. The synth lab's four are parsed by tools/synthlab_parity.py under one grammar per family, over the hundred saved control rows and a thousand fresh resets — material under the gitignored reference/, so that section skips, in writing, on a clone that does not have it, and make check still passes on a stranger's clone. Those same four are then run against ../interop/exploit_battery.py ↗, the cheap solvers of the section above, with the control rows scored beside them and the charter's pre-registered bars graded at the end: a battery that cannot re-derive its own answers stops the recording with nothing written, and a failed bar is recorded and carried out in the exit status, because a transcript that shows its own red rows is worth more than a missing one. And ../interop/bandit_env.py ↗ is bandit.c ported onto the reference's own server — their create_app, their pydantic validation, under uvicorn — so the same trainer at the same seed can be pointed at two implementations of one game. Both must print the same curve before a throughput ratio is printed at all; the recorder diffs them and says so either way. All four sections skip, in writing, when what they need is absent.

§ 9 · What this directory is not

What this directory is not

It is not a benchmark suite. There is exactly one comparison in the recording and it is between two implementations of one game, on one machine, with one request in flight — a per-request latency figure and not a verdict on anybody's project.

Its runs are not part of make check: a gate that binds ports and trains for a minute is a gate contributors learn to route around. The sources are gated like every other consumer — arcade joins ALL_SRC, NONWIRE_SRC and NOALLOC_SRC in tools/check.sh, so banned functions, the goto rule, the OE_TRY dialect and allocator confinement all cover this directory — plus the two rules that are only about this directory: 19d cmps the banner against the charter's copy in every file here, and 19e reads back the shape of words.h.

The one place the arcade appears in a suite is make interop's eighth oracle, which trains bandit.c with upstream's own Python client — and that oracle SKIPS when the binary is absent, which is why no verdict in this repository depends on the arcade being present.

§ 10 · See also

See also