miniopenenv

The arcade

Fourteen small environment servers built on the OpenEnv protocol: ten games a trainer learns to play over a real WebSocket, and four task generators that mint a fresh problem per seed and grade the answer themselves. A third wing adds eight text-first math environments. Every transcript and every number on this page is recorded output from programs in arcade/.

built on the protocol · not part of the teaching core · Apache-2.0

The math room covers quantities, algebra, functions, transformations, discrete structures, probability, data, and checked reasoning. Its approachable guide, exact answer grammars, independent solver, and adversarial checks live in arcade/MATH.md. The eight plates are on this page too, with their recorded checks beside them.

The original gallery is fourteen servers in arcade/ ↗. Ten of them are games: a ten-armed bandit, a cart-pole, snake, 2048, Wordle, blackjack, a maze, pong, connect four and Kuhn poker. Four of them are the synth lab, and they are not games at all — they hand you a problem in text and grade the answer. Each one is a complete OpenEnv server, and each one is one file you can read in a sitting. Beside them are two clients: arcade/train.c ↗, which opens a WebSocket to whichever game is running and learns to play it, and arcade/synthlab.c ↗, which reads the task servers' text and answers it. The math room has a third client, mathlab.c ↗, which independently parses, solves, and attacks every math family.

OpenEnv is Hugging Face's standard for running reinforcement-learning environments as services. A client resets an environment, then steps it with actions; each step returns an observation and a reward. An episode is one run of that loop: a reset, then steps, until done. Neither client links any of these servers or includes any of their headers — every feature they use arrived as JSON on a socket, which is why one binary trains all ten games and would train an eleventh written in Python.

Nothing below is a diagram. make record-arcade starts each server on a pinned port, drives one episode by hand with oe_probe ↗, runs a client against it, and writes the session to arcade/recordings.txt ↗. The math room repeats that process with make record-math and arcade/math-recordings.txt ↗. This page may quote only those files: make site exits nonzero on a transcript line they do not contain, and on a figure in this prose neither recording printed.

INDEX

The plates, one click each

RUN ONEindex

Three commands

These servers are ordinary targets of the ordinary build, so there is nothing to install and nothing to configure. Build, start one, point a client at it:

$ make arcade
arcade: 16 binaries in build/rel/bin — bandit blackjack cartpole connect4 dyna filtro g2048 inducto kuhn logic maze pong snake synthlab train wordle
        plus oe_probe for poking them by hand
        try: make train-bandit   (or run one by hand: build/rel/bin/bandit --port 8801 &)
...
$ build/rel/bin/snake --host 127.0.0.1 --port 8803 &
...
$ build/rel/bin/train snake http://127.0.0.1:8803 --seed 7
The whole setup. make train-snake is the same three commands in one, make task-dyna is the same three for a synth-lab server with synthlab as the client, and make record-arcade is all fourteen end to end. make math-measure and make record-math are the matching commands for the math room. The server binds loopback because these are development servers; the seed makes the run repeatable, and it is the client's own default. A ... line is this page saying it cut something: the three commands are recorded in three different places and the output between them is below, plate by plate. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain, and any block that reorders one.

What follows is one plate per server, in the order the recording made them, which is the order the ports ascend. Each shows an episode driven by hand — so you can see the shape of an observation before anything learns — and then the client run. Both blocks are in the order the recording has them, from the reset to the close, because a transcript with a step quietly removed is a transcript whose own step_count contradicts it; where a plate abridges, it prints a ... line and you can see the cut.

One field is in every observation on this page, and it is the newest thing here. input_text is the whole decision as text: the situation, the picture where there is one, and how to reply on the last line. It carries nothing the other fields do not carry. So a trainer can paste it into a prompt without auditing the environment for a hole card. The protocol has never heard of it — like legal_actions on plate 9, it is an ordinary field with a naming convention around it. It is here so a language model can play these servers with no prompt hand-written per game, which is what the training walkthroughs now rely on.

THE GAMES · PLATES 1-10index

Ten games, one trainer

Ten games, and one trainer that plays them all. arcade/train.c opens a WebSocket to whichever server is running, and every feature it uses arrived as JSON on that socket. The ten are not ten of one kind. One has an opponent living inside the environment, moving between your action and your observation. One carves its world out of the reset seed. One pays a reward that can only be sampled, and stops below zero. One measures a metric that is the game itself. And the last is not estimating a value at all — its chart wins by falling.

PLATE 1 · BANDITindex

Ten arms, and the smallest complete loop there is

Ten slot machines in a row, each paying one or nothing at its own fixed probability; the seed decides which arm holds which. There is no state to speak of and no strategy to plan — the only way to learn a probability is to sample it, which makes this the shortest honest reinforcement-learning loop that exists.

$ build/rel/bin/bandit --host 127.0.0.1 --port 8801 &

$ build/rel/bin/oe_probe http://127.0.0.1:8801 --episode --steps 3 --action '{"arm":3}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ten arms, 0 to 9. Each pays 1 with its own fixed probability, else 0. The best arm pays 0.95; only its index changes.\nPull 1 of 100. No arm pulled yet.\nReply with one digit, the arm to pull: 0 to 9.","pulls":0,"last_arm":-1,"last_payout":0.0},"reward":null,"done":false}}
>> {"type":"step","data":{"arm":3}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ten arms, 0 to 9. Each pays 1 with its own fixed probability, else 0. The best arm pays 0.95; only its index changes.\nPull 2 of 100. Arm 3 paid 0 last time.\nReply with one digit, the arm to pull: 0 to 9.","pulls":1,"last_arm":3,"last_payout":0.0},"reward":0,"done":false}}
>> {"type":"step","data":{"arm":3}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ten arms, 0 to 9. Each pays 1 with its own fixed probability, else 0. The best arm pays 0.95; only its index changes.\nPull 3 of 100. Arm 3 paid 1 last time.\nReply with one digit, the arm to pull: 0 to 9.","pulls":2,"last_arm":3,"last_payout":1.0},"reward":1,"done":false}}
>> {"type":"step","data":{"arm":3}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ten arms, 0 to 9. Each pays 1 with its own fixed probability, else 0. The best arm pays 0.95; only its index changes.\nPull 4 of 100. Arm 3 paid 1 last time.\nReply with one digit, the arm to pull: 0 to 9.","pulls":3,"last_arm":3,"last_payout":1.0},"reward":1,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"winnings":2}}
>> {"type":"close"}
$ build/rel/bin/train bandit http://127.0.0.1:8801 --seed 7
train bandit — epsilon-greedy, sample-average estimates, 10 arms
learns    which of ten arms pays, from what came back when it pulled them
does not  see the ladder, carry an estimate to a new seed, or stop exploring
server    http://127.0.0.1:8801 — BanditEnvironment
run       200 episodes · seed 7 · eps 0.10 · sample-average · a new machine each visit

pulls             avg payout   curve  (bars span 0.63 to 0.83)
   1-  10               0.63
  11-  20               0.70   ##############
  21-  30               0.72   ##################
  31-  40               0.76   #########################
  41-  50               0.78   ##############################
  51-  60               0.77   ############################
  61-  70               0.79   ##############################
  71-  80               0.80   ##################################
  81-  90               0.83   ######################################
  91- 100               0.83   ########################################

200 episodes · 20000 steps · 0.2 s · 85710 steps/s · final avg 0.83
first ten pulls .63 · last ten .83 · favourite arm paid .82 · ladder .50 average, .95 best
Plate 1. The oldest plot in the subject, drawn in a fifth of a second. Read the x axis: it is pull number, not episode. The player forgets its table at every reset and the machine is a new one each visit, so what these ten rows average is what the forty-first pull paid across two hundred separate visits — which is where this agent's learning actually happens. It opens at .63 rather than .50 because a player that has been paid once starts exploiting immediately, and it ends at .83 against a ladder whose average arm pays .50 and whose best pays .95. Read the bar scale in the header too: the hashes run from this run's own worst bucket to its own best, so a full bar means the top of this curve and never "good". Two earlier versions of this trainer drew a flat line and a jagged one; the comment above play_bandit says what each of them got wrong. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"arm": 0..9} — required; anything else is an EXECUTION_ERROR frame
observationinput_text, pulls, last_arm, last_payout
rewardinteger 0 or 1, drawn at that arm's probability
doneafter a hundred pulls
PLATE 2 · CARTPOLEindex

A pole that falls over unless you keep moving

The classic control problem, with the Barto-Sutton-Anderson equations integrated at fifty steps a second: a cart on a track, a pole hinged on top, and two actions — push left, push right. It is the plainest picture of what a learning curve is, because the reward is one per step and the episode ends when the pole tips, so the return is how long it stayed up.

$ build/rel/bin/cartpole --host 127.0.0.1 --port 8802 &

$ build/rel/bin/oe_probe http://127.0.0.1:8802 --episode --steps 3 --action '{"push":1}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A pole stands on a cart that can only be pushed sideways. Keep it upright.\nCart 0.038 m, moving -0.007 m/s. Pole -0.047 rad, turning 0.047 rad/s. Positive is right; three decimals here, the fields carry the rest.\nStep 1 of 500. Past 2.4 m or past 0.2095 rad ends it.\nReply with one digit: 0 push left, 1 push right. There is no coasting.","x":0.038331080821364265,"x_dot":-0.006847200295149004,"theta":-0.04735662284074023,"theta_dot":0.04708819781538285},"reward":null,"done":false}}
>> {"type":"step","data":{"push":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A pole stands on a cart that can only be pushed sideways. Keep it upright.\nCart 0.038 m, moving 0.189 m/s. Pole -0.046 rad, turning -0.260 rad/s. Positive is right; three decimals here, the fields carry the rest.\nStep 2 of 500. Past 2.4 m or past 0.2095 rad ends it.\nReply with one digit: 0 push left, 1 push right. There is no coasting.","x":0.038194136815461285,"x_dot":0.18892069604837716,"theta":-0.04641485888443257,"theta_dot":-0.2601520728061378},"reward":1.0,"done":false}}
>> {"type":"step","data":{"push":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A pole stands on a cart that can only be pushed sideways. Keep it upright.\nCart 0.042 m, moving 0.385 m/s. Pole -0.052 rad, turning -0.567 rad/s. Positive is right; three decimals here, the fields carry the rest.\nStep 3 of 500. Past 2.4 m or past 0.2095 rad ends it.\nReply with one digit: 0 push left, 1 push right. There is no coasting.","x":0.04197255073642883,"x_dot":0.3846734497997366,"theta":-0.05161790034055533,"theta_dot":-0.5671060412153819},"reward":1.0,"done":false}}
>> {"type":"step","data":{"push":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A pole stands on a cart that can only be pushed sideways. Keep it upright.\nCart 0.050 m, moving 0.580 m/s. Pole -0.063 rad, turning -0.876 rad/s. Positive is right; three decimals here, the fields carry the rest.\nStep 4 of 500. Past 2.4 m or past 0.2095 rad ends it.\nReply with one digit: 0 push left, 1 push right. There is no coasting.","x":0.049666019732423565,"x_dot":0.5804800344054798,"theta":-0.06296002116486296,"theta_dot":-0.875593648147289},"reward":1.0,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"theta":-0.06296002116486296}}
>> {"type":"close"}
$ build/rel/bin/train cartpole http://127.0.0.1:8802 --seed 7
train cartpole — tabular Q-learning, 162 boxes x 2 actions
learns    when to push left and when to push right, from four numbers
does not  know the physics, look ahead, or tell two states in one box apart
server    http://127.0.0.1:8802 — CartPoleEnvironment
run       2000 episodes · seed 7 · eps 1.00->0.05 · alpha 0.10 · gamma 0.99

episodes          avg return   curve  (bars span 23.47 to 124.91)
   1- 200              23.47
 201- 400              23.89
 401- 600              30.54   ###
 601- 800              38.91   ######
 801-1000              49.16   ##########
1001-1200              49.02   ##########
1201-1400              77.77   #####################
1401-1600              79.81   ######################
1601-1800             110.38   ##################################
1801-2000             124.91   ########################################

2000 episodes · 121572 steps · 2.9 s · 41252 steps/s · final avg 124.91
best episode 500 steps · 103 of 162 boxes visited
Plate 2. This is the plate to read if you read one. A table of 162 boxes, nothing else — no network, no gradient, no physics — goes from dropping the pole in 23.47 steps to holding it for 124.91, and its best episode ran to the 500-step cap. It took 2.6 seconds of one core, and 121572 of those steps were round trips over a WebSocket. The claim is improvement and only improvement: 124.91 is not balanced, and this page will never call a cart-pole solved. Notice what the three hand-driven steps above already show — three pushes to the right, and theta falling further from upright each time. The state door answers with the value after the third of them, which is the observation directly above it. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"push": 0} left or {"push": 1} right — required
observationinput_text, then x, x_dot, theta, theta_dot — four reals, which the text rounds to three places and the fields carry whole
reward1.0 for every step the pole is still up
donethe cart leaves the track, the pole passes twelve degrees, or the step cap
PLATE 3 · SNAKEindex

The one everybody already knows

A snake on a small grid, a piece of food, and three choices at every step: turn left, go straight, turn right. Eating grows the tail and moves the food; a wall or your own body ends the episode. Upstream ships a snake environment of its own, on gym and marlenv; this is the same game written for this protocol in one C file, answering in microseconds. The shapes are ours, not theirs — the action here is turn, and an upstream-shaped action gets a validation error, which is the server behaving correctly.

$ build/rel/bin/snake --host 127.0.0.1 --port 8803 &

$ build/rel/bin/oe_probe http://127.0.0.1:8803 --episode --steps 3 --action '{"turn":1}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Snake, 12x12. 'O' head, 'o' body, '@' food, '.' empty. Eating grows you; a wall or your own body ends the game.\n............\n............\n............\n.......@....\n............\n............\n....ooO.....\n............\n............\n............\n............\n............\nHead at column 6, row 6, heading right. Food at column 7, row 3. Length 3, score 0.\nReply with one digit, a turn relative to your heading: 0 left, 1 straight, 2 right.","grid":"............\n............\n............\n.......@....\n............\n............\n....ooO.....\n............\n............\n............\n............\n............","head_x":6,"head_y":6,"food_x":7,"food_y":3,"dir":1,"length":3,"score":0},"reward":null,"done":false}}
>> {"type":"step","data":{"turn":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Snake, 12x12. 'O' head, 'o' body, '@' food, '.' empty. Eating grows you; a wall or your own body ends the game.\n............\n............\n............\n.......@....\n............\n............\n.....ooO....\n............\n............\n............\n............\n............\nHead at column 7, row 6, heading right. Food at column 7, row 3. Length 3, score 0.\nReply with one digit, a turn relative to your heading: 0 left, 1 straight, 2 right.","grid":"............\n............\n............\n.......@....\n............\n............\n.....ooO....\n............\n............\n............\n............\n............","head_x":7,"head_y":6,"food_x":7,"food_y":3,"dir":1,"length":3,"score":0},"reward":0.01,"done":false}}
>> {"type":"step","data":{"turn":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Snake, 12x12. 'O' head, 'o' body, '@' food, '.' empty. Eating grows you; a wall or your own body ends the game.\n............\n............\n............\n.......@....\n............\n............\n......ooO...\n............\n............\n............\n............\n............\nHead at column 8, row 6, heading right. Food at column 7, row 3. Length 3, score 0.\nReply with one digit, a turn relative to your heading: 0 left, 1 straight, 2 right.","grid":"............\n............\n............\n.......@....\n............\n............\n......ooO...\n............\n............\n............\n............\n............","head_x":8,"head_y":6,"food_x":7,"food_y":3,"dir":1,"length":3,"score":0},"reward":-0.01,"done":false}}
>> {"type":"step","data":{"turn":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Snake, 12x12. 'O' head, 'o' body, '@' food, '.' empty. Eating grows you; a wall or your own body ends the game.\n............\n............\n............\n.......@....\n............\n............\n.......ooO..\n............\n............\n............\n............\n............\nHead at column 9, row 6, heading right. Food at column 7, row 3. Length 3, score 0.\nReply with one digit, a turn relative to your heading: 0 left, 1 straight, 2 right.","grid":"............\n............\n............\n.......@....\n............\n............\n.......ooO..\n............\n............\n............\n............\n............","head_x":9,"head_y":6,"food_x":7,"food_y":3,"dir":1,"length":3,"score":0},"reward":-0.01,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"score":0}}
>> {"type":"close"}
$ build/rel/bin/train snake http://127.0.0.1:8803 --seed 7
train snake — tabular Q-learning, 288 states x 3 actions
learns    which turn to take from local danger and food direction
does not  see the whole board, plan ahead, or manage its tail
server    http://127.0.0.1:8803 — SnakeEnvironment
run       5000 episodes · seed 7 · eps 1.00->0.05 · alpha 0.20 · gamma 0.90

episodes          avg return   curve  (bars span -0.76 to 8.05)
   1- 500              -0.76
 501-1000              -0.49   #
1001-1500              -0.21   ###
1501-2000               0.30   #####
2001-2500               0.55   ######
2501-3000               1.08   ########
3001-3500               1.71   ###########
3501-4000               2.82   ################
4001-4500               4.87   ##########################
4501-5000               8.05   ########################################

5000 episodes · 216327 steps · 3.4 s · 63723 steps/s · final avg 8.05
last bucket avg score 8.45 · best score 25 · 254 of 288 states visited
Plate 3. The curve everybody wants: from losing (-0.76, a snake walking into walls) to 8.05, with the last five hundred games averaging 8.45 apples and one of them reaching 25. The state is three danger bits, a heading and the sign of the food's offset — 288 of them, of which it visited 254 — and it was read out of the grid string above by the trainer, not handed to it. The three hand-driven steps show the reward doing its job: the first move closes on the food and pays a hundredth, the next two overshoot it and are charged a hundredth each. Nothing here plans around its own tail, and the growing snake is what eventually kills it. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"turn": 0|1|2} — left, straight, right; defaults to straight
observationinput_text, grid as text rows, plus head_x, head_y, food_x, food_y, dir, length, score
reward1.0 for food, -1.0 for dying, and a hundredth either way for closing on the food or drifting off it
donewall, tail, a full board, or too long without eating
PLATE 4 · G2048index

2048, and the state that could not see a direction

The tile game, with the real merge rules: one merge per tile per move, a new 2 or 4 after any move that changed something, and a score that is the sum of what merged. It is here because the upstream project's own reinforcement-learning tutorial trains against a Python 2048 — and because it is the one game on this page whose first learner lost to random play, which turned out to be a lesson about features rather than about 2048.

$ build/rel/bin/g2048 --host 127.0.0.1 --port 8804 &

$ build/rel/bin/oe_probe http://127.0.0.1:8804 --episode --steps 3 --action '{"dir":0}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"2048 on a 4x4 board, '.' empty. Equal neighbours merge once each per move, and a merge scores the tile it makes.\n    .    .    .    .\n    2    .    .    .\n    .    .    .    .\n    .    .    .    4\nScore 0, best tile 4. A direction that shifts nothing is legal and does nothing.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","board":[0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,4],"score":0,"max_tile":4,"moved":false},"reward":null,"done":false}}
>> {"type":"step","data":{"dir":0}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"2048 on a 4x4 board, '.' empty. Equal neighbours merge once each per move, and a merge scores the tile it makes.\n    2    .    .    4\n    .    .    .    .\n    .    .    .    4\n    .    .    .    .\nScore 0, best tile 4. A direction that shifts nothing is legal and does nothing.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","board":[2,0,0,4,0,0,0,0,0,0,0,4,0,0,0,0],"score":0,"max_tile":4,"moved":true},"reward":0,"done":false}}
>> {"type":"step","data":{"dir":0}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"2048 on a 4x4 board, '.' empty. Equal neighbours merge once each per move, and a merge scores the tile it makes.\n    2    .    4    8\n    .    .    .    .\n    .    .    .    .\n    .    .    .    .\nScore 8, best tile 8. A direction that shifts nothing is legal and does nothing.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","board":[2,0,4,8,0,0,0,0,0,0,0,0,0,0,0,0],"score":8,"max_tile":8,"moved":true},"reward":8,"done":false}}
>> {"type":"step","data":{"dir":0}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"2048 on a 4x4 board, '.' empty. Equal neighbours merge once each per move, and a merge scores the tile it makes.\n    2    .    4    8\n    .    .    .    .\n    .    .    .    .\n    .    .    .    .\nScore 8, best tile 8. A direction that shifts nothing is legal and does nothing.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","board":[2,0,4,8,0,0,0,0,0,0,0,0,0,0,0,0],"score":8,"max_tile":8,"moved":false},"reward":0,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"score":8}}
>> {"type":"close"}
$ build/rel/bin/train g2048 http://127.0.0.1:8804 --seed 7
train g2048 — tabular Q-learning, 200 coarse states x 4 actions
learns    when to keep pushing one way and when the board wants another
does not  see the tiles it is moving, count merges, or reach 2048
server    http://127.0.0.1:8804 — Game2048Environment
run       6000 episodes · seed 7 · eps 0.20->0.02 · alpha 0.02 · gamma 0.95 · 200 random games first

episodes          avg return   curve  (bars span 1376.39 to 1875.63)
   1- 600            1376.39
 601-1200            1581.61   ################
1201-1800            1558.19   ###############
1801-2400            1703.13   ##########################
2401-3000            1730.76   ############################
3001-3600            1742.35   #############################
3601-4200            1875.63   ########################################
4201-4800            1782.37   #################################
4801-5400            1869.58   ########################################
5401-6000            1790.95   #################################

6000 episodes · 1056853 steps · 14.6 s · 72151 steps/s · final avg 1790.95
random baseline 200 games · avg score 979 · best tile 256 · learned best tile 512
Plate 4. The trainer measures random play first — 979 a game, and in two hundred games of it the board never got past 256 — and then beats it, from 1376.39 over the first six hundred games to 1790.95 over the last, reaching 512 along the way. That finale line used to report one best tile and attribute it to random play, and the figure was the learner's: the two runs now keep their own, which is both honest and the better result. What changed in the algorithm is worth more than either number. The first version of this state held three facts about the position: how big the biggest tile is, how empty the board is, whether that tile sits in a corner. None of the three distinguishes one direction from another, so a table indexed like that can only learn a fixed favourite push — and a fixed favourite push in 2048 jams the board, which the three hand-driven steps above show in miniature: the third {"dir":0} comes back "moved":false, reward 0, the board unchanged. That run finished under the random baseline it printed beside itself. Adding what the last move did (which way, and whether anything moved) turns a table of preferences into a table of plans: push this way while it works, and when it stops working push that way instead. It still will not reach 2048, and the curve plateaus after about 3600 games rather than climbing to the end — the last bucket is below the peak two buckets earlier. 1056853 steps went over the socket in 12.2 seconds, so that part is not what is stopping you. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"dir": 0|1|2|3} — up, right, down, left; required, and a move that shifts nothing is legal
observationinput_text, board as sixteen integers row-major, plus score, max_tile, moved — and moved is the field the trainer needed
rewardinteger: the score gained by this move
doneno direction moves anything
PLATE 5 · WORDLEindex

Strings in, colours back

The word game, with the real two-pass rule: greens first, then yellows against whatever letters are left, so a repeated letter behaves exactly as it does in the original. The action is a string rather than a number, which makes this the game on the page most like the ones people actually want to train language models against.

$ build/rel/bin/wordle --host 127.0.0.1 --port 8805 &

$ build/rel/bin/oe_probe http://127.0.0.1:8805 --episode --steps 3 --action '{"guess":"crane"}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Wordle: guess the hidden five-letter word in six tries. Each guess is scored letter by letter: G is the right letter in the right place, Y the right letter in the wrong place, X a letter the word does not have. A guess must be a word on the shared list; one that is not is scored and costs no attempt.\nGuess 1 of 6.\nReply with one five-letter word and nothing else.","feedback":"","valid":false,"guesses":0,"history":"","solved":false,"answer":""},"reward":null,"done":false}}
>> {"type":"step","data":{"guess":"crane"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Wordle: guess the hidden five-letter word in six tries. Each guess is scored letter by letter: G is the right letter in the right place, Y the right letter in the wrong place, X a letter the word does not have. A guess must be a word on the shared list; one that is not is scored and costs no attempt.\ncrane XXXYY\nGuess 2 of 6.\nReply with one five-letter word and nothing else.","feedback":"XXXYY","valid":true,"guesses":1,"history":"crane XXXYY","solved":false,"answer":""},"reward":0.1,"done":false}}
>> {"type":"step","data":{"guess":"crane"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Wordle: guess the hidden five-letter word in six tries. Each guess is scored letter by letter: G is the right letter in the right place, Y the right letter in the wrong place, X a letter the word does not have. A guess must be a word on the shared list; one that is not is scored and costs no attempt.\ncrane XXXYY\ncrane XXXYY\nGuess 3 of 6.\nReply with one five-letter word and nothing else.","feedback":"XXXYY","valid":true,"guesses":2,"history":"crane XXXYY\ncrane XXXYY","solved":false,"answer":""},"reward":0.1,"done":false}}
>> {"type":"step","data":{"guess":"crane"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Wordle: guess the hidden five-letter word in six tries. Each guess is scored letter by letter: G is the right letter in the right place, Y the right letter in the wrong place, X a letter the word does not have. A guess must be a word on the shared list; one that is not is scored and costs no attempt.\ncrane XXXYY\ncrane XXXYY\ncrane XXXYY\nGuess 4 of 6.\nReply with one five-letter word and nothing else.","feedback":"XXXYY","valid":true,"guesses":3,"history":"crane XXXYY\ncrane XXXYY\ncrane XXXYY","solved":false,"answer":""},"reward":0.1,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"guesses":3}}
>> {"type":"close"}
$ build/rel/bin/train wordle http://127.0.0.1:8805 --seed 7
train wordle — programmed deduction, epsilon-greedy over 40 openers
learns    only the opening word, scored by what the whole game returned
does not  learn the deduction: filtering the list by feedback is written here
server    http://127.0.0.1:8805 — WordleEnvironment
run       10000 games · seed 7 · eps 1.00->0.05 · 40 openers · 1466-word list

episodes          avg return   curve  (bars span 3.06 to 3.16)
    1- 1000             3.06
 1001- 2000             3.06
 2001- 3000             3.08   ######
 3001- 4000             3.09   ##########
 4001- 5000             3.11   ##################
 5001- 6000             3.10   ################
 6001- 7000             3.13   ############################
 7001- 8000             3.15   #####################################
 8001- 9000             3.15   ####################################
 9001-10000             3.16   ########################################

10000 episodes · 37843 steps · 0.9 s · 41457 steps/s · final avg 3.16
solved 99.0% in 3.76 guesses · favourite opener "arose" solves in 3.68 · worst of the 40 "fuzzy" 4.19
Plate 5. The deduction — filter the list by every piece of feedback so far, guess something still possible — is programmed, not learned; it solves 99.0% of games in 3.76 guesses and none of that is the curve. (That is the one place on this page the word solved appears: it is Wordle's own verb for finding the answer, said about the programmed part. No learner here is called solved.) The curve is the opening word, and this is also the plate where the bar scale matters most: the improvement is 3.06 to 3.16, and the header says so, so the forty hashes on the last row are the top of a tenth of a point and not a landslide. Two things had to be fixed before that tenth existed, neither of them the learner. The reward paid greens and yellows per guess, so a player who dawdled collected more wages than one who finished — the hand-driven episode above is that bug in three lines, guessing crane three times and being paid 0.1 each time for it. Solving now pays 0.50 more for every attempt it did not need. And the forty candidate openers were the forty best words in the list, which all solve in about the same number of guesses — there was nothing to choose between. Ranked by letter frequency and sampled across the whole ranking, the pool now runs from the best question in the dictionary to the worst, and from returns alone the trainer worked out that arose gets there in 3.68 guesses while fuzzy takes 4.19. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"guess": "crane"} — required, lowercased before it is judged
observationinput_text, feedback as five of G, Y and X, plus valid, guesses, history, solved, answer
rewarda tenth per green and a twentieth per yellow, one more for solving plus half an attempt's worth for every guess it did not need, a fifth back for a word not in the list
donesolved, six valid guesses, or the step cap
PLATE 6 · BLACKJACKindex

The textbook's own game, dealt over a socket

Sutton and Barto's Example 5.1, dealt over a socket: a deck that never runs out, one decision — hit or stand — and a dealer with no choices at all. It is the plate for a reward you can only sample, and for a curve that climbs and then stops below zero.

$ build/rel/bin/blackjack --host 127.0.0.1 --port 8806 &

$ build/rel/bin/oe_probe http://127.0.0.1:8806 --episode --steps 3 --action '{"hit":1}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Blackjack from an infinite deck. Your hand is 12; the dealer shows 1. You have no ace worth 11.\nOver 21 loses at once. On a stand the dealer draws to 17 and stands, soft 17 included.\nReply with one digit: 0 stand, 1 take a card.","player_sum":12,"dealer_up":1,"usable_ace":false,"dealer_sum":0},"reward":null,"done":false}}
>> {"type":"step","data":{"hit":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Blackjack from an infinite deck. Your hand is 17; the dealer shows 1. You have no ace worth 11.\nOver 21 loses at once. On a stand the dealer draws to 17 and stands, soft 17 included.\nReply with one digit: 0 stand, 1 take a card.","player_sum":17,"dealer_up":1,"usable_ace":false,"dealer_sum":0},"reward":0,"done":false}}
>> {"type":"step","data":{"hit":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Blackjack from an infinite deck. Your hand is 21; the dealer shows 1. You have no ace worth 11.\nOver 21 loses at once. On a stand the dealer draws to 17 and stands, soft 17 included.\nReply with one digit: 0 stand, 1 take a card.","player_sum":21,"dealer_up":1,"usable_ace":false,"dealer_sum":0},"reward":0,"done":false}}
>> {"type":"step","data":{"hit":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Blackjack from an infinite deck. Your hand is 31; the dealer shows 1. You have no ace worth 11.\nOver 21 loses at once. On a stand the dealer draws to 17 and stands, soft 17 included.\nThe hand is over: you 31, dealer 21.","player_sum":31,"dealer_up":1,"usable_ace":false,"dealer_sum":21},"reward":-1,"done":true}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"player_sum":31}}
>> {"type":"close"}
$ build/rel/bin/train blackjack http://127.0.0.1:8806 --seed 7
train blackjack — every-visit Monte Carlo control, 200 states x 2 actions
learns    when to hit and when to stand, from what whole hands paid
does not  hit under twelve by choice (that is a rule here), count cards, or win
server    http://127.0.0.1:8806 — BlackjackEnvironment
run       40000 hands · seed 7 · eps 1.00->0.05 · sample-average returns · no alpha, no gamma

hands             avg return   curve  (bars span -0.39 to -0.08)
    1- 4000            -0.39
 4001- 8000            -0.32   #########
 8001-12000            -0.29   #############
12001-16000            -0.25   ##################
16001-20000            -0.25   ##################
20001-24000            -0.21   #######################
24001-28000            -0.19   ##########################
28001-32000            -0.15   ###############################
32001-36000            -0.11   ####################################
36001-40000            -0.08   ########################################

40000 hands · 58170 steps · 1.2 s · 49238 steps/s · final avg -0.08
won 42.0% · pushed 7.7% · lost 50.3% over the last bucket · the dealer stands on every 17
Plate 6. Every other curve on this page ends higher than it started and means it. This one ends higher than it started and is still losing. That is the lesson. Sample-average returns over 200 states — no alpha, no gamma, because a hand is one undiscounted return — take the average from -0.39 to -0.08 across 40000 hands, and then stop. Against a dealer that stands on every 17, the best a by-the-book player earns is still a loss: learning here is losing less, and the plateau is where the truth is. The finale prints the split rather than implying it, and the last bucket won 42.0%, pushed 7.7% and lost 50.3%. One more thing the couplet says out loud: below twelve this trainer hits by RULE, the way the book does, because no single card can bust eleven. A programmed choice dressed up as a learned one is the one dishonesty this page exists to refuse. The hand above is not that policy: the probe knows one action and plays it. It is still what hitting always looks like in the end — twelve, seventeen, twenty-one, and the card that makes it 31. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"hit": 1} take a card, {"hit": 0} stand — required
observationinput_text, player_sum, dealer_up, usable_ace, and dealer_sum, which is 0 until the hand is over and the dealer's final total after
rewardinteger +1, 0 or -1, on the terminal step only; a non-terminal hit pays nothing
donea bust, or the dealer finishing once you stand
PLATE 7 · MAZEindex

The world is the seed

A maze of 225 squares, drawn as text, with you at the top left and the goal at the bottom right. It is not a fixture: every reset carves a new one out of the seed it was handed, so the observation below did not exist until it was asked for.

$ build/rel/bin/maze --host 127.0.0.1 --port 8807 &

$ build/rel/bin/oe_probe http://127.0.0.1:8807 --episode --steps 3 --action '{"move":1}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A 15x15 maze. '#' wall, '.' floor, 'A' you, 'G' the goal. A wall absorbs the move: the step is spent and you stay.\n###############\n#A..........#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############\nYou are at column 1, row 1; row 0 is the top. The goal is at column 13, row 13. Step 1 of 250.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","grid":"###############\n#A..........#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############","x":1,"y":1,"goal_x":13,"goal_y":13},"reward":null,"done":false}}
>> {"type":"step","data":{"move":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A 15x15 maze. '#' wall, '.' floor, 'A' you, 'G' the goal. A wall absorbs the move: the step is spent and you stay.\n###############\n#.A.........#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############\nYou are at column 2, row 1; row 0 is the top. The goal is at column 13, row 13. Step 2 of 250.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","grid":"###############\n#.A.........#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############","x":2,"y":1,"goal_x":13,"goal_y":13},"reward":-0.01,"done":false}}
>> {"type":"step","data":{"move":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A 15x15 maze. '#' wall, '.' floor, 'A' you, 'G' the goal. A wall absorbs the move: the step is spent and you stay.\n###############\n#..A........#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############\nYou are at column 3, row 1; row 0 is the top. The goal is at column 13, row 13. Step 3 of 250.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","grid":"###############\n#..A........#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############","x":3,"y":1,"goal_x":13,"goal_y":13},"reward":-0.01,"done":false}}
>> {"type":"step","data":{"move":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A 15x15 maze. '#' wall, '.' floor, 'A' you, 'G' the goal. A wall absorbs the move: the step is spent and you stay.\n###############\n#...A.......#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############\nYou are at column 4, row 1; row 0 is the top. The goal is at column 13, row 13. Step 4 of 250.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","grid":"###############\n#...A.......#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############","x":4,"y":1,"goal_x":13,"goal_y":13},"reward":-0.01,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"manhattan":21}}
>> {"type":"close"}
$ build/rel/bin/train maze http://127.0.0.1:8807 --seed 7
train maze — tabular Q-learning over position, 225 states x 4 actions
learns    the way through THIS maze, by bumping into every wall of it
does not  carry any of it to another seed: a table of squares is one map
server    http://127.0.0.1:8807 — MazeEnvironment
run       2000 episodes · seed 7 · eps 1.00->0.05 · alpha 0.20 · gamma 0.95 · one maze, replayed every episode

episodes          avg return   curve  (bars span -2.45 to 0.70)
   1- 200              -2.45
 201- 400              -1.32   ##############
 401- 600              -0.26   ############################
 601- 800               0.11   #################################
 801-1000               0.32   ###################################
1001-1200               0.45   #####################################
1201-1400               0.54   ######################################
1401-1600               0.62   #######################################
1601-1800               0.66   ########################################
1801-2000               0.70   ########################################

2000 episodes · 187862 steps · 3.0 s · 63405 steps/s · final avg 0.70
shortest path 28 steps · best walk 28 · first bucket averaged 248 steps to done, last 31
Plate 7. The carve is an iterative depth-first backtracker, driven entirely by the reset seed. That there is a way out is a theorem about it rather than a hope: the walk leaves a spanning tree of the cells, so exactly one corridor path joins any two of them. Start-to-goal is walkable for every seed there is, and the environment's own description says so — a procedurally generated world that cannot promise an exit is a slot machine wearing a maze costume. It is also why this is the one trainer here that does not advance its seed between episodes. A table indexed by position is a map of ONE maze, and this plate is about watching a map get drawn. It takes 248 steps to blunder into the goal over the first two hundred episodes, and 31 over the last two hundred. The shortest path is 28, measured by the trainer's own breadth-first search over the grid it had already been shown — reporting rather than playing, the same licence the bandit's ladder line gets. Hand it a different --seed and it is lost again, which is the honest reading of "procedurally generated". — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"move": 0..3} — up, right, down, left; required. Walking into a wall stays put and costs the step: a bump is how a blind walker finds a wall, not an error
observationinput_text, grid as text rows, plus x, y, goal_x, goal_y
reward1.0 for stepping onto the goal, -0.01 for every other step, bumps included — sparse on purpose, because the signal lives at the end
donethe goal, or the step cap
PLATE 8 · PONGindex

The most famous rally in computing, one paddle of it

A ball on a unit court, a paddle on the left wall, and three choices: up, stay, down. Where the ball meets the paddle decides where it goes next, so placement is worth learning, and what the curve measures is the rally itself.

$ build/rel/bin/pong --host 127.0.0.1 --port 8808 &

$ build/rel/bin/oe_probe http://127.0.0.1:8808 --episode --steps 3 --action '{"paddle":1}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Half a game of Pong: one paddle on the left wall of a square court, y = 0 at the top. Return the ball; a miss ends the episode.\nBall at x 0.500, y 0.730, moving -0.040 across and -0.004 down. Your paddle centre 0.500, half-height 0.080, travel 0.050 a step. Rally 0. Three decimals here.\nReply with one digit: 0 up, 1 stay, 2 down.","ball_x":0.5,"ball_y":0.7299864849281856,"ball_vx":-0.04,"ball_vy":-0.004108320177089402,"paddle_y":0.5,"rally":0},"reward":null,"done":false}}
>> {"type":"step","data":{"paddle":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Half a game of Pong: one paddle on the left wall of a square court, y = 0 at the top. Return the ball; a miss ends the episode.\nBall at x 0.460, y 0.726, moving -0.040 across and -0.004 down. Your paddle centre 0.500, half-height 0.080, travel 0.050 a step. Rally 0. Three decimals here.\nReply with one digit: 0 up, 1 stay, 2 down.","ball_x":0.46,"ball_y":0.7258781647510962,"ball_vx":-0.04,"ball_vy":-0.004108320177089402,"paddle_y":0.5,"rally":0},"reward":0.0,"done":false}}
>> {"type":"step","data":{"paddle":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Half a game of Pong: one paddle on the left wall of a square court, y = 0 at the top. Return the ball; a miss ends the episode.\nBall at x 0.420, y 0.722, moving -0.040 across and -0.004 down. Your paddle centre 0.500, half-height 0.080, travel 0.050 a step. Rally 0. Three decimals here.\nReply with one digit: 0 up, 1 stay, 2 down.","ball_x":0.42000000000000004,"ball_y":0.7217698445740068,"ball_vx":-0.04,"ball_vy":-0.004108320177089402,"paddle_y":0.5,"rally":0},"reward":0.0,"done":false}}
>> {"type":"step","data":{"paddle":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Half a game of Pong: one paddle on the left wall of a square court, y = 0 at the top. Return the ball; a miss ends the episode.\nBall at x 0.380, y 0.718, moving -0.040 across and -0.004 down. Your paddle centre 0.500, half-height 0.080, travel 0.050 a step. Rally 0. Three decimals here.\nReply with one digit: 0 up, 1 stay, 2 down.","ball_x":0.38000000000000006,"ball_y":0.7176615243969174,"ball_vx":-0.04,"ball_vy":-0.004108320177089402,"paddle_y":0.5,"rally":0},"reward":0.0,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"rally":0}}
>> {"type":"close"}
$ build/rel/bin/train pong http://127.0.0.1:8808 --seed 7
train pong — tabular Q-learning, 126 boxes x 3 actions
learns    to be where the ball will arrive, from the gap it can see now
does not  aim its returns, read the english it puts on, or see the court
server    http://127.0.0.1:8808 — PongEnvironment
run       2500 episodes · seed 7 · eps 1.00->0.05 · alpha 0.20 · gamma 0.95

episodes          avg return   curve  (bars span -0.77 to 4.13)
   1- 250              -0.77
 251- 500              -0.70   #
 501- 750              -0.51   ##
 751-1000              -0.41   ###
1001-1250              -0.22   ####
1251-1500               0.04   #######
1501-1750               0.42   ##########
1751-2000               1.06   ###############
2001-2250               2.77   #############################
2251-2500               4.13   ########################################

2500 episodes · 230050 steps · 4.9 s · 46729 steps/s · final avg 4.13
longest rally 29 returns · first bucket averaged -0.8, last 4.1 · 84 of 126 boxes visited
Plate 8. There is no second number on this plate, and that is the point of it. The reward is +1.0 on the step the paddle returns the ball and -1.0 on the step it misses, and a miss ends the episode. So an episode's return is its rally minus the one serve it dropped: the column marked avg return going from -0.77 to 4.13 is the rally growing. No proxy, no invented score, nothing to argue with. The longest single rally in the run was 29 returns. It gets there on 126 boxes and visited 84 of them: the ball-to-paddle gap in seven slices, the ball's distance in three, and the sign of each velocity. That is enough to be where the ball will arrive. It is not enough to decide where to send it back, and this trainer never tries. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"paddle": 0|1|2} — up, stay, down; defaults to stay
observationinput_text, ball_x, ball_y, ball_vx, ball_vy, paddle_y, and rally, the returns so far this episode
reward1.0 the step it returns the ball, -1.0 the step it misses, 0.0 otherwise
donea miss, or the step cap
PLATE 9 · CONNECT4index

The opponent lives in the env, and legal_actions is just a field

Seven columns, six rows, and an opponent that is part of the environment: you drop a disc, and before the observation comes back it has dropped one too. It wins if it can and blocks if it must, so what the curve shows is a learner discovering that stopping a four is worth about as much as taking one.

$ build/rel/bin/connect4 --host 127.0.0.1 --port 8809 &

$ build/rel/bin/oe_probe http://127.0.0.1:8809 --episode --steps 3 --action '{"col":3}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Connect Four, seven columns by six rows. You are 'X' and move first; the environment is 'O' and answers inside your step. Four in a line wins.\n0123456\n.......\n.......\n.......\n.......\n.......\n.......\nColumns with room: 0 1 2 3 4 5 6. It has not answered yet.\nReply with one digit, the column to drop into.","board":".......\n.......\n.......\n.......\n.......\n.......","legal_actions":[0,1,2,3,4,5,6],"last_env_col":-1},"reward":null,"done":false}}
>> {"type":"step","data":{"col":3}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Connect Four, seven columns by six rows. You are 'X' and move first; the environment is 'O' and answers inside your step. Four in a line wins.\n0123456\n.......\n.......\n.......\n.......\n.......\n..OX...\nColumns with room: 0 1 2 3 4 5 6. It last played column 2.\nReply with one digit, the column to drop into.","board":".......\n.......\n.......\n.......\n.......\n..OX...","legal_actions":[0,1,2,3,4,5,6],"last_env_col":2},"reward":0,"done":false}}
>> {"type":"step","data":{"col":3}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Connect Four, seven columns by six rows. You are 'X' and move first; the environment is 'O' and answers inside your step. Four in a line wins.\n0123456\n.......\n.......\n.......\n.......\n...X...\n.OOX...\nColumns with room: 0 1 2 3 4 5 6. It last played column 1.\nReply with one digit, the column to drop into.","board":".......\n.......\n.......\n.......\n...X...\n.OOX...","legal_actions":[0,1,2,3,4,5,6],"last_env_col":1},"reward":0,"done":false}}
>> {"type":"step","data":{"col":3}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Connect Four, seven columns by six rows. You are 'X' and move first; the environment is 'O' and answers inside your step. Four in a line wins.\n0123456\n.......\n.......\n...O...\n...X...\n...X...\n.OOX...\nColumns with room: 0 1 2 3 4 5 6. It last played column 3.\nReply with one digit, the column to drop into.","board":".......\n.......\n...O...\n...X...\n...X...\n.OOX...","legal_actions":[0,1,2,3,4,5,6],"last_env_col":3},"reward":0,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"discs":6}}
>> {"type":"close"}
$ build/rel/bin/train connect4 http://127.0.0.1:8809 --seed 7
train connect4 — afterstate values, 72 positions after the drop, masked
learns    which column leaves the better position: taking a four, and stopping one
does not  see a fork coming, plan two moves ahead, or see most of the board
server    http://127.0.0.1:8809 — Connect4Environment
run       5000 games · seed 7 · eps 1.00->0.05 · alpha 0.15 · gamma 0.95 · masked by legal_actions

episodes          avg return   curve  (bars span -0.85 to -0.03)
   1- 500              -0.85
 501-1000              -0.80   ###
1001-1500              -0.79   ###
1501-2000              -0.66   ##########
2001-2500              -0.61   ############
2501-3000              -0.50   #################
3001-3500              -0.34   #########################
3501-4000              -0.23   ##############################
4001-4500              -0.09   #####################################
4501-5000              -0.03   ########################################

5000 episodes · 56020 steps · 1.1 s · 49172 steps/s · final avg -0.03
won 45.6% · lost 48.8% · drew 5.6% over the last bucket · blocked 76.4% of the threats it was shown
Plate 9. Read the observation again: legal_actions. The protocol has never heard of that field. This environment invented it, because its action space shrinks as columns fill. The schema advertises it exactly the way it advertises board, and the trainer pulls it out of the observation like any other field and masks its argmax with it. That is the whole demonstration: masked by legal_actions is in the run line above, and that array is the mask's only source. Computing legality client-side from the rules would have been easy, and would have quietly ended it — the protocol carries observations, and what an observation means is the environment's own business. Upstream's connect4 environment publishes a legal set too, because its author had the same problem, not because a standard told either of them to. It is a real constraint and not a decoration: a col that is in range but full is still refused, with the protocol's EXECUTION_ERROR frame. That is the arcade's first error that depends on the state and not on the argument. What the mask bought, over the last five hundred games: 45.6% won, 48.8% lost, 5.6% drawn, and 76.4% of the threats it was shown blocked in the column it was shown them in. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"col": 0..6} — required. A full column is in range and still refused, with an EXECUTION_ERROR frame
observationinput_text, board as six text rows, top row first, plus legal_actions — the columns not yet full — and last_env_col, where the env just played
rewardinteger on the terminal step: +1 your four, -1 its four, 0 a full board
doneeither four, or the board fills
PLATE 10 · KUHNindex

The smallest real poker, and a chart that wins downward

Three cards, one chip each in the pot, and one decision per player: pass, or bet a chip. One session deals both seats, and the observation only ever describes the seat to act — so the other player's card stays off the wire until the hand is over.

$ build/rel/bin/kuhn --host 127.0.0.1 --port 8810 &

$ build/rel/bin/oe_probe http://127.0.0.1:8810 --episode --steps 3 --action '{"bet":1}' --past-done | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Kuhn poker, three cards: 1 jack, 2 queen, 3 king. One to each seat, one face down, a chip anted by both. You play both seats in turn.\nYou are seat 0 and hold the jack. Pot 2. Betting so far: nothing.\nFacing a bet, 0 folds and 1 calls. Pass-pass or a call goes to showdown, and the higher card takes half the pot.\nReply with one digit: 0 pass, 1 bet one chip.","to_act":0,"card":1,"history":"","pot":2,"deal":""},"reward":null,"done":false}}
>> {"type":"step","data":{"bet":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Kuhn poker, three cards: 1 jack, 2 queen, 3 king. One to each seat, one face down, a chip anted by both. You play both seats in turn.\nYou are seat 1 and hold the king. Pot 3. Betting so far: bet.\nFacing a bet, 0 folds and 1 calls. Pass-pass or a call goes to showdown, and the higher card takes half the pot.\nReply with one digit: 0 pass, 1 bet one chip.","to_act":1,"card":3,"history":"b","pot":3,"deal":""},"reward":0,"done":false}}
>> {"type":"step","data":{"bet":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Kuhn poker, three cards: 1 jack, 2 queen, 3 king. One to each seat, one face down, a chip anted by both. You play both seats in turn.\nYou are seat 0 and hold the jack. Pot 4. Betting so far: bet, bet.\nFacing a bet, 0 folds and 1 calls. Pass-pass or a call goes to showdown, and the higher card takes half the pot.\nThe hand is over. Both cards: JK.","to_act":0,"card":1,"history":"bb","pot":4,"deal":"JK"},"reward":-2,"done":true}}
>> {"type":"step","data":{"bet":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Kuhn poker, three cards: 1 jack, 2 queen, 3 king. One to each seat, one face down, a chip anted by both. You play both seats in turn.\nYou are seat 0 and hold the jack. Pot 4. Betting so far: bet, bet.\nFacing a bet, 0 folds and 1 calls. Pass-pass or a call goes to showdown, and the higher card takes half the pot.\nThe hand is over. Both cards: JK.","to_act":0,"card":1,"history":"bb","pot":4,"deal":"JK"},"reward":null,"done":true}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":2,"pot":4}}
>> {"type":"close"}
$ build/rel/bin/train kuhn http://127.0.0.1:8810 --seed 7
train kuhn — outcome-sampling regret matching, 12 information sets
learns    a mixed strategy for both seats, from what one sampled hand paid
does not  reach Nash, see the other seat's card, or ever stop exploring
server    http://127.0.0.1:8810 — KuhnPokerEnvironment
run       60000 hands · seed 7 · regret matching · 0.10 exploration floor · 12 information sets

hands             exploitability   curve  (bars span 0.13 down to 0.02)
    1- 6000             0.13
 6001-12000             0.05   ############################
12001-18000             0.04   #################################
18001-24000             0.02   ########################################
24001-30000             0.03   #####################################
30001-36000             0.03   ######################################
36001-42000             0.03   ####################################
42001-48000             0.05   ###############################
48001-54000             0.06   ###########################
54001-60000             0.04   #################################

60000 hands · 143480 steps · 2.6 s · 54924 steps/s · final avg 0.04
exploitability 0.041 chips a hand · seat 0 averaged -0.060 (the equilibrium pays -1/18) · bluffs a jack 15% · bets a king 41% · calls a bet with the queen 35% (one in three at equilibrium)
Plate 10. Every other chart here reads a tall bar as a good one. This one measures exploitability: how much a player who knew this strategy exactly could take off it, in chips per hand. Smaller is better, so the bars had to be told. They fill toward the run's own best bucket, and the header writes the span top-first — bars span 0.13 down to 0.02. Forty hashes still means the best this run reached, and the words down to say which way that is. What falls is a distance, never an arrival. The run ends at 0.041 chips a hand and is still moving, so the strongest word this page may use is toward. The learner is different in kind as well: regret matching rather than a value table, playing a mixed strategy and accumulating the average of it. The average is the object that converges; the current strategy never has to. The frequencies it settled on are the game's own known shapes. It bluffs a jack 15% of the time and bets a king 41%, where the known equilibria bet the king three times as often as they bluff the jack. It calls with the queen 35% of the time against one in three at equilibrium, and seat 0 averaged -0.060 a hand in a game whose equilibrium pays -1/18. One last thing, in the hand above rather than the chart: the third bet is taken after done came back true, which is what --past-done is for. The answer is the whole after-done rule in one frame — the terminal observation byte for byte, reward null, done still true — and the state door still says step_count 2, because a step past the end changes nothing, the count included. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"bet": 0} pass or {"bet": 1} bet a chip — required
observationinput_text, to_act, card — that seat's card and never the other's — history as a string of p and b, pot, and deal, empty until the hand is over
rewardinteger, seat 0's chips, on the terminal step; the game is zero-sum and the trainer flips the sign for seat 1 itself
donea fold, pass-pass, or a called bet
THE SYNTH LAB · PLATES 11-14index

Four tasks that are not games

The four plates that follow are not games. Each server mints a problem as text, computes its own answer in the same call, and pays reward 1 for an exact match and 0 for anything else. Four tasks, four skills: dyna is state tracking, inducto is rule induction, logic is circuit evaluation, and filtro is source separation.

Reset is a generator, and every reset mints a fresh task from its seed. The runs below open with the measurement: a thousand seeds, a thousand distinct tasks, no repeats, at tens of thousands of tasks a second over the wire. The same seed twice is byte-identical. Difficulty is a knob on reset (1, 2 or 3). Dyna, logic and filtro sweep all three and print a row apiece, so the table names what the knob moves. Inducto holds it at 3 and spends its run on a learning curve instead.

Each task is a prompt, an answer, and a reward the environment can defend — the verifiable-reward shape a GRPO-style trainer consumes. (GRPO: sample several answers per prompt, score each, push the model toward the ones the reward preferred; TRL ↗'s GRPOTrainer is one.) Each generator was measured against a battery of cheap readers — answer by position, by output length, by one token, by the previous answer — and cut until those shortcuts price at chance. Each caption below names what its task resists, and the battery prints the ledger. To train a language model against them, start at docs/TRAINING.md.

Three of the four runs below print learns nothing on their own second line. That is because these are text-in, text-out tasks for a language model, not value-table games. The client parses the text and computes the answer. That proves the reward channel end to end, and prices what guessing pays beside it. Only inducto has a decision buried in it, and only inducto gets a curve.

PLATE 11 · DYNA · STATE TRACKINGindex

State tracking: the answer exists only after the replay

State tracking is holding a changing arrangement in your head, and this server grades exactly that. A list of symbols is shuffled, then a run of operations is applied to it — swap two, rotate left, rotate right, reverse a segment — and three questions are asked about the arrangement that comes out: what is standing at this position, or where did this element end up. The answers are nowhere in the text. They exist only after somebody replays every operation in order, which is what makes this state tracking and not a lookup.

$ build/rel/bin/dyna --host 127.0.0.1 --port 8821 &

$ build/rel/bin/oe_probe http://127.0.0.1:8821 --episode --steps 4 --action '{"answer":"B"}' --past-done | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"STATE [C, F, A, D, E, B] || OPS SWAP(5,0) | ROTATE_LEFT(2) | REVERSE(2,3) | REVERSE(2,3) | SWAP(4,5) | ROTATE_LEFT(4) | SWAP(3,5) | SWAP(4,3) | ROTATE_RIGHT(2) | ROTATE_LEFT(5) | ROTATE_LEFT(3) | REVERSE(2,5) | REVERSE(0,3) | ROTATE_RIGHT(1) | REVERSE(0,5) | ROTATE_LEFT(1) | ROTATE_LEFT(5) | ROTATE_LEFT(2) | REVERSE(3,5) | ROTATE_LEFT(5) | SWAP(0,2) | REVERSE(0,5) | REVERSE(0,2) | ROTATE_RIGHT(4) | SWAP(4,0) | REVERSE(2,4) | ROTATE_LEFT(1) | SWAP(1,4) | ROTATE_RIGHT(3) | SWAP(0,1) || QUERY element D ->","difficulty":1,"state_size":6,"num_ops":30,"query_index":1,"answer":"","final_state":""},"reward":null,"done":false}}
>> {"type":"step","data":{"answer":"B"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"STATE [C, F, A, D, E, B] || OPS SWAP(5,0) | ROTATE_LEFT(2) | REVERSE(2,3) | REVERSE(2,3) | SWAP(4,5) | ROTATE_LEFT(4) | SWAP(3,5) | SWAP(4,3) | ROTATE_RIGHT(2) | ROTATE_LEFT(5) | ROTATE_LEFT(3) | REVERSE(2,5) | REVERSE(0,3) | ROTATE_RIGHT(1) | REVERSE(0,5) | ROTATE_LEFT(1) | ROTATE_LEFT(5) | ROTATE_LEFT(2) | REVERSE(3,5) | ROTATE_LEFT(5) | SWAP(0,2) | REVERSE(0,5) | REVERSE(0,2) | ROTATE_RIGHT(4) | SWAP(4,0) | REVERSE(2,4) | ROTATE_LEFT(1) | SWAP(1,4) | ROTATE_RIGHT(3) | SWAP(0,1) || QUERY position 1 ->","difficulty":1,"state_size":6,"num_ops":30,"query_index":2,"answer":"","final_state":""},"reward":0,"done":false}}
>> {"type":"step","data":{"answer":"B"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"STATE [C, F, A, D, E, B] || OPS SWAP(5,0) | ROTATE_LEFT(2) | REVERSE(2,3) | REVERSE(2,3) | SWAP(4,5) | ROTATE_LEFT(4) | SWAP(3,5) | SWAP(4,3) | ROTATE_RIGHT(2) | ROTATE_LEFT(5) | ROTATE_LEFT(3) | REVERSE(2,5) | REVERSE(0,3) | ROTATE_RIGHT(1) | REVERSE(0,5) | ROTATE_LEFT(1) | ROTATE_LEFT(5) | ROTATE_LEFT(2) | REVERSE(3,5) | ROTATE_LEFT(5) | SWAP(0,2) | REVERSE(0,5) | REVERSE(0,2) | ROTATE_RIGHT(4) | SWAP(4,0) | REVERSE(2,4) | ROTATE_LEFT(1) | SWAP(1,4) | ROTATE_RIGHT(3) | SWAP(0,1) || QUERY element E ->","difficulty":1,"state_size":6,"num_ops":30,"query_index":3,"answer":"","final_state":""},"reward":1,"done":false}}
>> {"type":"step","data":{"answer":"B"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"STATE [C, F, A, D, E, B] || OPS SWAP(5,0) | ROTATE_LEFT(2) | REVERSE(2,3) | REVERSE(2,3) | SWAP(4,5) | ROTATE_LEFT(4) | SWAP(3,5) | SWAP(4,3) | ROTATE_RIGHT(2) | ROTATE_LEFT(5) | ROTATE_LEFT(3) | REVERSE(2,5) | REVERSE(0,3) | ROTATE_RIGHT(1) | REVERSE(0,5) | ROTATE_LEFT(1) | ROTATE_LEFT(5) | ROTATE_LEFT(2) | REVERSE(3,5) | ROTATE_LEFT(5) | SWAP(0,2) | REVERSE(0,5) | REVERSE(0,2) | ROTATE_RIGHT(4) | SWAP(4,0) | REVERSE(2,4) | ROTATE_LEFT(1) | SWAP(1,4) | ROTATE_RIGHT(3) | SWAP(0,1) || QUERY element E ->","difficulty":1,"state_size":6,"num_ops":30,"query_index":3,"answer":"0","final_state":"[E, B, C, A, F, D]"},"reward":0,"done":true}}
>> {"type":"step","data":{"answer":"B"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"STATE [C, F, A, D, E, B] || OPS SWAP(5,0) | ROTATE_LEFT(2) | REVERSE(2,3) | REVERSE(2,3) | SWAP(4,5) | ROTATE_LEFT(4) | SWAP(3,5) | SWAP(4,3) | ROTATE_RIGHT(2) | ROTATE_LEFT(5) | ROTATE_LEFT(3) | REVERSE(2,5) | REVERSE(0,3) | ROTATE_RIGHT(1) | REVERSE(0,5) | ROTATE_LEFT(1) | ROTATE_LEFT(5) | ROTATE_LEFT(2) | REVERSE(3,5) | ROTATE_LEFT(5) | SWAP(0,2) | REVERSE(0,5) | REVERSE(0,2) | ROTATE_RIGHT(4) | SWAP(4,0) | REVERSE(2,4) | ROTATE_LEFT(1) | SWAP(1,4) | ROTATE_RIGHT(3) | SWAP(0,1) || QUERY element E ->","difficulty":1,"state_size":6,"num_ops":30,"query_index":3,"answer":"0","final_state":"[E, B, C, A, F, D]"},"reward":null,"done":true}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"num_ops":30}}
>> {"type":"close"}
$ build/rel/bin/synthlab dyna http://127.0.0.1:8821 --seed 7
synthlab dyna — programmed replay, a seeded chance arm, presets 1-3 swept, three queries a task
learns    nothing — the replay is programmed: parse the text, apply the ops, answer
does not  dress computation as learning; the chance arm beside it is what guessing pays
server    http://127.0.0.1:8821 — DynaEnvironment
run       1500 solver episodes · 1500 chance episodes · seed 7 · 500 per preset

demo      1000 seeds -> 1000 distinct tasks · 0 repeats · distinct by 64-bit FNV-1a over input_text
          seed 7 twice -> byte-identical
          62171 tasks/s over the WebSocket

preset  knobs                                         exact match   chance answers/s
     1  state 5-7, ops 20-30                            1500/1500 252/1500     55313
     2  state 8-12, ops 25-40                           1500/1500 165/1500     52369
     3  state 13-20, ops 40-60                          1500/1500  97/1500     44577

3000 tasks · 9000 steps · 0.2 s · 46154 steps/s
answered 4500/4500 · chance 514/4500 · two implementations agreed on every query
Plate 11. Read the three demo lines before the table, because they are the whole reason this wing exists. A thousand resets produced a thousand distinct input_text values with zero repeats — and the line names its own judge, distinct by 64-bit FNV-1a over input_text, because a hash equality is the check that actually ran. Reset the same seed twice and the two texts are byte-identical, so the distribution is a faucet and not a slot machine. The episode is three steps long and the hand-driven exchange above is why: one arrangement, three questions, a step each — query_index counts 1, 2, 3 while STATE and OPS hold the same bytes and only the QUERY tail moves — then a fourth step past the end, frozen, which is the after-done rule plate 10 explains. Each answer is graded on its own step, so a task pays 0 to 3 and the sweep's columns count answers: a programmed replay took all 4500, and a guess of the right shape — a uniform element or a uniform index, to match whichever kind of question was asked — took 514. What this task resists is answer reuse. No two of the three queries name the same place in the final arrangement, so copying the previous answer scores 0.0% here. That zero is bought, not free: the guesser that inverts it, crossing the previous answers off and picking uniformly, is paid +0.9 points at difficulty 1 for it, and the battery prints that row (uniform-minus-previous) beside the zero rather than resting on the zero alone. It resists skimming the operations too. A reader that sums the rotations and ignores SWAP and REVERSE — no list tracking at all — is inside half a point of chance at every level: 16.4%, 9.8% and 6.3% against 17.0%, 10.2% and 6.2%. A short run of permutations does not mix, which is why the presets draw 20-30, 25-40 and 40-60 operations. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"answer": "B"} — a string. There is no illegal answer, only a wrong one: a missing field reads as the empty string and scores zero. A wrong-typed one never gets that far — {"answer": 5} is a VALIDATION_ERROR from the framework, because a string field takes no coercion, and the environment never sees the step
reset{"seed": N, "difficulty": 1|2|3} — the difficulty moves state_size and num_ops; anything that is not 1, 2 or 3 means 1
observationinput_text, difficulty, state_size, num_ops, query_index — which of the three questions you are holding — and answer and final_state, both empty until done, because a reveal after the first question would answer the other two
rewardinteger 1 on exact match after whitespace is collapsed, 0 otherwise, per step. No partial credit inside a question and no shaping; an episode pays 0 to 3
donethe third answer. Three steps need the WebSocket: POST /step builds a fresh environment per call and would answer question one every time
PLATE 12 · INDUCTO · RULE INDUCTIONindex

Rule induction: the filter is programmed; only the bet is learned

Rule induction is naming a hidden transformation from worked examples of it. Here the hidden rule is drawn from a fixed inventory — reverse, rotate, drop, interleave, sort, mirror, a Caesar shift, and composites of those — applied to a few short sequences of letters, which you are shown before being handed one more sequence to transform. The shots do not always pin the rule down. That gap is the task.

$ build/rel/bin/inducto --host 127.0.0.1 --port 8822 &

$ build/rel/bin/oe_probe http://127.0.0.1:8822 --episode --steps 3 --action '{"answer":"A B C"}' --past-done | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ex1: [H A D A J A B G D B H] -> [H H H A A D D A A J J A A B B G G D D B B H] | Ex2: [F C C E J B I A B J A] -> [A F F C C C C E E J J B B I I A A B B J J A] | Ex3: [I I B F E J H F] -> [F I I I I B B F F E E J J H H F] | Ex4: [H B B A C A F E J] -> [J H H B B B B A A C C A A F F E E J] | Ex5: [F F E F G J B E D J A] -> [A F F F F E E F F G G J J B B E E D D J J A] | Query: [B J G J] ->","difficulty":1,"num_examples":5,"answer":"","rule":"","rule_complexity":0},"reward":null,"done":false}}
>> {"type":"step","data":{"answer":"A B C"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ex1: [H A D A J A B G D B H] -> [H H H A A D D A A J J A A B B G G D D B B H] | Ex2: [F C C E J B I A B J A] -> [A F F C C C C E E J J B B I I A A B B J J A] | Ex3: [I I B F E J H F] -> [F I I I I B B F F E E J J H H F] | Ex4: [H B B A C A F E J] -> [J H H B B B B A A C C A A F F E E J] | Ex5: [F F E F G J B E D J A] -> [A F F F F E E F F G G J J B B E E D D J J A] | Query: [B J G J] ->","difficulty":1,"num_examples":5,"answer":"J B B J J G G J","rule":"DUPLICATE_EACH_THEN_ROTATE_RIGHT_1","rule_complexity":1},"reward":0,"done":true}}
>> {"type":"step","data":{"answer":"A B C"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ex1: [H A D A J A B G D B H] -> [H H H A A D D A A J J A A B B G G D D B B H] | Ex2: [F C C E J B I A B J A] -> [A F F C C C C E E J J B B I I A A B B J J A] | Ex3: [I I B F E J H F] -> [F I I I I B B F F E E J J H H F] | Ex4: [H B B A C A F E J] -> [J H H B B B B A A C C A A F F E E J] | Ex5: [F F E F G J B E D J A] -> [A F F F F E E F F G G J J B B E E D D J J A] | Query: [B J G J] ->","difficulty":1,"num_examples":5,"answer":"J B B J J G G J","rule":"DUPLICATE_EACH_THEN_ROTATE_RIGHT_1","rule_complexity":1},"reward":null,"done":true}}
>> {"type":"step","data":{"answer":"A B C"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ex1: [H A D A J A B G D B H] -> [H H H A A D D A A J J A A B B G G D D B B H] | Ex2: [F C C E J B I A B J A] -> [A F F C C C C E E J J B B I I A A B B J J A] | Ex3: [I I B F E J H F] -> [F I I I I B B F F E E J J H H F] | Ex4: [H B B A C A F E J] -> [J H H B B B B A A C C A A F F E E J] | Ex5: [F F E F G J B E D J A] -> [A F F F F E E F F G G J J B B E E D D J J A] | Query: [B J G J] ->","difficulty":1,"num_examples":5,"answer":"J B B J J G G J","rule":"DUPLICATE_EACH_THEN_ROTATE_RIGHT_1","rule_complexity":1},"reward":null,"done":true}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":1,"num_examples":5}}
>> {"type":"close"}
$ build/rel/bin/synthlab inducto http://127.0.0.1:8822 --seed 7
synthlab inducto — programmed consistency filter, sample-average prior over 52 rules
learns    which consistent rule to bet on when the shots leave a tie, from reward alone
does not  see the rule name before done, or meet a rule outside the generator's 52
server    http://127.0.0.1:8822 — InductoEnvironment
run       60000 tasks · seed 7 · difficulty 3 · eps 1.00->0.05 · 52 sample averages

demo      1000 seeds -> 1000 distinct tasks · 0 repeats · distinct by 64-bit FNV-1a over input_text
          seed 7 twice -> byte-identical
          71847 tasks/s over the WebSocket

ties              exact match   curve  (bars span 41.27 to 64.02)
   1- 189              41.27
 190- 378              48.15   ############
 379- 567              48.68   #############
 568- 756              58.20   ##############################
 757- 945              57.14   ############################
 946-1134              51.32   ##################
1135-1323              58.20   ##############################
1324-1512              58.20   ##############################
1513-1701              62.43   #####################################
1702-1890              64.02   ########################################

60000 tasks · 60000 steps · 1.5 s · 39520 steps/s · answered 59145/60000
1890 ties · 3.1% of tasks (consistent rules disagreed on the query) · first bucket 41.3% · last 64.0% · complexity-1 rules priced at 0.19 over 286 bets
tie oracles · uniform draw 45.3% · plurality 46.1% · draw-weighted 68.0%, the Bayes ceiling (difficulty 3 mixes a 12-rule and a 20-rule pool half and half, so a complexity-3 survivor is 5/3 the bet a complexity-2 one is)
Plate 12. This is the wing's one learning curve, and it plots ties: the tasks where more than one rule survived the filter and the survivors disagreed about the query, which is where a bet is the only thing that answers. Everything else is deduction and was already right. The first bucket is its own control — epsilon starts at 1.00, so those ties were broken by drawing uniformly among survivors — and the exact-match rate on the ties climbs from 41.3% to 64.0% as a sample-average prior learns which survivor to bet on from reward alone. Read the last line before believing the curve: the run measures three tie oracles it never lets the learner see, and the draw-weighted one — the Bayes ceiling, because difficulty 3 draws a complexity-3 rule five times for every three complexity-2 rules — pays 68.0%. The curve is climbing toward that and has not reached it. Bucket size is 189 ties, and a bucket that small still swings ten points, which is why the shape matters and no single bucket does. What this task resists is naming the rule by output length alone. The inventory declares fifty-two rules, placed so that every output length in every pool has at least two producers — the tightest class in any pool holds exactly two. So length alone identifies the rule on 0.0% of tasks, and the ties that closure creates — 3.1% of tasks at difficulty 3 — are what this curve is drawn from. What the closure does not buy is worth saying in the same breath, because a closed inventory of fifty-two is only about 5.7 bits: keep each output's length and the letter at each end and the rule is still named on 79.9% of difficulty-3 tasks; keep every letter but throw the order away and it is named on 10.2%. Length alone names nothing, which was the shortcut to close; the rest is evidence a solver is supposed to use. And the shot range runs 1-3 at the hardest difficulty, so a one-shot prompt is real — 33.6% of difficulty-3 tasks — and eleven of every twelve of the ties this curve is drawn from come from exactly those draws. Raising the floor was pre-registered in arcade/CHARTER.md and then declined in writing, because it would have deleted the only decision this plate ever plots. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"answer": "A B C"} — the transformed sequence, space-separated. Every string is a legal answer; only the target is right
reset{"seed": N, "difficulty": 1|2|3} — the difficulty moves the shot count and which complexity pools the rule is drawn from
observationinput_text, difficulty, num_examples, and answer, rule and rule_complexity — the name arrives only after done
rewardinteger 1 on exact match, 0 otherwise
donethe answer step
PLATE 13 · LOGIC · CIRCUIT EVALUATIONindex

Circuit evaluation: two implementations of one specification, agreeing

Circuit evaluation is running a printed circuit in your head, gate by gate. A small boolean circuit is printed as text: input bits, then gates — AND, OR, XOR, NAND, NOR, NOT — each reading inputs or earlier gates, and one gate is queried. The answer is one bit, which makes a coin worth exactly half and makes the interesting number the one the coin does not get.

$ build/rel/bin/logic --host 127.0.0.1 --port 8823 &

$ build/rel/bin/oe_probe http://127.0.0.1:8823 --episode --steps 3 --action '{"answer":"1"}' --past-done | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"INPUTS x0=0 x1=1 x2=0 x3=1 || GATES g0=XOR(x1,x0) | g1=XOR(x0,x2) | g2=AND(x3,g0) | g3=OR(g0,x3) | g4=OR(g0,g1) | g5=XOR(g2,g4) | g6=AND(g2,g4) || QUERY g6 ->","difficulty":1,"num_inputs":4,"depth":2,"width":3,"num_gates":7,"cone_gates":5,"xor_count":3,"answer":""},"reward":null,"done":false}}
>> {"type":"step","data":{"answer":"1"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"INPUTS x0=0 x1=1 x2=0 x3=1 || GATES g0=XOR(x1,x0) | g1=XOR(x0,x2) | g2=AND(x3,g0) | g3=OR(g0,x3) | g4=OR(g0,g1) | g5=XOR(g2,g4) | g6=AND(g2,g4) || QUERY g6 ->","difficulty":1,"num_inputs":4,"depth":2,"width":3,"num_gates":7,"cone_gates":5,"xor_count":3,"answer":"1"},"reward":1,"done":true}}
>> {"type":"step","data":{"answer":"1"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"INPUTS x0=0 x1=1 x2=0 x3=1 || GATES g0=XOR(x1,x0) | g1=XOR(x0,x2) | g2=AND(x3,g0) | g3=OR(g0,x3) | g4=OR(g0,g1) | g5=XOR(g2,g4) | g6=AND(g2,g4) || QUERY g6 ->","difficulty":1,"num_inputs":4,"depth":2,"width":3,"num_gates":7,"cone_gates":5,"xor_count":3,"answer":"1"},"reward":null,"done":true}}
>> {"type":"step","data":{"answer":"1"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"INPUTS x0=0 x1=1 x2=0 x3=1 || GATES g0=XOR(x1,x0) | g1=XOR(x0,x2) | g2=AND(x3,g0) | g3=OR(g0,x3) | g4=OR(g0,g1) | g5=XOR(g2,g4) | g6=AND(g2,g4) || QUERY g6 ->","difficulty":1,"num_inputs":4,"depth":2,"width":3,"num_gates":7,"cone_gates":5,"xor_count":3,"answer":"1"},"reward":null,"done":true}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":1,"num_gates":7}}
>> {"type":"close"}
$ build/rel/bin/synthlab logic http://127.0.0.1:8823 --seed 7
synthlab logic — programmed evaluation, a coin arm, cone depth and XOR density swept
learns    nothing — the evaluation is programmed, gate by gate in definition order
does not  dress computation as learning; the coin beside it is what chance pays
server    http://127.0.0.1:8823 — LogicEnvironment
run       1500 solver episodes · 1500 chance episodes · seed 7 · 500 per preset

demo      1000 seeds -> 1000 distinct tasks · 0 repeats · distinct by 64-bit FNV-1a over input_text
          seed 7 twice -> byte-identical
          61898 tasks/s over the WebSocket

preset  knobs                                         exact match   chance answers/s
     1  inputs 3-4, cone 4-6, depth 2-3, xor .10          500/500  248/500     39258
     2  inputs 4-5, cone 7-10, depth 3-5, xor .30         500/500  234/500     36891
     3  inputs 5-6, cone 12-16, depth 5-8, xor .50        500/500  235/500     32624

3000 tasks · 3000 steps · 0.1 s · 30155 steps/s
answered 1500/1500 · chance 717/1500 · two implementations agreed on every task
Plate 13. The environment evaluates its circuit when it mints it; the client evaluates the same circuit from input_text alone, having never seen the environment's code. They agreed on all fifteen hundred, and the coin beside them took 717 — which is what a one-bit answer pays and is the reason this plate has a sweep and not a curve. That agreement is the plate's whole point: a reward is only worth training against when two implementations of the specification produce the same number, and here the number is 1500/1500 with a coin at half. What this task resists is answering from the queried operator's bias. A gate's truth table is lopsided — AND answers 1 on one row in four, OR on three — so a circuit that drew its inputs first would hand the queried gate its operator's bias. Here the answer bit is a fair coin flipped before the circuit exists, and the inputs are drawn last from the assignments that produce it. So the reader that reads exactly one token — the operator name of the gate the QUERY names — is paid 51.4%, 49.9% and 48.7%, a coin's wage. Every one of the seven other readers in the battery is inside two points of even too. And a gate no input can move, which no coin can be forced on, occurs on 0.0% of tasks. Padding is bounded as well. A circuit can carry gates the queried gate never reads, so this generator draws the queried gate's cone — the gates the answer actually depends on — FIRST and pads outward to a cap, and the printed circuit is at least 70% cone at every difficulty. Difficulty buys depth and XOR density rather than decoration: the cone goes from 5.027 gates to 13.991 while the print goes from 6.549 to 18.973. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"answer": "1"} — the queried gate's output bit, as a string
reset{"seed": N, "difficulty": 1|2|3} — the difficulty moves the input count, the cone size, its depth and the XOR share
observationinput_text, difficulty, num_inputs, depth, width, num_gates, cone_gates — how many of those gates the answer depends on — xor_count, and answer, empty until done
rewardinteger 1 on exact match, 0 otherwise
donethe answer step
PLATE 14 · FILTRO · SOURCE SEPARATIONindex

Source separation: half the context is there to be ignored

Source separation is following one signal while a second is woven through it. Two graphs are interleaved into one list of edges. One uses Greek letters, the other colour names, and they never share a node; the query walks a stated number of hops through one of them. Every edge belonging to the other stream is there to be suppressed, and at the hardest preset the interleaving stops being a neat alternation and comes in shuffled or in runs.

$ build/rel/bin/filtro --host 127.0.0.1 --port 8824 &

$ build/rel/bin/oe_probe http://127.0.0.1:8824 --episode --steps 3 --action '{"answer":"alpha"}' --past-done | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"EDGES A:beta->theta | B:black->pink | A:eta->zeta | B:blue->purple | A:alpha->gamma | B:pink->yellow | A:gamma->epsilon | B:purple->orange | A:epsilon->beta | B:orange->black | A:zeta->delta | B:red->blue | A:delta->alpha | B:yellow->green | A:theta->eta | B:green->red || QUERY A:zeta hops=2 ->","difficulty":1,"num_nodes":8,"num_hops":2,"pattern":"alternating","answer":"","distractor":""},"reward":null,"done":false}}
>> {"type":"step","data":{"answer":"alpha"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"EDGES A:beta->theta | B:black->pink | A:eta->zeta | B:blue->purple | A:alpha->gamma | B:pink->yellow | A:gamma->epsilon | B:purple->orange | A:epsilon->beta | B:orange->black | A:zeta->delta | B:red->blue | A:delta->alpha | B:yellow->green | A:theta->eta | B:green->red || QUERY A:zeta hops=2 ->","difficulty":1,"num_nodes":8,"num_hops":2,"pattern":"alternating","answer":"alpha","distractor":"black"},"reward":1,"done":true}}
>> {"type":"step","data":{"answer":"alpha"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"EDGES A:beta->theta | B:black->pink | A:eta->zeta | B:blue->purple | A:alpha->gamma | B:pink->yellow | A:gamma->epsilon | B:purple->orange | A:epsilon->beta | B:orange->black | A:zeta->delta | B:red->blue | A:delta->alpha | B:yellow->green | A:theta->eta | B:green->red || QUERY A:zeta hops=2 ->","difficulty":1,"num_nodes":8,"num_hops":2,"pattern":"alternating","answer":"alpha","distractor":"black"},"reward":null,"done":true}}
>> {"type":"step","data":{"answer":"alpha"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"EDGES A:beta->theta | B:black->pink | A:eta->zeta | B:blue->purple | A:alpha->gamma | B:pink->yellow | A:gamma->epsilon | B:purple->orange | A:epsilon->beta | B:orange->black | A:zeta->delta | B:red->blue | A:delta->alpha | B:yellow->green | A:theta->eta | B:green->red || QUERY A:zeta hops=2 ->","difficulty":1,"num_nodes":8,"num_hops":2,"pattern":"alternating","answer":"alpha","distractor":"black"},"reward":null,"done":true}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":1,"num_hops":2}}
>> {"type":"close"}
$ build/rel/bin/synthlab filtro http://127.0.0.1:8824 --seed 7
synthlab filtro — programmed walk of the queried stream, a uniform-node arm, hops 1..n-1 swept
learns    nothing — the walk is programmed and reads only A: edges
does not  dress computation as learning; suppressing B: costs a parser nothing, and says so
server    http://127.0.0.1:8824 — FiltroEnvironment
run       1500 solver episodes · 1500 chance episodes · seed 7 · 500 per preset

demo      1000 seeds -> 1000 distinct tasks · 0 repeats · distinct by 64-bit FNV-1a over input_text
          seed 7 twice -> byte-identical
          66308 tasks/s over the WebSocket

preset  knobs                                         exact match   chance answers/s
     1  nodes 5-8, hops 1..n-1, alternating               500/500   92/500     39583
     2  nodes 8-12, hops 1..n-1, alternating/random       500/500   44/500     37826
     3  nodes 10-15, hops 1..n-1, random/blocked          500/500   43/500     35716

3000 tasks · 3000 steps · 0.1 s · 31622 steps/s
answered 1500/1500 · chance 179/1500 · two implementations agreed on every task
Plate 14. Read the edge list above before the table, because the order of it is the plate. What this task resists is reading by position. Stream A's edges are printed in an order that has nothing to do with the walk — beta, then eta, then alpha — so hops=k never collapses into "read k edges further down the list". The battery's positional reader, which never separates the streams at all, scores 16.8% at the hardest preset, against 18.4% for a guesser that reads nothing but the hop count. And the A-list chains end-to-start on only 9.4% of tasks at that preset, which is what a shuffled list does by arithmetic. That 18.4% is the looser of the two baselines the battery prints, and deliberately so: true chance here is 8.9% — a walk of 1..n-1 on a ring of n can never come home, so a guesser who only crosses off the start pays 1/(n-1), the row called uniform-minus-start. The positional reader is graded against the looser one because that baseline also collects every hops=1 row for free, and taking those is the only thing the reader does that a correct solver does not. Against the honest floor it is 16.8% to 8.9%, and against the fair comparison 16.8% to 18.4%. And hops is drawn uniform on 1..n-1 rather than from a preset window, so no fixed offset and no floor — "never the immediate successor" would be one — beats a uniform draw over the nodes the walk can reach. The sweep stays a sweep: a parser reads the A: edges, never looks at B:, and walks, taking every one of fifteen hundred against 179 for the uniform guess. Suppression is a difficulty for attention, not for a parser, and nothing here pretends otherwise. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
the contractshape
action{"answer": "alpha"} — the node the walk ends on
reset{"seed": N, "difficulty": 1|2|3} — the difficulty moves the node count and the interleaving pattern; the hop count is drawn uniform on 1..n-1 at every difficulty
observationinput_text, difficulty, num_nodes, num_hops, pattern, and answer and distractor — the walk through the other stream from the mirrored start, the answer the noise was selling — both empty until done
rewardinteger 1 on exact match, 0 otherwise
donethe answer step
THE MATH ROOM · PLATES 15-22index

Eight kinds of mathematical work

The math room turns a problem into ordinary text and asks for one bounded answer. It spans practical quantities, symbolic structure, change, space, finite systems, uncertainty, observed data, and proof. Those are different objects to reason about, not eight skins over the same arithmetic.

Every task states its conventions and return grammar inside input_text. The environment grades with exact arithmetic or a small certificate checker, then reveals the answer only after the step has ended. The independent mathlab client solves from the prompt, tries a wrong answer, repeats the seed, checks oversized input, and makes sure a finished episode stays frozen. The full design, including the curriculum table and the limits of this first release, is in the math-room guide.

shared contractshape
reset{"seed": N, "difficulty": 1|2|3} — later presets mix in new reasoning structures without removing the earlier ones
observationinput_text, a task-family label, small public size fields, and an empty answer
action{"answer": "..."} — the exact grammar is the final line of the prompt
rewardinteger 1 when the bounded grader accepts the answer, integer 0 otherwise; no model judges another model
PLATE 15 · MEASUREindex

Keep the unit attached to the number

measure begins with one rate, then adds proportional scaling and clock conversion with an optional pause. Units are part of the canonical answer, so a bare number cannot earn credit by accident.

$ build/rel/bin/mathlab measure http://127.0.0.1:8831 --seed 7
mathlab measure — quantities, units, and practical models
server    http://127.0.0.1:8831 — MeasureEnvironment
run       300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset  tasks   exact   wrong accepted   repeats
     1    100     100                0         0
     2    100     100                0         0
     3    100     100                0         1
300 tasks · 300/300 exact · 0 wrong answers accepted · 1 repeated prompts within a preset
A practical model with a hard boundary. The prompt must decide which time actually counts before multiplication begins. The verifier recomputes the result from that text, while the server owns only generation and grading. — make record-math; site-check refuses any line arcade/math-recordings.txt does not contain.
PLATE 16 · ALGEBRAindex

Solve for the whole answer, not one convenient root

algebra moves from a linear equation to a unique system and then an integer-root quadratic. Its quadratic answer is a complete ordered set, which closes the common shortcut of returning only one valid root.

$ build/rel/bin/mathlab algebra http://127.0.0.1:8832 --seed 7
mathlab algebra — expressions and complete solution sets
server    http://127.0.0.1:8832 — AlgebraEnvironment
run       300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset  tasks   exact   wrong accepted   repeats
     1    100     100                0         0
     2    100     100                0         0
     3    100     100                0         0
300 tasks · 300/300 exact · 0 wrong answers accepted · 0 repeated prompts within a preset
Completeness is gradeable. Coefficients and equation shapes vary, but the answer boundary stays small. Exact sets make partial credit impossible to win through formatting tricks. — make record-math; site-check refuses any line arcade/math-recordings.txt does not contain.
PLATE 17 · CHANGEindex

Read a function as a rule for variation

change asks for a derivative at a point, a definite integral, or a global maximum on a closed interval. The harder presets change what must be reasoned about; they do not merely widen the coefficients.

$ build/rel/bin/mathlab change http://127.0.0.1:8833 --seed 7
mathlab change — functions, rates, accumulation, and extrema
server    http://127.0.0.1:8833 — ChangeEnvironment
run       300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset  tasks   exact   wrong accepted   repeats
     1    100     100                0         0
     2    100     100                0         0
     3    100     100                0         0
300 tasks · 300/300 exact · 0 wrong answers accepted · 0 repeated prompts within a preset
Three meanings of change. A learner has to identify the requested object before calculating it. Mixing earlier families into later presets keeps the difficulty label from naming the operation. — make record-math; site-check refuses any line arcade/math-recordings.txt does not contain.
PLATE 18 · SPACEindex

Let coordinates replace an unreliable sketch

space begins with squared distance, then applies a matrix to a vector and finally tracks area through a linear map. Exact integer answers keep the geometry legible and avoid a hidden floating-point tolerance.

$ build/rel/bin/mathlab space http://127.0.0.1:8834 --seed 7
mathlab space — coordinates and linear transformations
server    http://127.0.0.1:8834 — SpaceEnvironment
run       300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset  tasks   exact   wrong accepted   repeats
     1    100     100                0         0
     2    100     100                0         0
     3    100     100                0         0
300 tasks · 300/300 exact · 0 wrong answers accepted · 0 repeated prompts within a preset
The representation does useful work. Coordinates make every premise textual and every transformation checkable. No image parser or learned visual judge sits between the answer and the reward. — make record-math; site-check refuses any line arcade/math-recordings.txt does not contain.
PLATE 19 · DISCRETEindex

Advance one exact state at a time

discrete covers recurrences, shortest paths, and constrained unordered selection. The changing tokens alter transitions, graph edges, or constraints, so diversity changes the decision rather than decorating it.

$ build/rel/bin/mathlab discrete http://127.0.0.1:8835 --seed 7
mathlab discrete — recurrences, finite structures, and counts
server    http://127.0.0.1:8835 — DiscreteEnvironment
run       300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset  tasks   exact   wrong accepted   repeats
     1    100     100                0         0
     2    100     100                0         0
     3    100     100                0         0
300 tasks · 300/300 exact · 0 wrong answers accepted · 0 repeated prompts within a preset
Finite does not mean one-dimensional. State evolution, graph search, and counting demand different intermediate representations while sharing exact, inexpensive graders. — make record-math; site-check refuses any line arcade/math-recordings.txt does not contain.
PLATE 20 · CHANCEindex

Begin with the sample space the process uses

chance moves from one draw to dependent draws and conditioning. Replacement and ordering are stated in the prompt, and every probability is returned as a reduced fraction rather than an approximate decimal.

$ build/rel/bin/mathlab chance http://127.0.0.1:8836 --seed 7
mathlab chance — finite probability and conditioning
server    http://127.0.0.1:8836 — ChanceEnvironment
run       300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset  tasks   exact   wrong accepted   repeats
     1    100     100                0         0
     2    100     100                0         1
     3    100     100                0         0
300 tasks · 300/300 exact · 0 wrong answers accepted · 1 repeated prompts within a preset
Uncertainty, exact reward. The random process is part of the text, but the grader is deterministic. A reduced fraction gives one canonical boundary for equivalent probabilities. — make record-math; site-check refuses any line arcade/math-recordings.txt does not contain.
PLATE 21 · EVIDENCEindex

Aggregate the rows before comparing the groups

evidence presents tables that call for aggregate rates, weighted means, or medians under a stated missing-value rule. It stays separate from probability because these rows have already happened; the job is to summarize observed data.

$ build/rel/bin/mathlab evidence http://127.0.0.1:8837 --seed 7
mathlab evidence — tables, aggregation, and statistics
server    http://127.0.0.1:8837 — EvidenceEnvironment
run       300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset  tasks   exact   wrong accepted   repeats
     1    100     100                0         0
     2    100     100                0         0
     3    100     100                0         0
300 tasks · 300/300 exact · 0 wrong answers accepted · 0 repeated prompts within a preset
Close to everyday mathematical work. The learner must read a table, choose the declared statistic, and keep numerator and denominator aligned. Exact arithmetic makes that workflow cheap to grade at scale. — make record-math; site-check refuses any line arcade/math-recordings.txt does not contain.
PLATE 22 · REASONindex

Prove the claim or exhibit a countermodel

reason does not compare one hidden answer string. It accepts a valid sequence of applicable rules or a complete assignment that satisfies the facts and rules while making the claim false. Both are checked for meaning, so alternate valid certificates can earn the same reward.

$ build/rel/bin/mathlab reason http://127.0.0.1:8838 --seed 7
mathlab reason — checked proof or countermodel certificates
server    http://127.0.0.1:8838 — ReasonEnvironment
run       300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset  tasks   exact   wrong accepted   repeats
     1    100     100                0         2
     2    100     100                0         0
     3    100     100                0         0
300 tasks · 300/300 exact · 0 wrong answers accepted · 2 repeated prompts within a preset
The work is the certificate. A bare verdict earns nothing. The grader replays every proof step or checks every truth assignment, which makes the reward both flexible and auditable. — make record-math; site-check refuses any line arcade/math-recordings.txt does not contain.
GRADED BY SOMEBODY ELSEindex

Judges that are not ours

A page that grades its own games is a page that agrees with itself. So the recording ends with three judges: the upstream project's own conformance suite, a battery of cheap readers with a control population beside it, and the same game timed on two servers — theirs and ours. The first is openenv validate, the reference's own conformance suite — the reference being huggingface/OpenEnv, the Python implementation this repository was captured from. Six required criteria, run over the network against three of the ten, with nothing of ours in the loop but the server being graded. Connect four is one of the three on purpose — if a made-up observation field like legal_actions were going to trouble somebody else's conformance suite, this is where it would show.

$ build/rel/bin/bandit --host 127.0.0.1 --port 8801 &

$ (cd reference/OpenEnv && PYTHONPATH=src .venv-oe/bin/python -m openenv.cli validate --url http://127.0.0.1:8801) | sed -n '/"summary"/,/}/p'
  "summary": {
    "passed_count": 6,
    "total_count": 6,
    "failed_criteria": [],
    "required_passed_count": 6,
    "required_total_count": 6
  },

$ build/rel/bin/snake --host 127.0.0.1 --port 8803 &

$ (cd reference/OpenEnv && PYTHONPATH=src .venv-oe/bin/python -m openenv.cli validate --url http://127.0.0.1:8803) | sed -n '/"summary"/,/}/p'
  "summary": {
    "passed_count": 6,
    "total_count": 6,
    "failed_criteria": [],
    "required_passed_count": 6,
    "required_total_count": 6
  },

$ build/rel/bin/connect4 --host 127.0.0.1 --port 8809 &

$ (cd reference/OpenEnv && PYTHONPATH=src .venv-oe/bin/python -m openenv.cli validate --url http://127.0.0.1:8809) | sed -n '/"summary"/,/}/p'
  "summary": {
    "passed_count": 6,
    "total_count": 6,
    "failed_criteria": [],
    "required_passed_count": 6,
    "required_total_count": 6
  },
Six of six, three times. The same validator that grades envs/counter.c in make interop, pointed at a slot machine, at snake, and at the one with an opponent living inside it. Being fun is not a licence to be non-conforming: these get GET /schema, request validation, the sixteen routes and the session machinery from the same library a stranger installs, so passing is what should happen — and until this ran, nobody had checked. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.

The second judge is the battery the synth-lab captions have been quoting. A task that a fixed-position reader answers without doing the work is a task in the shape of a task. So a battery of deliberately stupid solvers runs over 2,000 fresh tasks per family per difficulty. The same solvers run over a second population the header below calls real rows: a frozen sample of rows in the same grammar, a hundred per family, held as the control. A fixed file is where a cheap reader gets paid, so a claim of resistance needs a population that does not resist beside it.

$ python3 interop/exploit_battery.py --tasks 2000 --grade
exploit battery — cheap heuristics vs 2000 of ours per family per difficulty, vs 100 real rows per family
...
  heuristic                        ours d1   ours d2   ours d3   real d1   real d2   real d3  real ALL
...
  positional-reader                  34.0%     23.1%     16.8%    100.0%    100.0%    100.0%    100.0%
...
  uniform-minus-start                19.1%     11.4%      8.9%     12.2%     12.8%     12.2%     12.4%
...
  one-hop-free uniform               37.4%     23.3%     18.4%     29.6%     31.6%     33.1%     30.9%
...
  [struct] path-order A-links        19.4%     11.2%      9.4%    100.0%    100.0%    100.0%    100.0%
...
  [struct] rows fully path-ordered      1.2%      0.0%      0.0%    100.0%    100.0%    100.0%    100.0%
...
  final-gate operator prior          51.4%     49.9%     48.7%     62.7%     78.6%     53.8%     66.0%
...
  [struct] queried gate constant      0.0%      0.0%      0.0%     22.0%     10.7%      7.7%     17.0%
...
  net-rotation only                  16.4%      9.8%      6.3%     13.7%     21.1%      9.1%     16.0%
...
  copy-previous-answer                0.0%      0.0%      0.0%      7.8%      7.9%      4.5%      7.5%
...
  [struct] inverse-redundant episodes      0.0%      0.0%      0.0%     35.3%     44.7%     45.5%     40.0%
...
  [struct] length-only identifies      0.0%      0.0%      0.0%     20.0%     16.3%     12.5%     16.0%
...
oracle: this file's solver re-derived the graded answer on every task of both populations, all four families.
...
The contrast is the claim. Abridged at every ...; the recording carries four whole tables and a graded bar list. One frame governs every pair: a fixed file rewards position, padding and repetition, and a generator that shuffles emission, draws the answer before the circuit, and re-mints per seed does not — the pairs across are that sentence, measured. A positional reader that never separates filtro's two streams takes 100.0% of the real rows and 16.8% of ours; the control's A-lists chain end-to-start on 100.0% of rows and ours on 9.4%; a one-token reader of the queried gate's operator takes 66.0% there and 48.7% here, and 17.0% of the control circuits have a queried gate no input bit can move against 0.0% of ours; copy-the-previous-answer pays 7.5% there and 0.0% here, and 40.0% of the control episodes ask one question twice (a position and the element standing at it) against 0.0% of ours; summing the rotations while ignoring SWAP and REVERSE takes 16.0% there against 12.1% chance, and 6.3% here at difficulty 3 against a 6.2% chance; output length alone names the rule on 16.0% of the control rows and 0.0% of ours. Two baselines are printed for filtro and the looser one is deliberate: uniform-minus-start is the honest floor a walk of 1..n-1 leaves (19.1 / 11.4 / 8.9), while one-hop-free uniform also collects every hops=1 row for free (37.4 / 23.3 / 18.4) — the positional reader is graded against the second because taking the one-hop rows is the only thing it does that a correct solver does not. The oracle line is what licenses the rest — the battery re-derived every graded answer of both populations from the prompt, so it is measuring the tasks and not its own bugs — and the bars under it are the charter's, written before the run: a named reader more than four points above the sharpest cheap baseline is a red run, and the recorder carries the failure out in its exit status. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.

The third judge is a price tag. interop/bandit_env.py is arcade/bandit.c ported onto the reference's own server — their create_app, their routing, their pydantic validation, under uvicorn — so the same trainer, at the same seed, over the same protocol, can be pointed at two implementations of one game. Both ports are deterministic in the seed, so both runs must print the same curve; the recorder diffs them and prints a ratio only when they match.

$ (cd reference/OpenEnv && PYTHONPATH=src:../../interop .venv-oe/bin/python -m uvicorn bandit_env:app --host 127.0.0.1 --port 8811) &

$ build/rel/bin/train bandit http://127.0.0.1:8811 --seed 7
...
run       200 episodes · seed 7 · eps 0.10 · sample-average · a new machine each visit
...
   1-  10               0.63
...
  91- 100               0.83   ########################################
...

...
Both runs printed the same curve, line for line: the two servers
dealt the same ladder and paid the same coins, so what is left
between them is the cost of serving one step.
Same game, two servers. Abridged at every ..., from a recording that carries both curves whole — the ten rows here are identical to plate 1's, which is the point and is why the recorder prints a ratio at all. That ratio compares two implementations of one game — not two algorithms, and not two projects. One client, one connection, one request in flight: it is latency, not saturation, and the reference is doing general work that a single hand-written env does not do. The direction reverses too, in make interop: its eighth oracle runs epsilon-greedy inside upstream's own EnvClient against arcade/bandit.c and asserts the learner improved, so each side has now trained the other's. — make record-arcade; site-check refuses any line arcade/recordings.txt does not contain.
THE BOUNDARYindex

Which side of the line this is

This repository has two directories of environments and they are not the same kind of thing. envs/ teaches the protocol: those files are corpus-coupled, their bytes are diffed against captured recordings, and they may not change. arcade/ is built on the protocol and is free — it is what a stranger writes after reading the tour. Every file under it opens with the same seven lines, and a rule in tools/check.sh reads them back: it takes the seven out of arcade/CHARTER.md and compares them, byte for byte, against the head of every arcade source, so the marking cannot drift one file at a time.

ARCADE — built ON the OpenEnv protocol, not part of the teaching core. envs/ is the other side of this boundary: corpus-coupled oracles that may not change, because tests diff their bytes against recordings. Delete this directory and no corpus case, oracle verdict or generated ledger in this repository changes: the one oracle that names a file here skips without it. These are ordinary downstream environments, written with the same public API any stranger gets: <openenv/openenv.h>, oe_main, and nothing else.

That is the useful half of the marking. Every server gets the whole library — sixteen routes, GET /schema, WebSocket sessions, request validation, the production flag — from the same public header a consumer installs, and they get it without being allowed to touch the evidence.

The word in that banner is proof, and it is narrow because the experiment has been run. Move arcade/ aside and what fails is citation and count — dead links to this directory's note, a program tally, this page's quotations — while every corpus case, oracle verdict and conformance figure stands. The one place the arcade appears in a suite is make interop's eighth oracle, and it skips when the binary is absent. The directory's own note is arcade/README.md.

BRING A REAL LEARNERindex

What these are for

The clients on this page are deliberately small. The game client holds ten short algorithms; the synth client holds four exact readers; the math client parses and independently solves eight task families. None is a network, so the loop and the source of reward remain legible. The environments are not small in the same way. Ten are the fast twins of material the upstream project already teaches with — huggingface/OpenEnv ↗ ships a snake environment, a 2048 training tutorial and a Wordle task for GRPO, plus a blackjack example, a maze, a grid world and a connect four environment — four are task generators with a verifiable reward, and the math room adds eight more. That is what a GRPO-style trainer wants and what the boundary note means when it points at GRPOTrainer. They answer between 30155 and 85710 steps a second on one core, round trip, over a real WebSocket. The ten are not those upstream environments and do not pretend to be: the action names here are ours, so a client written for upstream's snake gets a validation error rather than a game.

Each server holds eight sessions at once by default, so eight rollout workers can share one process, and every one of them gets its own environment instance. Nothing here needs a GPU, a container or a Python runtime. If your learner is slower than the recorded server loops — and a learner that reads a board and computes a gradient will be — then the environment is not your bottleneck, which is the only claim this page is making.

These servers are also built to be trained against by a language model, and that is three steps rather than a project. make arcade-up starts all fourteen and writes build/rel/arcade.json, an object from each server's name to its base URL; make math-up does the same for the math room in build/rel/math.json. The adapters this repository ships read those manifests, ask each server GET /schema what its action field is called, and take the prompt out of input_text. So there is no table of ports to type and no prompt to write per environment. make adapters drives those adapters against the running arcade with no GPU and no model weights. Three walkthroughs — TRL, prime-rl, and the Tinker API via SkyRL — are in docs/TRAINING.md, which says which parts of them have been run here and which have not.

Read next: the tour, which is how these were written; the annotated episode, which is what the transcripts above look like byte by byte; or arcade/README.md, the directory's own note, which is where the ports, the seeds and both clients' flags are written down.