The arcade
Fourteen small environment servers built on the
OpenEnv protocol: ten games a trainer learns to play over a real WebSocket,
and four task generators that mint a fresh problem per seed and grade the
answer themselves. A third wing adds eight text-first math environments.
Every transcript and every number on this page is recorded output from
programs in arcade/.
The math room covers quantities, algebra, functions, transformations,
discrete structures, probability, data, and checked reasoning. Its
approachable guide, exact answer grammars, independent solver, and adversarial
checks live in
arcade/MATH.md.
The eight plates are on this page too, with their recorded checks beside
them.
The original gallery is fourteen servers in
arcade/ ↗.
Ten of them are games: a ten-armed bandit, a cart-pole, snake, 2048, Wordle,
blackjack, a maze, pong, connect four and Kuhn poker. Four of them are the
synth lab, and they are not games at all — they hand you a problem in text and
grade the answer. Each one is a complete OpenEnv server, and each one is one
file you can read in a sitting. Beside them are two clients:
arcade/train.c ↗,
which opens a WebSocket to whichever game is running and learns to play it,
and
arcade/synthlab.c ↗,
which reads the task servers' text and answers it. The math room has a third
client, mathlab.c ↗,
which independently parses, solves, and attacks every math family.
OpenEnv is Hugging Face's standard for running reinforcement-learning
environments as services. A client resets an environment, then steps it with
actions; each step returns an observation and a reward. An episode is
one run of that loop: a reset, then steps, until done. Neither
client links any of these servers or includes any of their headers — every
feature they use arrived as JSON on a socket, which is why one binary trains
all ten games and would train an eleventh written in Python.
Nothing below is a diagram. make record-arcade starts each
server on a pinned port, drives one episode by hand with
oe_probe ↗,
runs a client against it, and writes the session to
arcade/recordings.txt ↗.
The math room repeats that process with make record-math and
arcade/math-recordings.txt ↗.
This page may quote only those files: make site exits nonzero on a
transcript line they do not contain, and on a figure in this prose neither
recording printed.
The plates, one click each
Three commands
These servers are ordinary targets of the ordinary build, so there is nothing to install and nothing to configure. Build, start one, point a client at it:
$ make arcade
arcade: 16 binaries in build/rel/bin — bandit blackjack cartpole connect4 dyna filtro g2048 inducto kuhn logic maze pong snake synthlab train wordle
plus oe_probe for poking them by hand
try: make train-bandit (or run one by hand: build/rel/bin/bandit --port 8801 &)
...
$ build/rel/bin/snake --host 127.0.0.1 --port 8803 &
...
$ build/rel/bin/train snake http://127.0.0.1:8803 --seed 7make train-snake is the same three commands in one,
make task-dyna is the same three for a synth-lab server with
synthlab as the client, and make record-arcade is all
fourteen end to end. make math-measure and
make record-math are the matching commands for the math room. The server binds
loopback because these are development servers; the seed makes the run
repeatable, and it is the client's own default. A ... line is
this page saying it cut something: the three commands are recorded in three
different places and the output between them is below, plate by plate.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain, and any block that reorders one.What follows is one plate per server, in the order the recording made them,
which is the order the ports ascend. Each shows an episode driven by hand —
so you can see the shape of an observation before anything learns — and then
the client run. Both blocks are in the order the
recording has them, from the reset to the close, because a transcript with a
step quietly removed is a transcript whose own step_count
contradicts it; where a plate abridges, it prints a ... line and
you can see the cut.
One field is in every observation on this page, and it is the newest thing
here. input_text is the whole decision as text: the situation, the
picture where there is one, and how to reply on the last line. It carries
nothing the other fields do not carry. So a trainer can paste it into a prompt
without auditing the environment for a hole card. The protocol has never heard
of it — like legal_actions on plate 9, it is an ordinary field with
a naming convention around it. It is here so a language model can play these
servers with no prompt hand-written per game, which is what the
training walkthroughs now rely on.
Ten games, one trainer
Ten games, and one trainer that plays them all.
arcade/train.c opens a WebSocket to whichever server is running,
and every feature it uses arrived as JSON on that socket. The ten are not ten
of one kind. One has an opponent living inside the environment, moving between
your action and your observation. One carves its world out of the reset seed.
One pays a reward that can only be sampled, and stops below zero. One measures
a metric that is the game itself. And the last is not estimating a value at
all — its chart wins by falling.
Ten arms, and the smallest complete loop there is
Ten slot machines in a row, each paying one or nothing at its own fixed probability; the seed decides which arm holds which. There is no state to speak of and no strategy to plan — the only way to learn a probability is to sample it, which makes this the shortest honest reinforcement-learning loop that exists.
$ build/rel/bin/bandit --host 127.0.0.1 --port 8801 &
$ build/rel/bin/oe_probe http://127.0.0.1:8801 --episode --steps 3 --action '{"arm":3}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ten arms, 0 to 9. Each pays 1 with its own fixed probability, else 0. The best arm pays 0.95; only its index changes.\nPull 1 of 100. No arm pulled yet.\nReply with one digit, the arm to pull: 0 to 9.","pulls":0,"last_arm":-1,"last_payout":0.0},"reward":null,"done":false}}
>> {"type":"step","data":{"arm":3}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ten arms, 0 to 9. Each pays 1 with its own fixed probability, else 0. The best arm pays 0.95; only its index changes.\nPull 2 of 100. Arm 3 paid 0 last time.\nReply with one digit, the arm to pull: 0 to 9.","pulls":1,"last_arm":3,"last_payout":0.0},"reward":0,"done":false}}
>> {"type":"step","data":{"arm":3}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ten arms, 0 to 9. Each pays 1 with its own fixed probability, else 0. The best arm pays 0.95; only its index changes.\nPull 3 of 100. Arm 3 paid 1 last time.\nReply with one digit, the arm to pull: 0 to 9.","pulls":2,"last_arm":3,"last_payout":1.0},"reward":1,"done":false}}
>> {"type":"step","data":{"arm":3}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ten arms, 0 to 9. Each pays 1 with its own fixed probability, else 0. The best arm pays 0.95; only its index changes.\nPull 4 of 100. Arm 3 paid 1 last time.\nReply with one digit, the arm to pull: 0 to 9.","pulls":3,"last_arm":3,"last_payout":1.0},"reward":1,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"winnings":2}}
>> {"type":"close"}
$ build/rel/bin/train bandit http://127.0.0.1:8801 --seed 7
train bandit — epsilon-greedy, sample-average estimates, 10 arms
learns which of ten arms pays, from what came back when it pulled them
does not see the ladder, carry an estimate to a new seed, or stop exploring
server http://127.0.0.1:8801 — BanditEnvironment
run 200 episodes · seed 7 · eps 0.10 · sample-average · a new machine each visit
pulls avg payout curve (bars span 0.63 to 0.83)
1- 10 0.63
11- 20 0.70 ##############
21- 30 0.72 ##################
31- 40 0.76 #########################
41- 50 0.78 ##############################
51- 60 0.77 ############################
61- 70 0.79 ##############################
71- 80 0.80 ##################################
81- 90 0.83 ######################################
91- 100 0.83 ########################################
200 episodes · 20000 steps · 0.2 s · 85710 steps/s · final avg 0.83
first ten pulls .63 · last ten .83 · favourite arm paid .82 · ladder .50 average, .95 best
play_bandit says what each of them got wrong.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"arm": 0..9} — required; anything else is an
EXECUTION_ERROR frame |
| observation | input_text, pulls,
last_arm, last_payout |
| reward | integer 0 or 1, drawn at that arm's probability |
| done | after a hundred pulls |
A pole that falls over unless you keep moving
The classic control problem, with the Barto-Sutton-Anderson equations integrated at fifty steps a second: a cart on a track, a pole hinged on top, and two actions — push left, push right. It is the plainest picture of what a learning curve is, because the reward is one per step and the episode ends when the pole tips, so the return is how long it stayed up.
$ build/rel/bin/cartpole --host 127.0.0.1 --port 8802 &
$ build/rel/bin/oe_probe http://127.0.0.1:8802 --episode --steps 3 --action '{"push":1}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A pole stands on a cart that can only be pushed sideways. Keep it upright.\nCart 0.038 m, moving -0.007 m/s. Pole -0.047 rad, turning 0.047 rad/s. Positive is right; three decimals here, the fields carry the rest.\nStep 1 of 500. Past 2.4 m or past 0.2095 rad ends it.\nReply with one digit: 0 push left, 1 push right. There is no coasting.","x":0.038331080821364265,"x_dot":-0.006847200295149004,"theta":-0.04735662284074023,"theta_dot":0.04708819781538285},"reward":null,"done":false}}
>> {"type":"step","data":{"push":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A pole stands on a cart that can only be pushed sideways. Keep it upright.\nCart 0.038 m, moving 0.189 m/s. Pole -0.046 rad, turning -0.260 rad/s. Positive is right; three decimals here, the fields carry the rest.\nStep 2 of 500. Past 2.4 m or past 0.2095 rad ends it.\nReply with one digit: 0 push left, 1 push right. There is no coasting.","x":0.038194136815461285,"x_dot":0.18892069604837716,"theta":-0.04641485888443257,"theta_dot":-0.2601520728061378},"reward":1.0,"done":false}}
>> {"type":"step","data":{"push":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A pole stands on a cart that can only be pushed sideways. Keep it upright.\nCart 0.042 m, moving 0.385 m/s. Pole -0.052 rad, turning -0.567 rad/s. Positive is right; three decimals here, the fields carry the rest.\nStep 3 of 500. Past 2.4 m or past 0.2095 rad ends it.\nReply with one digit: 0 push left, 1 push right. There is no coasting.","x":0.04197255073642883,"x_dot":0.3846734497997366,"theta":-0.05161790034055533,"theta_dot":-0.5671060412153819},"reward":1.0,"done":false}}
>> {"type":"step","data":{"push":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A pole stands on a cart that can only be pushed sideways. Keep it upright.\nCart 0.050 m, moving 0.580 m/s. Pole -0.063 rad, turning -0.876 rad/s. Positive is right; three decimals here, the fields carry the rest.\nStep 4 of 500. Past 2.4 m or past 0.2095 rad ends it.\nReply with one digit: 0 push left, 1 push right. There is no coasting.","x":0.049666019732423565,"x_dot":0.5804800344054798,"theta":-0.06296002116486296,"theta_dot":-0.875593648147289},"reward":1.0,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"theta":-0.06296002116486296}}
>> {"type":"close"}
$ build/rel/bin/train cartpole http://127.0.0.1:8802 --seed 7
train cartpole — tabular Q-learning, 162 boxes x 2 actions
learns when to push left and when to push right, from four numbers
does not know the physics, look ahead, or tell two states in one box apart
server http://127.0.0.1:8802 — CartPoleEnvironment
run 2000 episodes · seed 7 · eps 1.00->0.05 · alpha 0.10 · gamma 0.99
episodes avg return curve (bars span 23.47 to 124.91)
1- 200 23.47
201- 400 23.89
401- 600 30.54 ###
601- 800 38.91 ######
801-1000 49.16 ##########
1001-1200 49.02 ##########
1201-1400 77.77 #####################
1401-1600 79.81 ######################
1601-1800 110.38 ##################################
1801-2000 124.91 ########################################
2000 episodes · 121572 steps · 2.9 s · 41252 steps/s · final avg 124.91
best episode 500 steps · 103 of 162 boxes visited
theta falling further from upright each time. The state door
answers with the value after the third of them, which is the observation
directly above it.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"push": 0} left or {"push": 1}
right — required |
| observation | input_text, then x,
x_dot, theta, theta_dot — four reals, which
the text rounds to three places and the fields carry whole |
| reward | 1.0 for every step the pole is still
up |
| done | the cart leaves the track, the pole passes twelve degrees, or the step cap |
The one everybody already knows
A snake on a small grid, a piece of food, and three choices at every step:
turn left, go straight, turn right. Eating grows the tail and moves the food;
a wall or your own body ends the episode. Upstream ships a snake environment of
its own, on gym and marlenv; this is the same game written for this protocol in
one C file, answering in microseconds. The shapes are ours, not theirs — the
action here is turn, and an upstream-shaped action
gets a validation error, which is the server behaving correctly.
$ build/rel/bin/snake --host 127.0.0.1 --port 8803 &
$ build/rel/bin/oe_probe http://127.0.0.1:8803 --episode --steps 3 --action '{"turn":1}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Snake, 12x12. 'O' head, 'o' body, '@' food, '.' empty. Eating grows you; a wall or your own body ends the game.\n............\n............\n............\n.......@....\n............\n............\n....ooO.....\n............\n............\n............\n............\n............\nHead at column 6, row 6, heading right. Food at column 7, row 3. Length 3, score 0.\nReply with one digit, a turn relative to your heading: 0 left, 1 straight, 2 right.","grid":"............\n............\n............\n.......@....\n............\n............\n....ooO.....\n............\n............\n............\n............\n............","head_x":6,"head_y":6,"food_x":7,"food_y":3,"dir":1,"length":3,"score":0},"reward":null,"done":false}}
>> {"type":"step","data":{"turn":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Snake, 12x12. 'O' head, 'o' body, '@' food, '.' empty. Eating grows you; a wall or your own body ends the game.\n............\n............\n............\n.......@....\n............\n............\n.....ooO....\n............\n............\n............\n............\n............\nHead at column 7, row 6, heading right. Food at column 7, row 3. Length 3, score 0.\nReply with one digit, a turn relative to your heading: 0 left, 1 straight, 2 right.","grid":"............\n............\n............\n.......@....\n............\n............\n.....ooO....\n............\n............\n............\n............\n............","head_x":7,"head_y":6,"food_x":7,"food_y":3,"dir":1,"length":3,"score":0},"reward":0.01,"done":false}}
>> {"type":"step","data":{"turn":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Snake, 12x12. 'O' head, 'o' body, '@' food, '.' empty. Eating grows you; a wall or your own body ends the game.\n............\n............\n............\n.......@....\n............\n............\n......ooO...\n............\n............\n............\n............\n............\nHead at column 8, row 6, heading right. Food at column 7, row 3. Length 3, score 0.\nReply with one digit, a turn relative to your heading: 0 left, 1 straight, 2 right.","grid":"............\n............\n............\n.......@....\n............\n............\n......ooO...\n............\n............\n............\n............\n............","head_x":8,"head_y":6,"food_x":7,"food_y":3,"dir":1,"length":3,"score":0},"reward":-0.01,"done":false}}
>> {"type":"step","data":{"turn":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Snake, 12x12. 'O' head, 'o' body, '@' food, '.' empty. Eating grows you; a wall or your own body ends the game.\n............\n............\n............\n.......@....\n............\n............\n.......ooO..\n............\n............\n............\n............\n............\nHead at column 9, row 6, heading right. Food at column 7, row 3. Length 3, score 0.\nReply with one digit, a turn relative to your heading: 0 left, 1 straight, 2 right.","grid":"............\n............\n............\n.......@....\n............\n............\n.......ooO..\n............\n............\n............\n............\n............","head_x":9,"head_y":6,"food_x":7,"food_y":3,"dir":1,"length":3,"score":0},"reward":-0.01,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"score":0}}
>> {"type":"close"}
$ build/rel/bin/train snake http://127.0.0.1:8803 --seed 7
train snake — tabular Q-learning, 288 states x 3 actions
learns which turn to take from local danger and food direction
does not see the whole board, plan ahead, or manage its tail
server http://127.0.0.1:8803 — SnakeEnvironment
run 5000 episodes · seed 7 · eps 1.00->0.05 · alpha 0.20 · gamma 0.90
episodes avg return curve (bars span -0.76 to 8.05)
1- 500 -0.76
501-1000 -0.49 #
1001-1500 -0.21 ###
1501-2000 0.30 #####
2001-2500 0.55 ######
2501-3000 1.08 ########
3001-3500 1.71 ###########
3501-4000 2.82 ################
4001-4500 4.87 ##########################
4501-5000 8.05 ########################################
5000 episodes · 216327 steps · 3.4 s · 63723 steps/s · final avg 8.05
last bucket avg score 8.45 · best score 25 · 254 of 288 states visited
grid string
above by the trainer, not handed to it. The three hand-driven steps show the
reward doing its job: the first move closes on the food and pays a hundredth,
the next two overshoot it and are charged a hundredth each. Nothing here plans
around its own tail, and the growing snake is what eventually kills it.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"turn": 0|1|2} — left, straight, right;
defaults to straight |
| observation | input_text, grid as text
rows, plus head_x, head_y, food_x,
food_y, dir, length,
score |
| reward | 1.0 for food, -1.0 for dying,
and a hundredth either way for closing on the food or drifting off it |
| done | wall, tail, a full board, or too long without eating |
2048, and the state that could not see a direction
The tile game, with the real merge rules: one merge per tile per move, a new 2 or 4 after any move that changed something, and a score that is the sum of what merged. It is here because the upstream project's own reinforcement-learning tutorial trains against a Python 2048 — and because it is the one game on this page whose first learner lost to random play, which turned out to be a lesson about features rather than about 2048.
$ build/rel/bin/g2048 --host 127.0.0.1 --port 8804 &
$ build/rel/bin/oe_probe http://127.0.0.1:8804 --episode --steps 3 --action '{"dir":0}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"2048 on a 4x4 board, '.' empty. Equal neighbours merge once each per move, and a merge scores the tile it makes.\n . . . .\n 2 . . .\n . . . .\n . . . 4\nScore 0, best tile 4. A direction that shifts nothing is legal and does nothing.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","board":[0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,4],"score":0,"max_tile":4,"moved":false},"reward":null,"done":false}}
>> {"type":"step","data":{"dir":0}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"2048 on a 4x4 board, '.' empty. Equal neighbours merge once each per move, and a merge scores the tile it makes.\n 2 . . 4\n . . . .\n . . . 4\n . . . .\nScore 0, best tile 4. A direction that shifts nothing is legal and does nothing.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","board":[2,0,0,4,0,0,0,0,0,0,0,4,0,0,0,0],"score":0,"max_tile":4,"moved":true},"reward":0,"done":false}}
>> {"type":"step","data":{"dir":0}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"2048 on a 4x4 board, '.' empty. Equal neighbours merge once each per move, and a merge scores the tile it makes.\n 2 . 4 8\n . . . .\n . . . .\n . . . .\nScore 8, best tile 8. A direction that shifts nothing is legal and does nothing.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","board":[2,0,4,8,0,0,0,0,0,0,0,0,0,0,0,0],"score":8,"max_tile":8,"moved":true},"reward":8,"done":false}}
>> {"type":"step","data":{"dir":0}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"2048 on a 4x4 board, '.' empty. Equal neighbours merge once each per move, and a merge scores the tile it makes.\n 2 . 4 8\n . . . .\n . . . .\n . . . .\nScore 8, best tile 8. A direction that shifts nothing is legal and does nothing.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","board":[2,0,4,8,0,0,0,0,0,0,0,0,0,0,0,0],"score":8,"max_tile":8,"moved":false},"reward":0,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"score":8}}
>> {"type":"close"}
$ build/rel/bin/train g2048 http://127.0.0.1:8804 --seed 7
train g2048 — tabular Q-learning, 200 coarse states x 4 actions
learns when to keep pushing one way and when the board wants another
does not see the tiles it is moving, count merges, or reach 2048
server http://127.0.0.1:8804 — Game2048Environment
run 6000 episodes · seed 7 · eps 0.20->0.02 · alpha 0.02 · gamma 0.95 · 200 random games first
episodes avg return curve (bars span 1376.39 to 1875.63)
1- 600 1376.39
601-1200 1581.61 ################
1201-1800 1558.19 ###############
1801-2400 1703.13 ##########################
2401-3000 1730.76 ############################
3001-3600 1742.35 #############################
3601-4200 1875.63 ########################################
4201-4800 1782.37 #################################
4801-5400 1869.58 ########################################
5401-6000 1790.95 #################################
6000 episodes · 1056853 steps · 14.6 s · 72151 steps/s · final avg 1790.95
random baseline 200 games · avg score 979 · best tile 256 · learned best tile 512
{"dir":0} comes back
"moved":false, reward 0, the board unchanged. That run finished
under the random baseline it printed beside itself. Adding what the last move
did (which way, and whether anything moved) turns a table of preferences into a
table of plans: push this way while it works, and when it stops working push
that way instead. It still will not reach 2048, and the curve plateaus after
about 3600 games rather than climbing to the end — the last bucket is below the
peak two buckets earlier. 1056853 steps went over the socket in 12.2 seconds,
so that part is not what is stopping you.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"dir": 0|1|2|3} — up, right, down, left;
required, and a move that shifts nothing is legal |
| observation | input_text, board as
sixteen integers row-major, plus score, max_tile,
moved — and moved is the field the trainer
needed |
| reward | integer: the score gained by this move |
| done | no direction moves anything |
Strings in, colours back
The word game, with the real two-pass rule: greens first, then yellows against whatever letters are left, so a repeated letter behaves exactly as it does in the original. The action is a string rather than a number, which makes this the game on the page most like the ones people actually want to train language models against.
$ build/rel/bin/wordle --host 127.0.0.1 --port 8805 &
$ build/rel/bin/oe_probe http://127.0.0.1:8805 --episode --steps 3 --action '{"guess":"crane"}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Wordle: guess the hidden five-letter word in six tries. Each guess is scored letter by letter: G is the right letter in the right place, Y the right letter in the wrong place, X a letter the word does not have. A guess must be a word on the shared list; one that is not is scored and costs no attempt.\nGuess 1 of 6.\nReply with one five-letter word and nothing else.","feedback":"","valid":false,"guesses":0,"history":"","solved":false,"answer":""},"reward":null,"done":false}}
>> {"type":"step","data":{"guess":"crane"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Wordle: guess the hidden five-letter word in six tries. Each guess is scored letter by letter: G is the right letter in the right place, Y the right letter in the wrong place, X a letter the word does not have. A guess must be a word on the shared list; one that is not is scored and costs no attempt.\ncrane XXXYY\nGuess 2 of 6.\nReply with one five-letter word and nothing else.","feedback":"XXXYY","valid":true,"guesses":1,"history":"crane XXXYY","solved":false,"answer":""},"reward":0.1,"done":false}}
>> {"type":"step","data":{"guess":"crane"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Wordle: guess the hidden five-letter word in six tries. Each guess is scored letter by letter: G is the right letter in the right place, Y the right letter in the wrong place, X a letter the word does not have. A guess must be a word on the shared list; one that is not is scored and costs no attempt.\ncrane XXXYY\ncrane XXXYY\nGuess 3 of 6.\nReply with one five-letter word and nothing else.","feedback":"XXXYY","valid":true,"guesses":2,"history":"crane XXXYY\ncrane XXXYY","solved":false,"answer":""},"reward":0.1,"done":false}}
>> {"type":"step","data":{"guess":"crane"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Wordle: guess the hidden five-letter word in six tries. Each guess is scored letter by letter: G is the right letter in the right place, Y the right letter in the wrong place, X a letter the word does not have. A guess must be a word on the shared list; one that is not is scored and costs no attempt.\ncrane XXXYY\ncrane XXXYY\ncrane XXXYY\nGuess 4 of 6.\nReply with one five-letter word and nothing else.","feedback":"XXXYY","valid":true,"guesses":3,"history":"crane XXXYY\ncrane XXXYY\ncrane XXXYY","solved":false,"answer":""},"reward":0.1,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"guesses":3}}
>> {"type":"close"}
$ build/rel/bin/train wordle http://127.0.0.1:8805 --seed 7
train wordle — programmed deduction, epsilon-greedy over 40 openers
learns only the opening word, scored by what the whole game returned
does not learn the deduction: filtering the list by feedback is written here
server http://127.0.0.1:8805 — WordleEnvironment
run 10000 games · seed 7 · eps 1.00->0.05 · 40 openers · 1466-word list
episodes avg return curve (bars span 3.06 to 3.16)
1- 1000 3.06
1001- 2000 3.06
2001- 3000 3.08 ######
3001- 4000 3.09 ##########
4001- 5000 3.11 ##################
5001- 6000 3.10 ################
6001- 7000 3.13 ############################
7001- 8000 3.15 #####################################
8001- 9000 3.15 ####################################
9001-10000 3.16 ########################################
10000 episodes · 37843 steps · 0.9 s · 41457 steps/s · final avg 3.16
solved 99.0% in 3.76 guesses · favourite opener "arose" solves in 3.68 · worst of the 40 "fuzzy" 4.19
crane three times and being paid 0.1 each
time for it. Solving now pays 0.50 more for every attempt it did not need. And
the forty candidate openers were the forty best words in the
list, which all solve in about the same number of guesses — there was nothing
to choose between. Ranked by letter frequency and sampled across the whole
ranking, the pool now runs from the best question in the dictionary to the
worst, and from returns alone the trainer worked out that
arose gets there in 3.68 guesses while fuzzy takes
4.19.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"guess": "crane"} — required, lowercased
before it is judged |
| observation | input_text, feedback as
five of G, Y and X, plus valid, guesses,
history, solved, answer |
| reward | a tenth per green and a twentieth per yellow, one more for solving plus half an attempt's worth for every guess it did not need, a fifth back for a word not in the list |
| done | solved, six valid guesses, or the step cap |
The textbook's own game, dealt over a socket
Sutton and Barto's Example 5.1, dealt over a socket: a deck that never runs out, one decision — hit or stand — and a dealer with no choices at all. It is the plate for a reward you can only sample, and for a curve that climbs and then stops below zero.
$ build/rel/bin/blackjack --host 127.0.0.1 --port 8806 &
$ build/rel/bin/oe_probe http://127.0.0.1:8806 --episode --steps 3 --action '{"hit":1}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Blackjack from an infinite deck. Your hand is 12; the dealer shows 1. You have no ace worth 11.\nOver 21 loses at once. On a stand the dealer draws to 17 and stands, soft 17 included.\nReply with one digit: 0 stand, 1 take a card.","player_sum":12,"dealer_up":1,"usable_ace":false,"dealer_sum":0},"reward":null,"done":false}}
>> {"type":"step","data":{"hit":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Blackjack from an infinite deck. Your hand is 17; the dealer shows 1. You have no ace worth 11.\nOver 21 loses at once. On a stand the dealer draws to 17 and stands, soft 17 included.\nReply with one digit: 0 stand, 1 take a card.","player_sum":17,"dealer_up":1,"usable_ace":false,"dealer_sum":0},"reward":0,"done":false}}
>> {"type":"step","data":{"hit":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Blackjack from an infinite deck. Your hand is 21; the dealer shows 1. You have no ace worth 11.\nOver 21 loses at once. On a stand the dealer draws to 17 and stands, soft 17 included.\nReply with one digit: 0 stand, 1 take a card.","player_sum":21,"dealer_up":1,"usable_ace":false,"dealer_sum":0},"reward":0,"done":false}}
>> {"type":"step","data":{"hit":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Blackjack from an infinite deck. Your hand is 31; the dealer shows 1. You have no ace worth 11.\nOver 21 loses at once. On a stand the dealer draws to 17 and stands, soft 17 included.\nThe hand is over: you 31, dealer 21.","player_sum":31,"dealer_up":1,"usable_ace":false,"dealer_sum":21},"reward":-1,"done":true}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"player_sum":31}}
>> {"type":"close"}
$ build/rel/bin/train blackjack http://127.0.0.1:8806 --seed 7
train blackjack — every-visit Monte Carlo control, 200 states x 2 actions
learns when to hit and when to stand, from what whole hands paid
does not hit under twelve by choice (that is a rule here), count cards, or win
server http://127.0.0.1:8806 — BlackjackEnvironment
run 40000 hands · seed 7 · eps 1.00->0.05 · sample-average returns · no alpha, no gamma
hands avg return curve (bars span -0.39 to -0.08)
1- 4000 -0.39
4001- 8000 -0.32 #########
8001-12000 -0.29 #############
12001-16000 -0.25 ##################
16001-20000 -0.25 ##################
20001-24000 -0.21 #######################
24001-28000 -0.19 ##########################
28001-32000 -0.15 ###############################
32001-36000 -0.11 ####################################
36001-40000 -0.08 ########################################
40000 hands · 58170 steps · 1.2 s · 49238 steps/s · final avg -0.08
won 42.0% · pushed 7.7% · lost 50.3% over the last bucket · the dealer stands on every 17
| the contract | shape |
|---|---|
| action | {"hit": 1} take a card, {"hit": 0}
stand — required |
| observation | input_text, player_sum,
dealer_up, usable_ace, and dealer_sum,
which is 0 until the hand is over and the dealer's final total after |
| reward | integer +1, 0 or -1, on the terminal step only; a non-terminal hit pays nothing |
| done | a bust, or the dealer finishing once you stand |
The world is the seed
A maze of 225 squares, drawn as text, with you at the top left and the goal
at the bottom right. It is not a fixture: every reset carves a new
one out of the seed it was handed, so the observation below did not exist until
it was asked for.
$ build/rel/bin/maze --host 127.0.0.1 --port 8807 &
$ build/rel/bin/oe_probe http://127.0.0.1:8807 --episode --steps 3 --action '{"move":1}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A 15x15 maze. '#' wall, '.' floor, 'A' you, 'G' the goal. A wall absorbs the move: the step is spent and you stay.\n###############\n#A..........#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############\nYou are at column 1, row 1; row 0 is the top. The goal is at column 13, row 13. Step 1 of 250.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","grid":"###############\n#A..........#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############","x":1,"y":1,"goal_x":13,"goal_y":13},"reward":null,"done":false}}
>> {"type":"step","data":{"move":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A 15x15 maze. '#' wall, '.' floor, 'A' you, 'G' the goal. A wall absorbs the move: the step is spent and you stay.\n###############\n#.A.........#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############\nYou are at column 2, row 1; row 0 is the top. The goal is at column 13, row 13. Step 2 of 250.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","grid":"###############\n#.A.........#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############","x":2,"y":1,"goal_x":13,"goal_y":13},"reward":-0.01,"done":false}}
>> {"type":"step","data":{"move":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A 15x15 maze. '#' wall, '.' floor, 'A' you, 'G' the goal. A wall absorbs the move: the step is spent and you stay.\n###############\n#..A........#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############\nYou are at column 3, row 1; row 0 is the top. The goal is at column 13, row 13. Step 3 of 250.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","grid":"###############\n#..A........#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############","x":3,"y":1,"goal_x":13,"goal_y":13},"reward":-0.01,"done":false}}
>> {"type":"step","data":{"move":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"A 15x15 maze. '#' wall, '.' floor, 'A' you, 'G' the goal. A wall absorbs the move: the step is spent and you stay.\n###############\n#...A.......#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############\nYou are at column 4, row 1; row 0 is the top. The goal is at column 13, row 13. Step 4 of 250.\nReply with one digit: 0 up, 1 right, 2 down, 3 left.","grid":"###############\n#...A.......#.#\n#.#########.#.#\n#.#.#.......#.#\n#.#.#.#######.#\n#.#.#.#.......#\n#.#.#####.###.#\n#.#.......#...#\n#.###.#####.###\n#.#...#...#...#\n#.#.###.#.###.#\n#.#.#.#.#.....#\n#.#.#.#.#######\n#...#........G#\n###############","x":4,"y":1,"goal_x":13,"goal_y":13},"reward":-0.01,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"manhattan":21}}
>> {"type":"close"}
$ build/rel/bin/train maze http://127.0.0.1:8807 --seed 7
train maze — tabular Q-learning over position, 225 states x 4 actions
learns the way through THIS maze, by bumping into every wall of it
does not carry any of it to another seed: a table of squares is one map
server http://127.0.0.1:8807 — MazeEnvironment
run 2000 episodes · seed 7 · eps 1.00->0.05 · alpha 0.20 · gamma 0.95 · one maze, replayed every episode
episodes avg return curve (bars span -2.45 to 0.70)
1- 200 -2.45
201- 400 -1.32 ##############
401- 600 -0.26 ############################
601- 800 0.11 #################################
801-1000 0.32 ###################################
1001-1200 0.45 #####################################
1201-1400 0.54 ######################################
1401-1600 0.62 #######################################
1601-1800 0.66 ########################################
1801-2000 0.70 ########################################
2000 episodes · 187862 steps · 3.0 s · 63405 steps/s · final avg 0.70
shortest path 28 steps · best walk 28 · first bucket averaged 248 steps to done, last 31
--seed and it is lost again, which is the honest reading
of "procedurally generated".
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"move": 0..3} — up, right, down, left;
required. Walking into a wall stays put and costs the step: a bump is how a
blind walker finds a wall, not an error |
| observation | input_text, grid as text
rows, plus x, y, goal_x,
goal_y |
| reward | 1.0 for stepping onto the goal,
-0.01 for every other step, bumps included — sparse on purpose,
because the signal lives at the end |
| done | the goal, or the step cap |
The most famous rally in computing, one paddle of it
A ball on a unit court, a paddle on the left wall, and three choices: up, stay, down. Where the ball meets the paddle decides where it goes next, so placement is worth learning, and what the curve measures is the rally itself.
$ build/rel/bin/pong --host 127.0.0.1 --port 8808 &
$ build/rel/bin/oe_probe http://127.0.0.1:8808 --episode --steps 3 --action '{"paddle":1}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Half a game of Pong: one paddle on the left wall of a square court, y = 0 at the top. Return the ball; a miss ends the episode.\nBall at x 0.500, y 0.730, moving -0.040 across and -0.004 down. Your paddle centre 0.500, half-height 0.080, travel 0.050 a step. Rally 0. Three decimals here.\nReply with one digit: 0 up, 1 stay, 2 down.","ball_x":0.5,"ball_y":0.7299864849281856,"ball_vx":-0.04,"ball_vy":-0.004108320177089402,"paddle_y":0.5,"rally":0},"reward":null,"done":false}}
>> {"type":"step","data":{"paddle":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Half a game of Pong: one paddle on the left wall of a square court, y = 0 at the top. Return the ball; a miss ends the episode.\nBall at x 0.460, y 0.726, moving -0.040 across and -0.004 down. Your paddle centre 0.500, half-height 0.080, travel 0.050 a step. Rally 0. Three decimals here.\nReply with one digit: 0 up, 1 stay, 2 down.","ball_x":0.46,"ball_y":0.7258781647510962,"ball_vx":-0.04,"ball_vy":-0.004108320177089402,"paddle_y":0.5,"rally":0},"reward":0.0,"done":false}}
>> {"type":"step","data":{"paddle":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Half a game of Pong: one paddle on the left wall of a square court, y = 0 at the top. Return the ball; a miss ends the episode.\nBall at x 0.420, y 0.722, moving -0.040 across and -0.004 down. Your paddle centre 0.500, half-height 0.080, travel 0.050 a step. Rally 0. Three decimals here.\nReply with one digit: 0 up, 1 stay, 2 down.","ball_x":0.42000000000000004,"ball_y":0.7217698445740068,"ball_vx":-0.04,"ball_vy":-0.004108320177089402,"paddle_y":0.5,"rally":0},"reward":0.0,"done":false}}
>> {"type":"step","data":{"paddle":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Half a game of Pong: one paddle on the left wall of a square court, y = 0 at the top. Return the ball; a miss ends the episode.\nBall at x 0.380, y 0.718, moving -0.040 across and -0.004 down. Your paddle centre 0.500, half-height 0.080, travel 0.050 a step. Rally 0. Three decimals here.\nReply with one digit: 0 up, 1 stay, 2 down.","ball_x":0.38000000000000006,"ball_y":0.7176615243969174,"ball_vx":-0.04,"ball_vy":-0.004108320177089402,"paddle_y":0.5,"rally":0},"reward":0.0,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"rally":0}}
>> {"type":"close"}
$ build/rel/bin/train pong http://127.0.0.1:8808 --seed 7
train pong — tabular Q-learning, 126 boxes x 3 actions
learns to be where the ball will arrive, from the gap it can see now
does not aim its returns, read the english it puts on, or see the court
server http://127.0.0.1:8808 — PongEnvironment
run 2500 episodes · seed 7 · eps 1.00->0.05 · alpha 0.20 · gamma 0.95
episodes avg return curve (bars span -0.77 to 4.13)
1- 250 -0.77
251- 500 -0.70 #
501- 750 -0.51 ##
751-1000 -0.41 ###
1001-1250 -0.22 ####
1251-1500 0.04 #######
1501-1750 0.42 ##########
1751-2000 1.06 ###############
2001-2250 2.77 #############################
2251-2500 4.13 ########################################
2500 episodes · 230050 steps · 4.9 s · 46729 steps/s · final avg 4.13
longest rally 29 returns · first bucket averaged -0.8, last 4.1 · 84 of 126 boxes visited
avg return going from -0.77 to 4.13 is
the rally growing. No proxy, no invented score, nothing to argue with. The
longest single rally in the run was 29 returns. It gets there on 126 boxes and
visited 84 of them: the ball-to-paddle gap in seven slices, the ball's distance
in three, and the sign of each velocity. That is enough to be where the ball
will arrive. It is not enough to decide where to send it back, and this trainer
never tries.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"paddle": 0|1|2} — up, stay, down;
defaults to stay |
| observation | input_text, ball_x,
ball_y, ball_vx, ball_vy,
paddle_y, and rally, the returns so far this
episode |
| reward | 1.0 the step it returns the ball,
-1.0 the step it misses, 0.0 otherwise |
| done | a miss, or the step cap |
The opponent lives in the env, and legal_actions is just a field
Seven columns, six rows, and an opponent that is part of the environment: you drop a disc, and before the observation comes back it has dropped one too. It wins if it can and blocks if it must, so what the curve shows is a learner discovering that stopping a four is worth about as much as taking one.
$ build/rel/bin/connect4 --host 127.0.0.1 --port 8809 &
$ build/rel/bin/oe_probe http://127.0.0.1:8809 --episode --steps 3 --action '{"col":3}' | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Connect Four, seven columns by six rows. You are 'X' and move first; the environment is 'O' and answers inside your step. Four in a line wins.\n0123456\n.......\n.......\n.......\n.......\n.......\n.......\nColumns with room: 0 1 2 3 4 5 6. It has not answered yet.\nReply with one digit, the column to drop into.","board":".......\n.......\n.......\n.......\n.......\n.......","legal_actions":[0,1,2,3,4,5,6],"last_env_col":-1},"reward":null,"done":false}}
>> {"type":"step","data":{"col":3}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Connect Four, seven columns by six rows. You are 'X' and move first; the environment is 'O' and answers inside your step. Four in a line wins.\n0123456\n.......\n.......\n.......\n.......\n.......\n..OX...\nColumns with room: 0 1 2 3 4 5 6. It last played column 2.\nReply with one digit, the column to drop into.","board":".......\n.......\n.......\n.......\n.......\n..OX...","legal_actions":[0,1,2,3,4,5,6],"last_env_col":2},"reward":0,"done":false}}
>> {"type":"step","data":{"col":3}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Connect Four, seven columns by six rows. You are 'X' and move first; the environment is 'O' and answers inside your step. Four in a line wins.\n0123456\n.......\n.......\n.......\n.......\n...X...\n.OOX...\nColumns with room: 0 1 2 3 4 5 6. It last played column 1.\nReply with one digit, the column to drop into.","board":".......\n.......\n.......\n.......\n...X...\n.OOX...","legal_actions":[0,1,2,3,4,5,6],"last_env_col":1},"reward":0,"done":false}}
>> {"type":"step","data":{"col":3}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Connect Four, seven columns by six rows. You are 'X' and move first; the environment is 'O' and answers inside your step. Four in a line wins.\n0123456\n.......\n.......\n...O...\n...X...\n...X...\n.OOX...\nColumns with room: 0 1 2 3 4 5 6. It last played column 3.\nReply with one digit, the column to drop into.","board":".......\n.......\n...O...\n...X...\n...X...\n.OOX...","legal_actions":[0,1,2,3,4,5,6],"last_env_col":3},"reward":0,"done":false}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"discs":6}}
>> {"type":"close"}
$ build/rel/bin/train connect4 http://127.0.0.1:8809 --seed 7
train connect4 — afterstate values, 72 positions after the drop, masked
learns which column leaves the better position: taking a four, and stopping one
does not see a fork coming, plan two moves ahead, or see most of the board
server http://127.0.0.1:8809 — Connect4Environment
run 5000 games · seed 7 · eps 1.00->0.05 · alpha 0.15 · gamma 0.95 · masked by legal_actions
episodes avg return curve (bars span -0.85 to -0.03)
1- 500 -0.85
501-1000 -0.80 ###
1001-1500 -0.79 ###
1501-2000 -0.66 ##########
2001-2500 -0.61 ############
2501-3000 -0.50 #################
3001-3500 -0.34 #########################
3501-4000 -0.23 ##############################
4001-4500 -0.09 #####################################
4501-5000 -0.03 ########################################
5000 episodes · 56020 steps · 1.1 s · 49172 steps/s · final avg -0.03
won 45.6% · lost 48.8% · drew 5.6% over the last bucket · blocked 76.4% of the threats it was shown
legal_actions. The protocol has never heard of that field. This
environment invented it, because its action space shrinks as columns fill. The
schema advertises it exactly the way it advertises board, and the
trainer pulls it out of the observation like any other field and masks its
argmax with it. That is the whole demonstration: masked by
legal_actions is in the run line above, and that array is the mask's only
source. Computing legality client-side from the rules would have been easy, and
would have quietly ended it — the protocol carries observations, and what an
observation means is the environment's own business. Upstream's
connect4 environment publishes a legal set too, because its author had the same
problem, not because a standard told either of them to. It is a real constraint
and not a decoration: a col that is in range but full is still
refused, with the protocol's EXECUTION_ERROR frame. That is the
arcade's first error that depends on the state and not on the argument. What the
mask bought, over the last five hundred games: 45.6% won, 48.8% lost, 5.6%
drawn, and 76.4% of the threats it was shown blocked in the column it was shown
them in.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"col": 0..6} — required. A full column is
in range and still refused, with an EXECUTION_ERROR frame |
| observation | input_text, board as six
text rows, top row first, plus legal_actions — the columns not yet
full — and last_env_col, where the env just played |
| reward | integer on the terminal step: +1 your four, -1 its four, 0 a full board |
| done | either four, or the board fills |
The smallest real poker, and a chart that wins downward
Three cards, one chip each in the pot, and one decision per player: pass, or bet a chip. One session deals both seats, and the observation only ever describes the seat to act — so the other player's card stays off the wire until the hand is over.
$ build/rel/bin/kuhn --host 127.0.0.1 --port 8810 &
$ build/rel/bin/oe_probe http://127.0.0.1:8810 --episode --steps 3 --action '{"bet":1}' --past-done | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Kuhn poker, three cards: 1 jack, 2 queen, 3 king. One to each seat, one face down, a chip anted by both. You play both seats in turn.\nYou are seat 0 and hold the jack. Pot 2. Betting so far: nothing.\nFacing a bet, 0 folds and 1 calls. Pass-pass or a call goes to showdown, and the higher card takes half the pot.\nReply with one digit: 0 pass, 1 bet one chip.","to_act":0,"card":1,"history":"","pot":2,"deal":""},"reward":null,"done":false}}
>> {"type":"step","data":{"bet":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Kuhn poker, three cards: 1 jack, 2 queen, 3 king. One to each seat, one face down, a chip anted by both. You play both seats in turn.\nYou are seat 1 and hold the king. Pot 3. Betting so far: bet.\nFacing a bet, 0 folds and 1 calls. Pass-pass or a call goes to showdown, and the higher card takes half the pot.\nReply with one digit: 0 pass, 1 bet one chip.","to_act":1,"card":3,"history":"b","pot":3,"deal":""},"reward":0,"done":false}}
>> {"type":"step","data":{"bet":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Kuhn poker, three cards: 1 jack, 2 queen, 3 king. One to each seat, one face down, a chip anted by both. You play both seats in turn.\nYou are seat 0 and hold the jack. Pot 4. Betting so far: bet, bet.\nFacing a bet, 0 folds and 1 calls. Pass-pass or a call goes to showdown, and the higher card takes half the pot.\nThe hand is over. Both cards: JK.","to_act":0,"card":1,"history":"bb","pot":4,"deal":"JK"},"reward":-2,"done":true}}
>> {"type":"step","data":{"bet":1}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Kuhn poker, three cards: 1 jack, 2 queen, 3 king. One to each seat, one face down, a chip anted by both. You play both seats in turn.\nYou are seat 0 and hold the jack. Pot 4. Betting so far: bet, bet.\nFacing a bet, 0 folds and 1 calls. Pass-pass or a call goes to showdown, and the higher card takes half the pot.\nThe hand is over. Both cards: JK.","to_act":0,"card":1,"history":"bb","pot":4,"deal":"JK"},"reward":null,"done":true}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":2,"pot":4}}
>> {"type":"close"}
$ build/rel/bin/train kuhn http://127.0.0.1:8810 --seed 7
train kuhn — outcome-sampling regret matching, 12 information sets
learns a mixed strategy for both seats, from what one sampled hand paid
does not reach Nash, see the other seat's card, or ever stop exploring
server http://127.0.0.1:8810 — KuhnPokerEnvironment
run 60000 hands · seed 7 · regret matching · 0.10 exploration floor · 12 information sets
hands exploitability curve (bars span 0.13 down to 0.02)
1- 6000 0.13
6001-12000 0.05 ############################
12001-18000 0.04 #################################
18001-24000 0.02 ########################################
24001-30000 0.03 #####################################
30001-36000 0.03 ######################################
36001-42000 0.03 ####################################
42001-48000 0.05 ###############################
48001-54000 0.06 ###########################
54001-60000 0.04 #################################
60000 hands · 143480 steps · 2.6 s · 54924 steps/s · final avg 0.04
exploitability 0.041 chips a hand · seat 0 averaged -0.060 (the equilibrium pays -1/18) · bluffs a jack 15% · bets a king 41% · calls a bet with the queen 35% (one in three at equilibrium)
bars span
0.13 down to 0.02. Forty hashes still means the best this run reached,
and the words down to say which way that is. What falls is a
distance, never an arrival. The run ends at 0.041 chips a hand and is still
moving, so the strongest word this page may use is toward. The learner
is different in kind as well: regret matching rather than a value table, playing
a mixed strategy and accumulating the average of it. The average is the object
that converges; the current strategy never has to. The frequencies it settled on
are the game's own known shapes. It bluffs a jack 15% of the time and bets a
king 41%, where the known equilibria bet the king three times as often as they
bluff the jack. It calls with the queen 35% of the time against one in three at
equilibrium, and seat 0 averaged -0.060 a hand in a game whose equilibrium pays
-1/18. One last thing, in the hand above rather than the chart: the third bet
is taken after done came back true, which is what
--past-done is for. The answer is the whole after-done rule in one
frame — the terminal observation byte for byte, reward null,
done still true — and the state door still says
step_count 2, because a step past the end changes nothing, the
count included.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"bet": 0} pass or {"bet": 1}
bet a chip — required |
| observation | input_text, to_act,
card — that seat's card and never the other's —
history as a string of p and b, pot, and
deal, empty until the hand is over |
| reward | integer, seat 0's chips, on the terminal step; the game is zero-sum and the trainer flips the sign for seat 1 itself |
| done | a fold, pass-pass, or a called bet |
Four tasks that are not games
The four plates that follow are not games. Each server mints a problem as
text, computes its own answer in the same call, and pays reward 1
for an exact match and 0 for anything else. Four tasks, four skills:
dyna is state tracking, inducto is
rule induction, logic is circuit evaluation, and
filtro is source separation.
Reset is a generator, and every reset mints a fresh task from its seed. The runs below open with the measurement: a thousand seeds, a thousand distinct tasks, no repeats, at tens of thousands of tasks a second over the wire. The same seed twice is byte-identical. Difficulty is a knob on reset (1, 2 or 3). Dyna, logic and filtro sweep all three and print a row apiece, so the table names what the knob moves. Inducto holds it at 3 and spends its run on a learning curve instead.
Each task is a prompt, an answer, and a reward the environment can defend —
the verifiable-reward shape a GRPO-style trainer consumes. (GRPO: sample
several answers per prompt, score each, push the model toward the ones the
reward preferred;
TRL ↗'s
GRPOTrainer is one.) Each generator was measured against a
battery of cheap readers — answer by position, by output length, by one token,
by the previous answer — and cut until those shortcuts price at chance. Each
caption below names what its task resists, and
the battery prints the ledger. To train a language model
against them, start at docs/TRAINING.md.
Three of the four runs below print learns nothing on their
own second line. That is because these are text-in, text-out tasks for a
language model, not value-table games. The client parses the text and computes
the answer. That proves the reward channel end to end, and prices what
guessing pays beside it. Only inducto has a decision buried in it,
and only inducto gets a curve.
State tracking: the answer exists only after the replay
State tracking is holding a changing arrangement in your head, and this server grades exactly that. A list of symbols is shuffled, then a run of operations is applied to it — swap two, rotate left, rotate right, reverse a segment — and three questions are asked about the arrangement that comes out: what is standing at this position, or where did this element end up. The answers are nowhere in the text. They exist only after somebody replays every operation in order, which is what makes this state tracking and not a lookup.
$ build/rel/bin/dyna --host 127.0.0.1 --port 8821 &
$ build/rel/bin/oe_probe http://127.0.0.1:8821 --episode --steps 4 --action '{"answer":"B"}' --past-done | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"STATE [C, F, A, D, E, B] || OPS SWAP(5,0) | ROTATE_LEFT(2) | REVERSE(2,3) | REVERSE(2,3) | SWAP(4,5) | ROTATE_LEFT(4) | SWAP(3,5) | SWAP(4,3) | ROTATE_RIGHT(2) | ROTATE_LEFT(5) | ROTATE_LEFT(3) | REVERSE(2,5) | REVERSE(0,3) | ROTATE_RIGHT(1) | REVERSE(0,5) | ROTATE_LEFT(1) | ROTATE_LEFT(5) | ROTATE_LEFT(2) | REVERSE(3,5) | ROTATE_LEFT(5) | SWAP(0,2) | REVERSE(0,5) | REVERSE(0,2) | ROTATE_RIGHT(4) | SWAP(4,0) | REVERSE(2,4) | ROTATE_LEFT(1) | SWAP(1,4) | ROTATE_RIGHT(3) | SWAP(0,1) || QUERY element D ->","difficulty":1,"state_size":6,"num_ops":30,"query_index":1,"answer":"","final_state":""},"reward":null,"done":false}}
>> {"type":"step","data":{"answer":"B"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"STATE [C, F, A, D, E, B] || OPS SWAP(5,0) | ROTATE_LEFT(2) | REVERSE(2,3) | REVERSE(2,3) | SWAP(4,5) | ROTATE_LEFT(4) | SWAP(3,5) | SWAP(4,3) | ROTATE_RIGHT(2) | ROTATE_LEFT(5) | ROTATE_LEFT(3) | REVERSE(2,5) | REVERSE(0,3) | ROTATE_RIGHT(1) | REVERSE(0,5) | ROTATE_LEFT(1) | ROTATE_LEFT(5) | ROTATE_LEFT(2) | REVERSE(3,5) | ROTATE_LEFT(5) | SWAP(0,2) | REVERSE(0,5) | REVERSE(0,2) | ROTATE_RIGHT(4) | SWAP(4,0) | REVERSE(2,4) | ROTATE_LEFT(1) | SWAP(1,4) | ROTATE_RIGHT(3) | SWAP(0,1) || QUERY position 1 ->","difficulty":1,"state_size":6,"num_ops":30,"query_index":2,"answer":"","final_state":""},"reward":0,"done":false}}
>> {"type":"step","data":{"answer":"B"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"STATE [C, F, A, D, E, B] || OPS SWAP(5,0) | ROTATE_LEFT(2) | REVERSE(2,3) | REVERSE(2,3) | SWAP(4,5) | ROTATE_LEFT(4) | SWAP(3,5) | SWAP(4,3) | ROTATE_RIGHT(2) | ROTATE_LEFT(5) | ROTATE_LEFT(3) | REVERSE(2,5) | REVERSE(0,3) | ROTATE_RIGHT(1) | REVERSE(0,5) | ROTATE_LEFT(1) | ROTATE_LEFT(5) | ROTATE_LEFT(2) | REVERSE(3,5) | ROTATE_LEFT(5) | SWAP(0,2) | REVERSE(0,5) | REVERSE(0,2) | ROTATE_RIGHT(4) | SWAP(4,0) | REVERSE(2,4) | ROTATE_LEFT(1) | SWAP(1,4) | ROTATE_RIGHT(3) | SWAP(0,1) || QUERY element E ->","difficulty":1,"state_size":6,"num_ops":30,"query_index":3,"answer":"","final_state":""},"reward":1,"done":false}}
>> {"type":"step","data":{"answer":"B"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"STATE [C, F, A, D, E, B] || OPS SWAP(5,0) | ROTATE_LEFT(2) | REVERSE(2,3) | REVERSE(2,3) | SWAP(4,5) | ROTATE_LEFT(4) | SWAP(3,5) | SWAP(4,3) | ROTATE_RIGHT(2) | ROTATE_LEFT(5) | ROTATE_LEFT(3) | REVERSE(2,5) | REVERSE(0,3) | ROTATE_RIGHT(1) | REVERSE(0,5) | ROTATE_LEFT(1) | ROTATE_LEFT(5) | ROTATE_LEFT(2) | REVERSE(3,5) | ROTATE_LEFT(5) | SWAP(0,2) | REVERSE(0,5) | REVERSE(0,2) | ROTATE_RIGHT(4) | SWAP(4,0) | REVERSE(2,4) | ROTATE_LEFT(1) | SWAP(1,4) | ROTATE_RIGHT(3) | SWAP(0,1) || QUERY element E ->","difficulty":1,"state_size":6,"num_ops":30,"query_index":3,"answer":"0","final_state":"[E, B, C, A, F, D]"},"reward":0,"done":true}}
>> {"type":"step","data":{"answer":"B"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"STATE [C, F, A, D, E, B] || OPS SWAP(5,0) | ROTATE_LEFT(2) | REVERSE(2,3) | REVERSE(2,3) | SWAP(4,5) | ROTATE_LEFT(4) | SWAP(3,5) | SWAP(4,3) | ROTATE_RIGHT(2) | ROTATE_LEFT(5) | ROTATE_LEFT(3) | REVERSE(2,5) | REVERSE(0,3) | ROTATE_RIGHT(1) | REVERSE(0,5) | ROTATE_LEFT(1) | ROTATE_LEFT(5) | ROTATE_LEFT(2) | REVERSE(3,5) | ROTATE_LEFT(5) | SWAP(0,2) | REVERSE(0,5) | REVERSE(0,2) | ROTATE_RIGHT(4) | SWAP(4,0) | REVERSE(2,4) | ROTATE_LEFT(1) | SWAP(1,4) | ROTATE_RIGHT(3) | SWAP(0,1) || QUERY element E ->","difficulty":1,"state_size":6,"num_ops":30,"query_index":3,"answer":"0","final_state":"[E, B, C, A, F, D]"},"reward":null,"done":true}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":3,"num_ops":30}}
>> {"type":"close"}
$ build/rel/bin/synthlab dyna http://127.0.0.1:8821 --seed 7
synthlab dyna — programmed replay, a seeded chance arm, presets 1-3 swept, three queries a task
learns nothing — the replay is programmed: parse the text, apply the ops, answer
does not dress computation as learning; the chance arm beside it is what guessing pays
server http://127.0.0.1:8821 — DynaEnvironment
run 1500 solver episodes · 1500 chance episodes · seed 7 · 500 per preset
demo 1000 seeds -> 1000 distinct tasks · 0 repeats · distinct by 64-bit FNV-1a over input_text
seed 7 twice -> byte-identical
62171 tasks/s over the WebSocket
preset knobs exact match chance answers/s
1 state 5-7, ops 20-30 1500/1500 252/1500 55313
2 state 8-12, ops 25-40 1500/1500 165/1500 52369
3 state 13-20, ops 40-60 1500/1500 97/1500 44577
3000 tasks · 9000 steps · 0.2 s · 46154 steps/s
answered 4500/4500 · chance 514/4500 · two implementations agreed on every query
demo lines before the table, because they are the whole reason
this wing exists. A thousand resets produced a thousand distinct
input_text values with zero repeats — and the line names its own
judge, distinct by 64-bit FNV-1a over input_text, because a hash
equality is the check that actually ran. Reset the same seed twice and the two
texts are byte-identical, so the distribution is a faucet and not a slot
machine. The episode is three steps long and the
hand-driven exchange above is why: one arrangement, three questions, a step
each — query_index counts 1, 2, 3 while STATE and
OPS hold the same bytes and only the QUERY tail moves
— then a fourth step past the end, frozen, which is the after-done rule plate
10 explains. Each answer is graded on its own step, so a task pays 0 to 3 and
the sweep's columns count answers: a programmed replay took all
4500, and a guess of the right shape — a uniform element or a
uniform index, to match whichever kind of question was asked — took 514.
What this task resists is answer reuse. No two of the three queries
name the same place in the final arrangement, so copying the previous answer
scores 0.0% here. That zero is bought, not free: the guesser that
inverts it, crossing the previous answers off and picking uniformly, is paid
+0.9 points at difficulty 1 for it, and the battery prints that row
(uniform-minus-previous) beside the zero rather than resting on the
zero alone. It resists skimming the operations too. A reader that sums the
rotations and ignores SWAP and REVERSE — no
list tracking at all — is inside half a point of chance at every level: 16.4%,
9.8% and 6.3% against 17.0%, 10.2% and 6.2%. A short run of
permutations does not mix, which is why the presets draw 20-30, 25-40 and
40-60 operations.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"answer": "B"} — a string. There is no
illegal answer, only a wrong one: a missing field reads as the empty
string and scores zero. A wrong-typed one never gets that far —
{"answer": 5} is a VALIDATION_ERROR from the framework,
because a string field takes no coercion, and the environment never sees the
step |
| reset | {"seed": N, "difficulty": 1|2|3} — the
difficulty moves state_size and num_ops; anything
that is not 1, 2 or 3 means 1 |
| observation | input_text, difficulty,
state_size, num_ops, query_index — which
of the three questions you are holding — and answer and
final_state, both empty until done, because a reveal
after the first question would answer the other two |
| reward | integer 1 on exact match after whitespace is collapsed, 0 otherwise, per step. No partial credit inside a question and no shaping; an episode pays 0 to 3 |
| done | the third answer. Three steps need the WebSocket: POST /step builds a fresh environment per call and would answer question one every time |
Rule induction: the filter is programmed; only the bet is learned
Rule induction is naming a hidden transformation from worked examples of it. Here the hidden rule is drawn from a fixed inventory — reverse, rotate, drop, interleave, sort, mirror, a Caesar shift, and composites of those — applied to a few short sequences of letters, which you are shown before being handed one more sequence to transform. The shots do not always pin the rule down. That gap is the task.
$ build/rel/bin/inducto --host 127.0.0.1 --port 8822 &
$ build/rel/bin/oe_probe http://127.0.0.1:8822 --episode --steps 3 --action '{"answer":"A B C"}' --past-done | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ex1: [H A D A J A B G D B H] -> [H H H A A D D A A J J A A B B G G D D B B H] | Ex2: [F C C E J B I A B J A] -> [A F F C C C C E E J J B B I I A A B B J J A] | Ex3: [I I B F E J H F] -> [F I I I I B B F F E E J J H H F] | Ex4: [H B B A C A F E J] -> [J H H B B B B A A C C A A F F E E J] | Ex5: [F F E F G J B E D J A] -> [A F F F F E E F F G G J J B B E E D D J J A] | Query: [B J G J] ->","difficulty":1,"num_examples":5,"answer":"","rule":"","rule_complexity":0},"reward":null,"done":false}}
>> {"type":"step","data":{"answer":"A B C"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ex1: [H A D A J A B G D B H] -> [H H H A A D D A A J J A A B B G G D D B B H] | Ex2: [F C C E J B I A B J A] -> [A F F C C C C E E J J B B I I A A B B J J A] | Ex3: [I I B F E J H F] -> [F I I I I B B F F E E J J H H F] | Ex4: [H B B A C A F E J] -> [J H H B B B B A A C C A A F F E E J] | Ex5: [F F E F G J B E D J A] -> [A F F F F E E F F G G J J B B E E D D J J A] | Query: [B J G J] ->","difficulty":1,"num_examples":5,"answer":"J B B J J G G J","rule":"DUPLICATE_EACH_THEN_ROTATE_RIGHT_1","rule_complexity":1},"reward":0,"done":true}}
>> {"type":"step","data":{"answer":"A B C"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ex1: [H A D A J A B G D B H] -> [H H H A A D D A A J J A A B B G G D D B B H] | Ex2: [F C C E J B I A B J A] -> [A F F C C C C E E J J B B I I A A B B J J A] | Ex3: [I I B F E J H F] -> [F I I I I B B F F E E J J H H F] | Ex4: [H B B A C A F E J] -> [J H H B B B B A A C C A A F F E E J] | Ex5: [F F E F G J B E D J A] -> [A F F F F E E F F G G J J B B E E D D J J A] | Query: [B J G J] ->","difficulty":1,"num_examples":5,"answer":"J B B J J G G J","rule":"DUPLICATE_EACH_THEN_ROTATE_RIGHT_1","rule_complexity":1},"reward":null,"done":true}}
>> {"type":"step","data":{"answer":"A B C"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"Ex1: [H A D A J A B G D B H] -> [H H H A A D D A A J J A A B B G G D D B B H] | Ex2: [F C C E J B I A B J A] -> [A F F C C C C E E J J B B I I A A B B J J A] | Ex3: [I I B F E J H F] -> [F I I I I B B F F E E J J H H F] | Ex4: [H B B A C A F E J] -> [J H H B B B B A A C C A A F F E E J] | Ex5: [F F E F G J B E D J A] -> [A F F F F E E F F G G J J B B E E D D J J A] | Query: [B J G J] ->","difficulty":1,"num_examples":5,"answer":"J B B J J G G J","rule":"DUPLICATE_EACH_THEN_ROTATE_RIGHT_1","rule_complexity":1},"reward":null,"done":true}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":1,"num_examples":5}}
>> {"type":"close"}
$ build/rel/bin/synthlab inducto http://127.0.0.1:8822 --seed 7
synthlab inducto — programmed consistency filter, sample-average prior over 52 rules
learns which consistent rule to bet on when the shots leave a tie, from reward alone
does not see the rule name before done, or meet a rule outside the generator's 52
server http://127.0.0.1:8822 — InductoEnvironment
run 60000 tasks · seed 7 · difficulty 3 · eps 1.00->0.05 · 52 sample averages
demo 1000 seeds -> 1000 distinct tasks · 0 repeats · distinct by 64-bit FNV-1a over input_text
seed 7 twice -> byte-identical
71847 tasks/s over the WebSocket
ties exact match curve (bars span 41.27 to 64.02)
1- 189 41.27
190- 378 48.15 ############
379- 567 48.68 #############
568- 756 58.20 ##############################
757- 945 57.14 ############################
946-1134 51.32 ##################
1135-1323 58.20 ##############################
1324-1512 58.20 ##############################
1513-1701 62.43 #####################################
1702-1890 64.02 ########################################
60000 tasks · 60000 steps · 1.5 s · 39520 steps/s · answered 59145/60000
1890 ties · 3.1% of tasks (consistent rules disagreed on the query) · first bucket 41.3% · last 64.0% · complexity-1 rules priced at 0.19 over 286 bets
tie oracles · uniform draw 45.3% · plurality 46.1% · draw-weighted 68.0%, the Bayes ceiling (difficulty 3 mixes a 12-rule and a 20-rule pool half and half, so a complexity-3 survivor is 5/3 the bet a complexity-2 one is)
arcade/CHARTER.md and then declined
in writing, because it would have
deleted the only decision this plate ever plots.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"answer": "A B C"} — the transformed
sequence, space-separated. Every string is a legal answer; only the target is
right |
| reset | {"seed": N, "difficulty": 1|2|3} — the
difficulty moves the shot count and which complexity pools the rule is drawn
from |
| observation | input_text, difficulty,
num_examples, and answer, rule and
rule_complexity — the name arrives only after
done |
| reward | integer 1 on exact match, 0 otherwise |
| done | the answer step |
Circuit evaluation: two implementations of one specification, agreeing
Circuit evaluation is running a printed circuit in your head, gate by gate.
A small boolean circuit is printed as text: input bits, then gates —
AND, OR, XOR, NAND,
NOR, NOT — each reading inputs or earlier gates, and
one gate is queried. The answer is one bit, which makes a coin worth exactly
half and makes the interesting number the one the coin does not get.
$ build/rel/bin/logic --host 127.0.0.1 --port 8823 &
$ build/rel/bin/oe_probe http://127.0.0.1:8823 --episode --steps 3 --action '{"answer":"1"}' --past-done | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"INPUTS x0=0 x1=1 x2=0 x3=1 || GATES g0=XOR(x1,x0) | g1=XOR(x0,x2) | g2=AND(x3,g0) | g3=OR(g0,x3) | g4=OR(g0,g1) | g5=XOR(g2,g4) | g6=AND(g2,g4) || QUERY g6 ->","difficulty":1,"num_inputs":4,"depth":2,"width":3,"num_gates":7,"cone_gates":5,"xor_count":3,"answer":""},"reward":null,"done":false}}
>> {"type":"step","data":{"answer":"1"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"INPUTS x0=0 x1=1 x2=0 x3=1 || GATES g0=XOR(x1,x0) | g1=XOR(x0,x2) | g2=AND(x3,g0) | g3=OR(g0,x3) | g4=OR(g0,g1) | g5=XOR(g2,g4) | g6=AND(g2,g4) || QUERY g6 ->","difficulty":1,"num_inputs":4,"depth":2,"width":3,"num_gates":7,"cone_gates":5,"xor_count":3,"answer":"1"},"reward":1,"done":true}}
>> {"type":"step","data":{"answer":"1"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"INPUTS x0=0 x1=1 x2=0 x3=1 || GATES g0=XOR(x1,x0) | g1=XOR(x0,x2) | g2=AND(x3,g0) | g3=OR(g0,x3) | g4=OR(g0,g1) | g5=XOR(g2,g4) | g6=AND(g2,g4) || QUERY g6 ->","difficulty":1,"num_inputs":4,"depth":2,"width":3,"num_gates":7,"cone_gates":5,"xor_count":3,"answer":"1"},"reward":null,"done":true}}
>> {"type":"step","data":{"answer":"1"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"INPUTS x0=0 x1=1 x2=0 x3=1 || GATES g0=XOR(x1,x0) | g1=XOR(x0,x2) | g2=AND(x3,g0) | g3=OR(g0,x3) | g4=OR(g0,g1) | g5=XOR(g2,g4) | g6=AND(g2,g4) || QUERY g6 ->","difficulty":1,"num_inputs":4,"depth":2,"width":3,"num_gates":7,"cone_gates":5,"xor_count":3,"answer":"1"},"reward":null,"done":true}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":1,"num_gates":7}}
>> {"type":"close"}
$ build/rel/bin/synthlab logic http://127.0.0.1:8823 --seed 7
synthlab logic — programmed evaluation, a coin arm, cone depth and XOR density swept
learns nothing — the evaluation is programmed, gate by gate in definition order
does not dress computation as learning; the coin beside it is what chance pays
server http://127.0.0.1:8823 — LogicEnvironment
run 1500 solver episodes · 1500 chance episodes · seed 7 · 500 per preset
demo 1000 seeds -> 1000 distinct tasks · 0 repeats · distinct by 64-bit FNV-1a over input_text
seed 7 twice -> byte-identical
61898 tasks/s over the WebSocket
preset knobs exact match chance answers/s
1 inputs 3-4, cone 4-6, depth 2-3, xor .10 500/500 248/500 39258
2 inputs 4-5, cone 7-10, depth 3-5, xor .30 500/500 234/500 36891
3 inputs 5-6, cone 12-16, depth 5-8, xor .50 500/500 235/500 32624
3000 tasks · 3000 steps · 0.1 s · 30155 steps/s
answered 1500/1500 · chance 717/1500 · two implementations agreed on every task
input_text alone, having never seen the environment's code. They
agreed on all fifteen hundred, and the coin beside them took 717 — which is
what a one-bit answer pays and is the reason this plate has a sweep and not a
curve. That agreement is the plate's whole point: a reward is only worth
training against when two implementations of the specification produce the same
number, and here the number is 1500/1500 with a coin at half.
What this task resists is answering from the queried operator's bias.
A gate's truth table is lopsided —
AND answers 1 on one row in four, OR on three — so a
circuit that drew its inputs first would hand the queried gate its
operator's bias. Here the answer bit is a fair coin flipped before the
circuit exists, and the inputs are drawn last from the assignments that produce
it. So the reader that reads exactly one token — the operator name of
the gate the QUERY names — is paid 51.4%,
49.9% and
48.7%, a coin's wage. Every one of the seven other readers
in the battery
is inside two points of even too. And a gate no input can move, which no coin
can be forced on, occurs on 0.0% of tasks.
Padding is bounded as well. A circuit can carry gates the queried gate never
reads, so this generator draws
the queried gate's cone — the gates the answer actually depends on — FIRST and
pads outward to a cap, and the printed circuit
is at least 70% cone at every difficulty.
Difficulty buys depth and XOR density rather than decoration: the
cone goes from 5.027 gates to 13.991 while the print goes from 6.549 to 18.973.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"answer": "1"} — the queried gate's output
bit, as a string |
| reset | {"seed": N, "difficulty": 1|2|3} — the
difficulty moves the input count, the cone size, its depth and the XOR
share |
| observation | input_text, difficulty,
num_inputs, depth, width,
num_gates, cone_gates — how many of those gates the
answer depends on — xor_count, and answer, empty
until done |
| reward | integer 1 on exact match, 0 otherwise |
| done | the answer step |
Source separation: half the context is there to be ignored
Source separation is following one signal while a second is woven through it. Two graphs are interleaved into one list of edges. One uses Greek letters, the other colour names, and they never share a node; the query walks a stated number of hops through one of them. Every edge belonging to the other stream is there to be suppressed, and at the hardest preset the interleaving stops being a neat alternation and comes in shuffled or in runs.
$ build/rel/bin/filtro --host 127.0.0.1 --port 8824 &
$ build/rel/bin/oe_probe http://127.0.0.1:8824 --episode --steps 3 --action '{"answer":"alpha"}' --past-done | grep '^[<>][<>] {'
>> {"type":"reset","data":{}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"EDGES A:beta->theta | B:black->pink | A:eta->zeta | B:blue->purple | A:alpha->gamma | B:pink->yellow | A:gamma->epsilon | B:purple->orange | A:epsilon->beta | B:orange->black | A:zeta->delta | B:red->blue | A:delta->alpha | B:yellow->green | A:theta->eta | B:green->red || QUERY A:zeta hops=2 ->","difficulty":1,"num_nodes":8,"num_hops":2,"pattern":"alternating","answer":"","distractor":""},"reward":null,"done":false}}
>> {"type":"step","data":{"answer":"alpha"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"EDGES A:beta->theta | B:black->pink | A:eta->zeta | B:blue->purple | A:alpha->gamma | B:pink->yellow | A:gamma->epsilon | B:purple->orange | A:epsilon->beta | B:orange->black | A:zeta->delta | B:red->blue | A:delta->alpha | B:yellow->green | A:theta->eta | B:green->red || QUERY A:zeta hops=2 ->","difficulty":1,"num_nodes":8,"num_hops":2,"pattern":"alternating","answer":"alpha","distractor":"black"},"reward":1,"done":true}}
>> {"type":"step","data":{"answer":"alpha"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"EDGES A:beta->theta | B:black->pink | A:eta->zeta | B:blue->purple | A:alpha->gamma | B:pink->yellow | A:gamma->epsilon | B:purple->orange | A:epsilon->beta | B:orange->black | A:zeta->delta | B:red->blue | A:delta->alpha | B:yellow->green | A:theta->eta | B:green->red || QUERY A:zeta hops=2 ->","difficulty":1,"num_nodes":8,"num_hops":2,"pattern":"alternating","answer":"alpha","distractor":"black"},"reward":null,"done":true}}
>> {"type":"step","data":{"answer":"alpha"}}
<< {"type":"observation","data":{"observation":{"metadata":{},"input_text":"EDGES A:beta->theta | B:black->pink | A:eta->zeta | B:blue->purple | A:alpha->gamma | B:pink->yellow | A:gamma->epsilon | B:purple->orange | A:epsilon->beta | B:orange->black | A:zeta->delta | B:red->blue | A:delta->alpha | B:yellow->green | A:theta->eta | B:green->red || QUERY A:zeta hops=2 ->","difficulty":1,"num_nodes":8,"num_hops":2,"pattern":"alternating","answer":"alpha","distractor":"black"},"reward":null,"done":true}}
>> {"type":"state"}
<< {"type":"state","data":{"episode_id":null,"step_count":1,"num_hops":2}}
>> {"type":"close"}
$ build/rel/bin/synthlab filtro http://127.0.0.1:8824 --seed 7
synthlab filtro — programmed walk of the queried stream, a uniform-node arm, hops 1..n-1 swept
learns nothing — the walk is programmed and reads only A: edges
does not dress computation as learning; suppressing B: costs a parser nothing, and says so
server http://127.0.0.1:8824 — FiltroEnvironment
run 1500 solver episodes · 1500 chance episodes · seed 7 · 500 per preset
demo 1000 seeds -> 1000 distinct tasks · 0 repeats · distinct by 64-bit FNV-1a over input_text
seed 7 twice -> byte-identical
66308 tasks/s over the WebSocket
preset knobs exact match chance answers/s
1 nodes 5-8, hops 1..n-1, alternating 500/500 92/500 39583
2 nodes 8-12, hops 1..n-1, alternating/random 500/500 44/500 37826
3 nodes 10-15, hops 1..n-1, random/blocked 500/500 43/500 35716
3000 tasks · 3000 steps · 0.1 s · 31622 steps/s
answered 1500/1500 · chance 179/1500 · two implementations agreed on every task
beta, then eta, then alpha — so
hops=k never collapses into "read k edges further down the list".
The battery's positional reader, which never separates the streams at all,
scores 16.8% at the hardest preset, against
18.4% for a guesser that reads nothing but the hop count. And the A-list
chains end-to-start on only 9.4% of tasks at that preset, which is
what a shuffled list does by arithmetic. That 18.4% is the
looser of the two baselines the battery prints, and deliberately so: true chance
here is 8.9% — a walk of 1..n-1 on a ring of n can never come home, so a guesser
who only crosses off the start pays 1/(n-1), the row called
uniform-minus-start. The positional reader is graded against the
looser one because that baseline also collects every hops=1 row for
free, and taking those is the only thing the reader does that a correct solver
does not. Against the honest floor it is 16.8% to 8.9%, and against the fair
comparison 16.8% to 18.4%.
And hops
is drawn uniform on 1..n-1 rather than from a preset window, so no fixed offset
and no floor — "never the immediate successor" would be one —
beats a uniform draw over the nodes the walk can reach.
The sweep stays a sweep: a parser reads the A: edges, never looks
at B:, and walks, taking every one of fifteen hundred against 179
for the uniform guess. Suppression is a difficulty for attention, not
for a parser, and nothing here pretends otherwise.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.| the contract | shape |
|---|---|
| action | {"answer": "alpha"} — the node the walk
ends on |
| reset | {"seed": N, "difficulty": 1|2|3} — the
difficulty moves the node count and the interleaving pattern; the hop count is
drawn uniform on 1..n-1 at every difficulty |
| observation | input_text, difficulty,
num_nodes, num_hops, pattern, and
answer and distractor — the walk through the other
stream from the mirrored start, the answer the noise was selling — both empty
until done |
| reward | integer 1 on exact match, 0 otherwise |
| done | the answer step |
Eight kinds of mathematical work
The math room turns a problem into ordinary text and asks for one bounded answer. It spans practical quantities, symbolic structure, change, space, finite systems, uncertainty, observed data, and proof. Those are different objects to reason about, not eight skins over the same arithmetic.
Every task states its conventions and return grammar inside
input_text. The environment grades with exact arithmetic or a
small certificate checker, then reveals the answer only after the step has
ended. The independent mathlab client solves from the prompt,
tries a wrong answer, repeats the seed, checks oversized input, and makes sure
a finished episode stays frozen. The full design, including the curriculum
table and the limits of this first release, is in
the math-room guide.
| shared contract | shape |
|---|---|
| reset | {"seed": N, "difficulty": 1|2|3} —
later presets mix in new reasoning structures without removing the earlier
ones |
| observation | input_text, a task-family label,
small public size fields, and an empty answer |
| action | {"answer": "..."} — the exact grammar is
the final line of the prompt |
| reward | integer 1 when the bounded grader accepts the answer, integer 0 otherwise; no model judges another model |
Keep the unit attached to the number
measure begins with one rate, then adds proportional scaling
and clock conversion with an optional pause. Units are part of the canonical
answer, so a bare number cannot earn credit by accident.
$ build/rel/bin/mathlab measure http://127.0.0.1:8831 --seed 7
mathlab measure — quantities, units, and practical models
server http://127.0.0.1:8831 — MeasureEnvironment
run 300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset tasks exact wrong accepted repeats
1 100 100 0 0
2 100 100 0 0
3 100 100 0 1
300 tasks · 300/300 exact · 0 wrong answers accepted · 1 repeated prompts within a presetSolve for the whole answer, not one convenient root
algebra moves from a linear equation to a unique system and
then an integer-root quadratic. Its quadratic answer is a complete ordered
set, which closes the common shortcut of returning only one valid root.
$ build/rel/bin/mathlab algebra http://127.0.0.1:8832 --seed 7
mathlab algebra — expressions and complete solution sets
server http://127.0.0.1:8832 — AlgebraEnvironment
run 300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset tasks exact wrong accepted repeats
1 100 100 0 0
2 100 100 0 0
3 100 100 0 0
300 tasks · 300/300 exact · 0 wrong answers accepted · 0 repeated prompts within a presetRead a function as a rule for variation
change asks for a derivative at a point, a definite integral,
or a global maximum on a closed interval. The harder presets change what must
be reasoned about; they do not merely widen the coefficients.
$ build/rel/bin/mathlab change http://127.0.0.1:8833 --seed 7
mathlab change — functions, rates, accumulation, and extrema
server http://127.0.0.1:8833 — ChangeEnvironment
run 300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset tasks exact wrong accepted repeats
1 100 100 0 0
2 100 100 0 0
3 100 100 0 0
300 tasks · 300/300 exact · 0 wrong answers accepted · 0 repeated prompts within a presetLet coordinates replace an unreliable sketch
space begins with squared distance, then applies a matrix to a
vector and finally tracks area through a linear map. Exact integer answers
keep the geometry legible and avoid a hidden floating-point tolerance.
$ build/rel/bin/mathlab space http://127.0.0.1:8834 --seed 7
mathlab space — coordinates and linear transformations
server http://127.0.0.1:8834 — SpaceEnvironment
run 300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset tasks exact wrong accepted repeats
1 100 100 0 0
2 100 100 0 0
3 100 100 0 0
300 tasks · 300/300 exact · 0 wrong answers accepted · 0 repeated prompts within a presetAdvance one exact state at a time
discrete covers recurrences, shortest paths, and constrained
unordered selection. The changing tokens alter transitions, graph edges, or
constraints, so diversity changes the decision rather than decorating it.
$ build/rel/bin/mathlab discrete http://127.0.0.1:8835 --seed 7
mathlab discrete — recurrences, finite structures, and counts
server http://127.0.0.1:8835 — DiscreteEnvironment
run 300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset tasks exact wrong accepted repeats
1 100 100 0 0
2 100 100 0 0
3 100 100 0 0
300 tasks · 300/300 exact · 0 wrong answers accepted · 0 repeated prompts within a presetBegin with the sample space the process uses
chance moves from one draw to dependent draws and
conditioning. Replacement and ordering are stated in the prompt, and every
probability is returned as a reduced fraction rather than an approximate
decimal.
$ build/rel/bin/mathlab chance http://127.0.0.1:8836 --seed 7
mathlab chance — finite probability and conditioning
server http://127.0.0.1:8836 — ChanceEnvironment
run 300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset tasks exact wrong accepted repeats
1 100 100 0 0
2 100 100 0 1
3 100 100 0 0
300 tasks · 300/300 exact · 0 wrong answers accepted · 1 repeated prompts within a presetAggregate the rows before comparing the groups
evidence presents tables that call for aggregate rates,
weighted means, or medians under a stated missing-value rule. It stays
separate from probability because these rows have already happened; the job
is to summarize observed data.
$ build/rel/bin/mathlab evidence http://127.0.0.1:8837 --seed 7
mathlab evidence — tables, aggregation, and statistics
server http://127.0.0.1:8837 — EvidenceEnvironment
run 300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset tasks exact wrong accepted repeats
1 100 100 0 0
2 100 100 0 0
3 100 100 0 0
300 tasks · 300/300 exact · 0 wrong answers accepted · 0 repeated prompts within a presetProve the claim or exhibit a countermodel
reason does not compare one hidden answer string. It accepts a
valid sequence of applicable rules or a complete assignment that satisfies
the facts and rules while making the claim false. Both are checked for
meaning, so alternate valid certificates can earn the same reward.
$ build/rel/bin/mathlab reason http://127.0.0.1:8838 --seed 7
mathlab reason — checked proof or countermodel certificates
server http://127.0.0.1:8838 — ReasonEnvironment
run 300 tasks · seed 7 · correct, wrong, deterministic, bounded, and frozen-terminal checks
preflight integer answer rejected · 511-byte answer safely scored 0
preset tasks exact wrong accepted repeats
1 100 100 0 2
2 100 100 0 0
3 100 100 0 0
300 tasks · 300/300 exact · 0 wrong answers accepted · 2 repeated prompts within a presetJudges that are not ours
A page that grades its own games is a page that agrees with itself. So the
recording ends with three judges: the upstream project's own conformance
suite, a battery of cheap readers with a control population beside it, and the
same game timed on two servers — theirs and ours.
The first is openenv validate, the reference's own conformance
suite — the reference being huggingface/OpenEnv, the Python implementation
this repository was captured from. Six required criteria, run over the network
against three of the ten,
with nothing of ours in the loop but the server being graded. Connect four is
one of the three on purpose — if a made-up observation field like
legal_actions were going to trouble somebody else's conformance
suite, this is where it would show.
$ build/rel/bin/bandit --host 127.0.0.1 --port 8801 &
$ (cd reference/OpenEnv && PYTHONPATH=src .venv-oe/bin/python -m openenv.cli validate --url http://127.0.0.1:8801) | sed -n '/"summary"/,/}/p'
"summary": {
"passed_count": 6,
"total_count": 6,
"failed_criteria": [],
"required_passed_count": 6,
"required_total_count": 6
},
$ build/rel/bin/snake --host 127.0.0.1 --port 8803 &
$ (cd reference/OpenEnv && PYTHONPATH=src .venv-oe/bin/python -m openenv.cli validate --url http://127.0.0.1:8803) | sed -n '/"summary"/,/}/p'
"summary": {
"passed_count": 6,
"total_count": 6,
"failed_criteria": [],
"required_passed_count": 6,
"required_total_count": 6
},
$ build/rel/bin/connect4 --host 127.0.0.1 --port 8809 &
$ (cd reference/OpenEnv && PYTHONPATH=src .venv-oe/bin/python -m openenv.cli validate --url http://127.0.0.1:8809) | sed -n '/"summary"/,/}/p'
"summary": {
"passed_count": 6,
"total_count": 6,
"failed_criteria": [],
"required_passed_count": 6,
"required_total_count": 6
},envs/counter.c in make interop,
pointed at a slot machine, at snake, and at the one with an opponent living
inside it. Being fun is not a licence to be non-conforming: these get
GET /schema, request validation, the sixteen routes and the
session machinery from the same library a stranger installs, so passing is
what should happen — and until this ran, nobody had checked.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.The second judge is the battery the synth-lab captions have been quoting.
A task that a fixed-position
reader answers without doing the work is a task in the shape of a task. So a
battery of deliberately stupid solvers runs over 2,000 fresh tasks per family
per difficulty. The same solvers run over a second population the
header below calls real rows: a frozen sample of rows in the same
grammar, a hundred per family, held as the control. A fixed file is where a
cheap reader gets paid, so a claim of resistance needs a population that does
not resist beside it.
$ python3 interop/exploit_battery.py --tasks 2000 --grade
exploit battery — cheap heuristics vs 2000 of ours per family per difficulty, vs 100 real rows per family
...
heuristic ours d1 ours d2 ours d3 real d1 real d2 real d3 real ALL
...
positional-reader 34.0% 23.1% 16.8% 100.0% 100.0% 100.0% 100.0%
...
uniform-minus-start 19.1% 11.4% 8.9% 12.2% 12.8% 12.2% 12.4%
...
one-hop-free uniform 37.4% 23.3% 18.4% 29.6% 31.6% 33.1% 30.9%
...
[struct] path-order A-links 19.4% 11.2% 9.4% 100.0% 100.0% 100.0% 100.0%
...
[struct] rows fully path-ordered 1.2% 0.0% 0.0% 100.0% 100.0% 100.0% 100.0%
...
final-gate operator prior 51.4% 49.9% 48.7% 62.7% 78.6% 53.8% 66.0%
...
[struct] queried gate constant 0.0% 0.0% 0.0% 22.0% 10.7% 7.7% 17.0%
...
net-rotation only 16.4% 9.8% 6.3% 13.7% 21.1% 9.1% 16.0%
...
copy-previous-answer 0.0% 0.0% 0.0% 7.8% 7.9% 4.5% 7.5%
...
[struct] inverse-redundant episodes 0.0% 0.0% 0.0% 35.3% 44.7% 45.5% 40.0%
...
[struct] length-only identifies 0.0% 0.0% 0.0% 20.0% 16.3% 12.5% 16.0%
...
oracle: this file's solver re-derived the graded answer on every task of both populations, all four families.
...
...; the recording carries four whole tables and a graded
bar list. One frame governs every pair: a fixed file rewards position, padding
and repetition, and a generator that shuffles emission, draws the answer before
the circuit, and re-mints per seed does not — the pairs across are that
sentence, measured. A positional reader that never separates
filtro's two streams takes 100.0% of the real rows and 16.8% of ours; the
control's A-lists chain end-to-start on 100.0% of rows and ours on 9.4%; a
one-token reader of the queried gate's operator takes 66.0% there and 48.7%
here, and 17.0% of the control circuits have a queried gate no input bit can
move
against 0.0% of ours; copy-the-previous-answer pays 7.5% there and 0.0% here,
and 40.0% of the control episodes ask one question twice (a position and the
element standing at it) against 0.0% of ours; summing the rotations while
ignoring SWAP and REVERSE takes 16.0% there against
12.1% chance, and 6.3% here at difficulty 3 against a 6.2% chance; output length alone
names the rule on 16.0% of the control rows and 0.0% of ours. Two baselines are
printed for filtro and the looser one is deliberate:
uniform-minus-start is the honest floor a walk of 1..n-1 leaves
(19.1 / 11.4 / 8.9), while one-hop-free uniform also collects every
hops=1 row for free (37.4 / 23.3 / 18.4) — the positional reader is
graded against the second because taking the one-hop rows is the only thing it
does that a correct solver does not. The oracle line is what licenses
the rest — the battery re-derived every graded answer of both populations from
the prompt, so it is measuring the tasks and not its own bugs — and the bars
under it are the charter's, written before the run: a named reader more than
four points above the sharpest cheap baseline is a red run, and the recorder
carries the failure out in its exit status.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.The third judge is a price tag. interop/bandit_env.py is
arcade/bandit.c ported onto the reference's own server — their
create_app, their routing, their pydantic validation, under
uvicorn — so the same trainer, at the same seed, over the same protocol, can be
pointed at two implementations of one game. Both ports are deterministic in
the seed, so both runs must print the same curve; the recorder diffs them and
prints a ratio only when they match.
$ (cd reference/OpenEnv && PYTHONPATH=src:../../interop .venv-oe/bin/python -m uvicorn bandit_env:app --host 127.0.0.1 --port 8811) &
$ build/rel/bin/train bandit http://127.0.0.1:8811 --seed 7
...
run 200 episodes · seed 7 · eps 0.10 · sample-average · a new machine each visit
...
1- 10 0.63
...
91- 100 0.83 ########################################
...
...
Both runs printed the same curve, line for line: the two servers
dealt the same ladder and paid the same coins, so what is left
between them is the cost of serving one step...., from a recording that carries both curves whole — the ten rows
here are identical to plate 1's, which is the point and is why the recorder
prints a ratio at all. That ratio compares two
implementations of one game — not two algorithms, and not two
projects. One client, one connection, one request in flight: it is latency, not
saturation, and the reference is doing general work that a single hand-written
env does not do. The direction reverses too, in
make interop: its eighth oracle runs epsilon-greedy inside
upstream's own EnvClient against arcade/bandit.c and
asserts the learner improved, so each side has now trained the other's.
— make record-arcade; site-check refuses any line
arcade/recordings.txt does not contain.Which side of the line this is
This repository has two directories of environments and they are not the
same kind of thing.
envs/ teaches the protocol: those
files are corpus-coupled, their bytes are diffed against captured recordings,
and they may not change. arcade/ is built on the
protocol and is free — it is what a stranger writes after reading the tour.
Every file under it opens with the same seven lines, and a rule in
tools/check.sh reads them back: it takes the seven out of
arcade/CHARTER.md and compares them, byte for byte, against the
head of every arcade source, so the marking cannot drift one file at a
time.
ARCADE — built ON the OpenEnv protocol, not part of the teaching core.envs/is the other side of this boundary: corpus-coupled oracles that may not change, because tests diff their bytes against recordings. Delete this directory and no corpus case, oracle verdict or generated ledger in this repository changes: the one oracle that names a file here skips without it. These are ordinary downstream environments, written with the same public API any stranger gets:<openenv/openenv.h>,oe_main, and nothing else.
That is the useful half of the marking. Every server gets the whole library —
sixteen routes, GET /schema, WebSocket sessions, request
validation, the production flag — from the same public header a consumer
installs, and they get it without being allowed to touch the evidence.
The word in that banner is proof, and it is narrow because the
experiment has been run. Move arcade/ aside and what fails is
citation and count — dead links to this directory's note, a program tally,
this page's quotations — while every corpus case, oracle verdict and
conformance figure stands. The one place the arcade appears in a suite is
make interop's eighth oracle, and it skips when the binary is
absent. The directory's own note is
arcade/README.md.
What these are for
The clients on this page are deliberately small. The game client holds ten
short algorithms; the synth client holds four exact readers; the math client
parses and independently solves eight task families. None is a network, so the
loop and the source of reward remain legible. The environments are not small
in the same way. Ten are the fast twins of material the upstream project already
teaches with —
huggingface/OpenEnv ↗
ships a snake environment, a 2048 training tutorial and a Wordle task for GRPO,
plus a blackjack example, a maze, a grid world and a connect four environment —
four are task generators with a verifiable reward, and the math room adds
eight more. That is what a
GRPO-style trainer wants and what
the boundary note means when it points
at GRPOTrainer. They
answer between 30155 and 85710 steps a second on one core, round trip, over a
real WebSocket. The ten are not those upstream environments and do not pretend
to be: the action names here are ours, so a client written for upstream's snake
gets a validation error rather than a game.
Each server holds eight sessions at once by default, so eight rollout workers can share one process, and every one of them gets its own environment instance. Nothing here needs a GPU, a container or a Python runtime. If your learner is slower than the recorded server loops — and a learner that reads a board and computes a gradient will be — then the environment is not your bottleneck, which is the only claim this page is making.
These servers are also built to be trained against by a language model, and
that is three steps rather than a project. make arcade-up starts
all fourteen and writes build/rel/arcade.json, an object from each
server's name to its base URL; make math-up does the same for the
math room in build/rel/math.json. The adapters this repository
ships read those manifests, ask each server GET /schema what its
action field is called, and take the prompt out of input_text. So
there is no table of ports to type and no prompt to write per environment.
make adapters drives those adapters against the running arcade
with no GPU and no model weights.
Three walkthroughs — TRL, prime-rl, and the Tinker API via SkyRL — are in
docs/TRAINING.md, which says which parts of
them have been run here and which have not.
Read next:
the tour, which is how these were written;
the annotated episode, which is what the transcripts
above look like byte by byte; or
arcade/README.md, the directory's
own note, which is where the ports, the seeds and both clients' flags are
written down.