Fuzzing, and the WebSocket conformance suite
Two programs point strangers' bytes at this library on purpose: the fuzzers, and the WebSocket conformance suite. This page is how each one works and what it found. They are separate targets because they answer different questions:
make fuzz-run # replay every seed through six targets, no mutation
make fuzz-run SECONDS=60 # ... and then mutate for a minute per target
make autobahn # 517 RFC 6455 conformance cases, in docker, 12.3 s
The deterministic replay — make fuzz-run with no SECONDS — is part of
make check. It has no tunable cost and needs no network, so the property
oracles — the independent checks that grade each target's output — and every
committed regression seed run on every gate. tools/check.sh builds the six
targets and replays each seed once, which adds about a second and a half here
once the targets are built. What stays out of the gate is what a contributor
would learn to skip: the mutation runs, whose cost is the SECONDS parameter,
and make autobahn, which needs a container and a network. make check stays
hermetic; with the fuzz replay and the two test legs it runs in about eight
seconds on this machine.
1. The six fuzz targets
| Target | Entry points | What it is looking for |
|---|---|---|
fuzz_json | oe_json_parse_diag, oe_json_write, oe_json_dup | escapes, surrogates, depth, 3 versus 3.0, the zero-copy claim |
fuzz_http | oe_http_head_scan, oe_http_parse, oe_http_handle | request smuggling, header caps, head/body arithmetic |
fuzz_ws | oe_ws_frame_parse, oe_ws_read | length forms, masking, fragmentation, control frames, buffer boundaries |
fuzz_session | oe_session_open, oe_session_frame, oe_ws_mcp_frame | the whole protocol, from one text frame |
fuzz_client | oe__client_decode, oe__client_ws_url | the other side of the wire: bytes a remote SERVER chose |
fuzz_tool | oe__decode_tool_action, oe__tool_find, an author's handler, oe_session_frame, oe_ws_mcp_frame, oe_http_handle | a tool NAME and an arguments object, which are the two attacker-controlled values that reach a table lookup and somebody else's code. All five doors, including the HTTP pair — fuzz_http drives the tool-less counter, so env->tools != NULL gated every tool line out of it, and until this target grew a fifth use nothing mutated a byte at do_call_tool, oe__encode_call_tool_http or oe__srv_mcp_route |
Each is one file in tests/fuzz/, each defines exactly LLVMFuzzerTestOneInput,
and each is built under ASan + UBSan + float-cast-overflow with
-fno-sanitize-recover=all. That third name is not redundant: it is inside
clang's undefined group and outside gcc's, and it is the check that found the
worst bug on this page.
A fuzz target that only waits for a segfault is worth about a tenth of one that checks invariants. Almost every memory error in this codebase would be caught by the sanitizer anyway. What the sanitizer cannot see is three things. A parser that returns a slice pointing at the right bytes with the wrong length. An encoder that emits a document we would refuse from a peer. A reader whose answer depends on how the peer chopped its writes. So every target carries oracles. The load-bearing ones:
fuzz_json: write, re-parse, write again — and require the two renderings to be identical. Never compared against the input, because a writer owes no one the peer's whitespace. Compared against itself, which is the property that fails the moment a byte survives the parser and does not survive the writer.fuzz_json: the source arena is released while theoe_json_dupcopy is still live, and the copy is then walked. That is the one escape from the memory rule, and it is only an escape if the copy shares nothing.fuzz_ws: no cut changes the answer. The same byte stream is read twice, once whole and once revealed one byte per call. The event sequence, the payload digest, the close code and every byte written back to the peer must match. This is the only oracle here that needs no specification to say what the right answer is; it needs only the two answers to agree. And it is the one that finds buffer-boundary bugs, which is the entire bug class of a frame reader.fuzz_ws: everything the reader writes back is parsed again, by us, in the peer's role. A server that emits a frame its own parser would refuse has broken the closing handshake for every peer.fuzz_session: every reply is re-parsed as JSON, must be an object, must carry one of the four envelopetypespellings, and anerrormust carry one of the seven codes.fuzz_http: the response'scontent-lengthis compared against the body that follows it. That is the same arithmetic the request side is checked for, applied to the direction where we are the one making the claim.fuzz_client: a derived WebSocket URL may not INTRODUCE a query or a fragment. Stated as a comparison with the input rather than as a rule about the output — and the first version got that wrong. See What it found, item four, where the fuzzer refuted this document's own oracle in nine seconds.fuzz_tool: an environment with no tool table cannot be made to recognise a tool action, by any document. That isGAP-22's gate: one declaration,env->tools != NULL, for both doors. It is the reasonenvs/echo.candenvs/counter.cdid not change a byte when the tool vocabulary landed. The same input is decoded twice and with a NULL output as well, because the decoder claims to be a pure function of the document and the gate.fuzz_tool: neitherecho_messagenorecho_with_lengthcan be made to FAIL. They read an argument with a fallback and put a value in the answer, which is the whole author contract section 9 of the installed header advertises. Anerroror amessagecoming back from one of them would mean a peer's bytes had reachedoe_tool_failthrough a table that never calls it. A promise made to environment authors is worth fuzzing.
What is NOT fuzzed, named rather than left to be discovered
src/client/client.c's socket half — the reads, the deadlines, the buffer growth.fuzz_clientcovers the two PURE functions a client applies to a server's bytes, which is everything in the client that decides anything. The socket half needs a peer and a clock, so a target for it would have to be a mutating server rather than a byte-span function. That is the honest shape of the remaining gap, and it is roughly 900 lines.src/server/serve.candconn.cunder hostile ACCEPT-time conditions. Both are exercised: the state machine byte-for-byte bytests/t_conn.cwith no socket, and a whole running server over real sockets bytests/t_conform.c(throughoe_server_fds+oe_server_tick) andtests/t_client.cpart 6 (throughoe_server_poll). What no target generates is the accept path's own failure modes — EMFILE, ECONNABORTED, a peer that dies mid-handshake. The livelocks fixed in this phase were found by a hand-written network fuzzer and anLD_PRELOADerrno shim, neither of which is in the tree.- Anything needing two processes.
make autobahnis the exception and it is a conformance suite rather than a fuzzer.
Invariants use OE_FZ_CHECK, never <assert.h>: assert compiles to nothing
under NDEBUG, and a fuzzing run that silently checks nothing is worse than no
fuzzing run because it produces a green transcript.
2. Two engines, and the difference is not cosmetic
tools/fuzz.sh decides by compiling a probe, not by looking for a binary:
a clang installed without its runtime library fails at the link, which is a
more confusing failure than not having clang at all.
| Engine | When | What it is |
|---|---|---|
| libFuzzer | clang -fsanitize=fuzzer links | coverage-guided. It watches which branches an input reached and keeps the ones that reach new branches. The real thing. |
| standalone | otherwise | tests/fuzz/standalone_main.c: corpus replay, then a black-box mutator. No coverage feedback at all. |
The standalone engine is strictly weaker and this document is not going to blur
that. gcc does not ship libFuzzer, and gcc's -fsanitize-coverage= offers only
trace-pc, whose callback identifies an edge by a return address. Using it
would put a compiler builtin and a second #ifdef-bearing file into a project
whose portability claim is every #ifdef is in src/wire/net.c. The trade
taken here is a weaker engine and an unbroken claim.
What makes the weaker engine useful anyway is the seed corpus and the
per-target dictionary. A blind byte mutator will not assemble
{"type":"reset" or the frame header 88 00 before the heat death of the
universe; oe_fz_dict hands it the tokens the grammar is made of. The
dictionary is a table of oe_str and not of C strings for exactly one reason:
half the interesting tokens in RFC 6455 contain a zero byte.
A crash is reproducible. The loop is a deterministic function of
(seed, corpus), the seed is printed at startup, and the input about to be
executed is written to --dump PATH first. So when the process dies, that
file is the input that killed it.
The input ceiling is chosen against the caps, not left to the engine, and
that is a correction rather than a flourish. tools/fuzz.sh passes
--max 786432 to the standalone engine and -max_len=786432 to libFuzzer.
Their defaults are 65,536 and 4,096 bytes; the smallest document that exhausts
OE_CONN_MSG_MAX is about 68 KB, and OE_WS_MAX_MESSAGE is 512 KB. So until
this was set, no execution in any campaign had ever reached an
arena-exhaustion path. The harnesses were right — fuzz_session.c,
fuzz_ws.c, fuzz_http.c and fuzz_client.c all build their arenas at the
real budgets — and only the input size was wrong. A fuzzer whose inputs cannot
reach a cap is not fuzzing that cap. 786,432 clears OE_WS_MAX_MESSAGE,
OE_HTTP_MAX_BODY and every parse budget with room, and stays under
standalone_main.c's own 1 MiB INPUT_MAX. Larger inputs cost executions per
second; §6 records what the rate actually is.
3. The corpus: derived, plus the seeds in the index
build/fuzz/corpus/<target>/ derived, build output, in .gitignore
tests/fuzz/corpus/<target>/ hand-written and regression, in the index
tests/fuzz/mkseeds.c reads tests/fixtures/reference-corpus.json — with this
project's own JSON parser, which is the parser it is seeding — and
writes 523 files:
- every recorded HTTP request;
- every WebSocket handshake;
- every request body and frame payload, in either direction;
- those payloads wrapped in real RFC 6455 frames with a fixed mask key;
- every server RESPONSE frame again as a
clientseed, because that target reads the direction the others write; - and every request recorded against one of the two servers that HAVE a tool
surface, again as a
toolseed.
Every byte of it came off a real Python server, so mutating it lands in the neighbourhood of valid rather than in the space of noise.
The tool selection is by PORT and not by content, which is worth one
sentence because the obvious rule is wrong. "This document mentions
tools/call" would seed the target with requests sent to interop/probe_env.py
— an environment with no tool table — and every line of tool code is behind
env->tools != NULL. A seed captured from a server without a tool surface puts
the mutator outside that gate rather than inside it. The port is a fact the
corpus records about each case, and it is the same fact tests/t_replay.c picks
an environment with.
tests/fuzz/corpus/ is the other half: 108 files a recording cannot contain,
because no conforming client sends them — 19 JSON, 18 HTTP, 25 WebSocket, 20
session, 8 client, 18 tool. Nesting at the depth cap and one past it; a lone
surrogate;
overlong UTF-8; Content-Length twice; Content-Length and Transfer-Encoding
together; a frame with the reserved bits set; a UTF-8 character split across a
fragment boundary; a control frame one byte over the limit. Ten of them are
named regress-* and are the inputs fuzzing has already produced; each is cited
from the comment beside the code it broke.
Under libFuzzer the generated directory is also the engine's WRITABLE corpus.
After a coverage-guided run it holds what fuzzing has found rather than only
what mkseeds put there — 2,429 files after the first campaign. It is build
output; only tests/fuzz/corpus/ is in the index.
And that word is counted rather than asserted. tools/fuzz.sh prints the
seed count as N committed only when git ls-files tests/fuzz/corpus | wc -l
equals what is on disk, and as N on disk, of which M committed otherwise. That
line used to say "82 committed" while git ls-files returned 0. The two
regression inputs for the two real bugs below — the headline result of this whole
page — were in no commit, and no cloner received them. A tool that asserts its own
claim can be wrong forever; one that measures it cannot.
4. What it found
Two defects in the library, one defect in this tooling, and three places where the code was right and a reasonable reading of it was not. No memory error: no crash, no leak, no ASan report, in any run. The bugs fuzzing found here were all of the other kind — arithmetic at a boundary, and undefined behaviour that a release build silently gets away with.
1. (int64_t) d on an out-of-range double — reachable in one request
coerce_int and coerce_bool in src/proto/validate.c — then at lines 518 and
588, which is where the sanitizer pointed — both wrote
if (d != (double) (int64_t) d) { ...it is not an integer... }
C99 6.3.1.4 makes a floating-to-integer conversion undefined — not lossy,
not implementation-defined — when the integral part is outside the target's
range. That line is the undefined conversion written twice. It is reachable
from one unauthenticated request, because the JSON grammar puts no bound on a
number's length and this parser degrades an integer too large for int64_t into
an OE_REAL:
POST /step {"action":{"amount":1e400}} → OE_REAL, +inf
POST /step {"action":{"amount":9223372036854775808}} → OE_REAL, 9.22e18
POST /mcp {"id": 666…6} (95 digits) → OE_REAL, 6.7e94
The /mcp route reaches it too, and by design: src/proto/jsonrpc.c:372 puts a
JSON-RPC id through oe__validate_coerce with OE_F_INT, which is the whole
reason "id":true echoes as 1.
On a build with the check enabled the process aborts, which turns one POST into
a remote kill. The fix is one helper, integral_real, which decides the range
on the double side, before a cast exists to be undefined. The boundary is
exactly where the obvious spelling is wrong: (double) INT64_MAX rounds up
to 2^63, so d <= (double) INT64_MAX admits precisely the one value that
overflows. -(double) INT64_MIN is exactly 2^63 and is the right bound. NaN and
both infinities fail the range test and are refused, which is the answer they
needed anyway.
The verdict a peer sees is unchanged — int_from_float, 422 — because the
refusal was always correct; it just travelled through undefined behaviour to
get there. Pinned by a_real_outside_int64_is_refused_and_never_converted in
tests/t_validate.c, which also records the fact that
-9223372036854775809 is accepted, as -9223372036854775808. The rounding
into a double happens before any range test can see it.
And the reason it had never been caught: -fsanitize=undefined does not mean
the same thing to the two compilers. float-cast-overflow is in clang's
undefined group and is not in gcc's. Every sanitizer run this project had
ever done used gcc, so the abort was invisible. The first coverage-guided run
found it 5,371,442 executions into its second target. Both the Makefile and
tools/fuzz.sh now name the check explicitly, so the two compilers check the
same set.
Both targets that reach the validator found it independently and from different
transports, two targets apart in the same campaign: fuzz_http on POST /mcp
with a 95-digit id, and fuzz_session on
{"type":"mcp",…,"id":8888…8881,…} after another 529,044 executions. Both
artifacts are in tests/fuzz/corpus/ verbatim —
corpus/http/regress-mcp-id-overflows-int64.raw and
corpus/session/regress-mcp-id-overflows-int64.txt — beside the hand-written
regress-step-amount-inf.raw, regress-step-amount-over-int64.raw,
corpus/session/regress-amount-inf.txt, regress-amount-over-int64.txt and
corpus/json/regress-huge-integer.json.
2. oe_json_dup refused a document oe_json_parse accepts
[ × 64 + 1 + ] × 64. The parser's depth cap counts containers and is
tested before one is opened, so a scalar inside the sixty-fourth costs no level
and the document is legal. dup_into tested the cap for every node,
including that scalar, and answered NULL. That falsified the sentence in
src/proto/value.c that said anything the parser accepted can be copied, and
it made the one documented escape from the memory rule fail on the one input
at the boundary.
The mutator produced it from the hand-written depth-at-cap.json seed, which
had left the innermost array empty and therefore missed the boundary by
exactly one level. the_parser_the_writer_and_the_copy_cap_at_the_same_level in
tests/t_json_parse.c now pins all three caps to one boundary, as a table,
because the failure was an off-by-one only one of the cells could see.
3. The bug in the fuzzing itself
The first libFuzzer run reported cov: 52 for a JSON parser. -fsanitize=fuzzer
was on the target and not on the library, so the only edges the engine could
see were the ones in tests/fuzz/fuzz_json.c. That is a coverage-guided fuzzer
guided by nothing, building and running and reporting success. -fsanitize=fuzzer-no-link
on every library translation unit is the fix, and the numbers moved from
cov: 52 to cov: 623 / 1344 / 284 / 933. It is written down because the
failure mode is silent and the transcript looks identical either way.
4. Three places the oracle was wrong and the code was right
Worth recording, because each is a thing a careful reader would also have got wrong:
- A malformed
HEADrequest gets a body-suppressed 400. OnHEAD /health … Conne\xbdtion: close, the0xBDis not a token byte, the parse fails — and the 400 still carriescontent-length: 24with no body, becauseoe_http_res_writeis handed the method the parser had already read. RFC 7231 §4.3.2 requires exactly that of every response to a HEAD.corpus/http/regress-head-bad-token.raw. - The writer may emit a document the parser refuses, by eight levels, by
design. A
VALIDATION_ERRORechoes the offending value inside four levels of envelope, soOE_JSON_ECHO_HEADROOMgives the writer more depth than the parser has — and an action at the parser's own cap therefore produces a reply past it. The alternative is that the deepest legal document costs the peer the answer and, on the WebSocket, the session. Both halves of that trade are argued at the#define; what the fuzzer added is the consequence seen from outside, which nothing had written down.corpus/session/deep-action.txt. - Names. The oracle listed
UNKNOWN_MESSAGE_TYPE(the wire spelling isUNKNOWN_TYPE) and two envelopetypevalues where there are four (observation,state,error,mcp). Trivial, and the reason the oracles are written from the wire and fromdocs/rather than by asking the implementation. An oracle that consults the code for the right answer agrees with it by construction and tests nothing. - A fourth, added with
fuzz_client, and it is the cleanest of them. The URL oracle asserted that a derived WebSocket URL contains no?, reading MUST 8.B.1's "no query parameter is ever added" as a property of the OUTPUT. Nine seconds into the first run:FUZZ INVARIANT FAILED url.p[i] != '?'. The rule is a property of the TRANSFORM. The reference is five lines of string surgery that neither parses the URL nor removes anything, sohttp://h/?x=1really does derive tows://h/?x=1/ws, upstream and here. A client that stripped the query would be diverging to tidy up after its caller. Restated as a comparison with the input, it holds. The oracle was written from a specification sentence and the specification sentence was about something else; that is the failure mode this whole section is about.
5. Autobahn
make autobahn runs the Autobahn|Testsuite fuzzingclient — the reference
test suite for RFC 6455, written by the people who wrote the reference
implementation of it — against tests/autobahn/echo.c, in the
crossbario/autobahn-testsuite container.
It really runs, and it runs against an echo server rather than against /ws.
Both halves of that sentence matter. pip install autobahntestsuite is a
Python 2 package and does not install on a modern interpreter (this machine has
Python 3.14.6 and no pip at all), so the container is the only route. Without
docker the target prints a SKIP that says so rather than pretending. And
Autobahn tests RFC 6455 by echoing: case 1.1.1 sends a text message and
asserts the same bytes come back. The /ws route answers with observations.
So pointing the suite at it would measure the protocol layer with an
instrument that knows nothing about it, and fail every payload comparison for
a reason that is not a defect. The layer Autobahn is about is src/wire/ws.c,
and tests/autobahn/echo.c is that layer with a socket and nothing in between.
The verdict is read by tests/autobahn/report.c, which parses Autobahn's own
index.json with this project's JSON parser and diffs it against
tests/autobahn/expected.txt. A case that is expected to fail and passes is
also a mismatch, and an expectation line that matches no case is a third.
A list of exceptions that only ever grows is a list nobody reads.
Recorded result, 517 cases:
OK / OK 258
expected exceptions 259
UNEXPECTED 0
stale expectations 0
The 259 exceptions are three groups, each one line of expected.txt away from
being invisible, and none of them is:
- 216 ×
UNIMPLEMENTED(12.x, 13.x) — permessage-deflate. RFC 7692 is an extension, not RFC 6455, andsrc/wire/ws.coffers no extensions at all: nothing is negotiated, so nothing can be negotiated away from underneath the frame parser. Every one of them closes cleanly. - 38 ×
FAILED(9.1.3–9.6.6) — the size caps.OE_WS_MAX_FRAMEis 256 KiB andOE_WS_MAX_MESSAGE_SERVERis 1 MiB; Autobahn's 9.x cases run to 16 MiB. Every one of the 38 is answered with close 1009 and a clean hangup — the cap working, uniformly. The cases under the caps (9.1.1, 9.1.2, 9.2.1, 9.2.2, and all of 9.7.x and 9.8.x, which send a thousand messages down one socket) echo normally. - 2 ×
NON-STRICT(6.4.3, 6.4.4) and 3 ×INFORMATIONAL— 6.4.3 and 6.4.4 send one frame in three TCP chops and would prefer the invalid UTF-8 refused at the chop.src/wire/ws.cvalidates UTF-8 incrementally across fragment boundaries, which is the boundary RFC 6455 defines; a TCP read boundary is not a protocol boundary. Autobahn accepts this: "Actual events match at least one expected."
The one defect the suite found
Case 9.2.3 — a 1 MiB binary message — originally reported remoteCloseCode: null and "did not receive message", where the identical text case 9.1.3
reported a clean 1009. The bug was in the harness, not the library, and it is
the classic one. close() on a socket whose receive queue still holds unread
bytes sends a TCP RST, and a RST discards whatever was still in flight —
including the Close frame we had just written. tests/autobahn/echo.c now
drains what is already queued before closing, bounded by OE_CONN_LINGER_MS,
which is what src/server/conn.c does for the same reason.
The shape of that drain is itself a recorded measurement. Draining with a
blocking read until EOF cost 7m43s and turned 207 clean closes into
FAILED BY CLIENT, because after a normal closing handshake the peer is
waiting for us to hang up. Reading only what is already there — stop after two
consecutive reads that would have blocked — costs two milliseconds on an idle
socket and the full run takes 12 seconds.
6. The runs these numbers came from
Both engines, against the code as it ships, 240 seconds per target on one core
of a 32-core machine. No crash, no leak, no sanitizer report, in either.
Both tables below are FOUR targets, because they were recorded before
fuzz_client existed. The five-target run is a few paragraphs down, labelled
with its own duration rather than folded in here.
libFuzzer, coverage-guided, clang 21.1.8, -O1 -g under ASan + UBSan +
float-cast-overflow:
=== fuzz_json Done 5,239,778 runs in 241s cov: 623 ft: 3594
=== fuzz_http Done 18,268,567 runs in 241s cov: 1481 ft: 4520
=== fuzz_ws Done 9,149,346 runs in 241s cov: 286 ft: 1447
=== fuzz_session Done 777,125 runs in 241s cov: 1024 ft: 5546
fuzz: 4 targets, no crash
cov is edges reached in the LIBRARY, not in the target — that distinction is
finding 3 above. fuzz_session is forty times slower per execution than
fuzz_http and reaches more features than any of them, which is what running
the whole protocol per input costs and buys.
The standalone engine, gcc 16.1.1, same flags, same 240 seconds:
=== fuzz_json 11,831,296 execs, no crash (seed 1) 207 seeds, 45 tokens
=== fuzz_http 13,739,008 execs, no crash (seed 1) 97 seeds, 58 tokens
=== fuzz_ws 14,984,192 execs, no crash (seed 1) 110 seeds, 48 tokens
=== fuzz_session 10,252,288 execs, no crash (seed 1) 59 seeds, 49 tokens
fuzz: 4 targets, no crash
Read the two tables together, because they do not say what they look like they
say. The black-box engine executes MORE inputs than the guided one on three of
the four targets, and thirteen times more on fuzz_session. It is still the
weaker engine. It spends those executions re-testing the shallow refusals it
already knows how to produce, while libFuzzer spends its own budget going one
branch deeper each time. Execution count is the number a
coverage-free fuzzer can report, and it is close to meaningless on its own. The
column that matters is the one only the guided run has: cov, ft, and the
corpus it grew from 391 files to 2,429.
fuzz_client is not in either table above, because it did not exist when they
were recorded, and re-running the guided campaign for one target would have
meant re-recording both. Here is its own run instead, standalone engine,
gcc 16.1.1, same flags, 120 seconds per target — the whole set, so the fifth
row is comparable with the four beside it rather than quoted alone:
=== fuzz_json 6,001,664 execs, no crash (seed 1) 207 seeds, 45 tokens
=== fuzz_http 6,968,320 execs, no crash (seed 1) 97 seeds, 58 tokens
=== fuzz_ws 7,636,992 execs, no crash (seed 1) 110 seeds, 48 tokens
=== fuzz_session 5,082,112 execs, no crash (seed 1) 59 seeds, 49 tokens
=== fuzz_client 7,748,608 execs, no crash (seed 1) 47 seeds, 36 tokens
fuzz: 5 targets, no crash
Its only finding so far is the one in What it found item 4 above, and it was a finding about this document rather than about the library.
And fuzz_tool is not in any of the three, for the same reason one layer
later — it arrived with the tool vocabulary, MCP-STORY.md §6 task 12. Its own
run, standalone engine, gcc 16.1.1, same flags, make fuzz-run SECONDS=60,
the whole set so the sixth row is comparable rather than quoted alone:
=== fuzz_json 3,007,488 execs, no crash (seed 1) 231 seeds, 45 tokens
=== fuzz_http 3,468,288 execs, no crash (seed 1) 109 seeds, 58 tokens
=== fuzz_ws 3,579,904 execs, no crash (seed 1) 112 seeds, 48 tokens
=== fuzz_session 2,664,448 execs, no crash (seed 1) 60 seeds, 49 tokens
=== fuzz_client 3,983,360 execs, no crash (seed 1) 48 seeds, 36 tokens
=== fuzz_tool 1,827,840 execs, no crash (seed 1) 35 seeds, 36 tokens
fuzz: 6 targets, no crash
fuzz_tool's exec rate is the lowest of the six and fell by a quarter when the
HTTP doors were added, which is the expected shape: one input now drives five
entry points instead of four, and the fifth builds and tears down an
environment. Fewer executions of strictly more code is the trade that was
wanted.
No finding. That is the honest report and it is a weaker statement than it
looks. fuzz_tool has never had a coverage-guided run at all. clang is not
installed on this machine (§7), so every number above it is black-box. And this
is the target where that gap costs the most: it is the newest, the
thinnest-seeded, and the only one whose reachable code includes a function
pointer out of somebody else's table. 2.4 million executions of a black-box
mutator against a target whose whole grammar is a dictionary token away is not
the same evidence as a coverage-guided campaign. The two attacker-controlled
values it exists for —
a tool NAME reaching a repr and a length cap, and an arguments object
reaching an environment author's handler — are exactly where a finding would be
most interesting. The seed count is the smallest of the six because the corpus
only holds tool traffic from two of its six servers. Eighteen of the
thirty-five are hand-written for the branches no recording reaches — a higher
proportion than any other target, and it says where the coverage actually
comes from. Five of those eighteen were added with the HTTP doors: the two
{"action":{…}} step bodies, the extra-key shape the reference refuses,
an INT64_MAX argument, and the undeclared-argument row from GAP-21.
And yet the weaker engine found one of the two library bugs, which is worth
saying because it cuts against the paragraph above it. oe_json_dup's
off-by-one came out of a plain gcc make fuzz-run SECONDS=420: the mutator
dropped 1e400 inside the depth-at-cap.json seed, which is one operator on
one seed and needs no coverage feedback at all. The undefined conversion needed
the guided engine and clang's sanitizer set. Two bugs, two engines, and neither
would have been found by only running the other.
7. What was actually run, on what
Every number in this document came off this machine. The two things worth saying plainly about how:
clang is not installed here. command -v clang finds nothing, so
make fuzz-run on this machine takes the standalone path — which is why that
path is written to be worth having rather than to be a stub. The libFuzzer half
was exercised by mounting the repository into the silkeh/clang container and
running the same tools/fuzz.sh inside it, unmodified:
docker run --rm --user "$(id -u):$(id -g)" -v "$PWD":/w -w /w silkeh/clang \
sh -c 'sh tools/fuzz.sh run 240'
--user is not optional and its absence was measured. Without it the
container runs as root, and the campaign works exactly as documented — 16,259
runs, no crash. But it leaves 1,662 root-owned files under build/fuzz/.
After that, make clean fails with rm: cannot remove 'build/fuzz/obj-libFuzzer/src_proto_jsonrpc.o': Permission denied, and so does
every subsequent host-side make fuzz. Repairing it needs another container
(docker run --rm -v "$PWD/build":/b alpine chown -R …). tools/autobahn.sh
two files away has always passed --user. This recipe is the one docker
invocation that is prose rather than script, which is exactly why it was the one
nobody ran end to end. build/ must stay host-owned.
That is also how cov: 52 was noticed, and therefore how the missing
-fsanitize=fuzzer-no-link was found. On a machine with clang installed, the
probe takes the coverage-guided path directly and standalone_main.c is never
built. So the second run hides clang from the probe on purpose: the fallback
nobody runs is the fallback that rots, and here nobody is running it for you.
Autobahn ran in its own container, crossbario/autobahn-testsuite
sha256:519915fb…, over --network host against a ws_echo bound to
127.0.0.1. The 517-case run takes twelve seconds wall clock and about ten
seconds of case time.
docker, gcc 16.1.1, Linux 7.1.4, 32 cores — one of which each fuzz target
uses.