The four asserts in eight lines

Every framework for building agents makes it easy to add a tool and hard to doubt one. I read twenty-five of them in their own documentation and published source: adding a tool is about four lines in nearly all of them, and not one asserts anything about the run that produced its results. A counting gate, a lock on the shared card, an artifact whose identity outlives its bytes, a judge’s reason that is checked rather than stored: none of the twenty-five ships any of them.

I did not set out to write a library. I ran a small fine-tuning harness on one consumer GPU for a fortnight and kept being wrong in the same four ways. These are those four, in the order the failures found me, with the incident behind each one and what it costs to add, measured rather than estimated. The code is public and installable; the harness it came out of is not.

1. The suite that reported fewer tests than it found

My suite printed Ran 3315 tests … OK on a tree where discovery found 3,822. No failure, no error, no skip line: a green tick over 87 percent of the suite, one command away from being pushed on. Two runs had been sharing one machine, and a runner reports what it ran and never what it was supposed to run. The case a count alone cannot see came later: a module that fails to import is replaced by one placeholder that runs like anything else, so the totals agree over a smaller tree. On a clean clone, 35 real cases became 2 placeholders while the count held.

assert_ran: the runner asserts that what ran is what was discovered, prints the denominator, names the modules that failed to import, and exits with its own code, checked before pass or fail is read.

2. The card taken twice

The lock existed and protected only the runs that remembered to take it: two full suites ran concurrently under two interpreters while nobody held it. Then a liveness check answered alive for a process id that belonged to the desktop shell, because every operating system reuses them. And within twenty minutes of the lock line gaining a process identity, a reader called a live lane a dead process, because two writers had filled the birth-time field in two notations and the comparison was a string.

hold: a lock file with a named holder, a purpose, the holder’s process id and its start time; a foreign lock refuses; a green run names the neighbours it can see; and a holder whose liveness cannot be decided is cannot decide, never free.

3. The artifact whose bytes are gone

A path is a location, not a version. A size-and-modification-time stamp is free and it is not identity: on one ordinary filesystem, two different files of the same length written back to back produced an identical stamp in 157 of 200 trials. The same shape cost me a picture in the other product: a chart persisted to a canvas file and wiped by the next write before anything saved it, and two run reports that a test rewrites on every push swept into commits by a blanket add.

witness: sha256 and size kept beside the path, so the record says read N of M and the identity outlives the bytes.

4. The judge reason nobody re-derived

A model asked to grade 72 rewrites that were not degradations kept 19, and every one of the 19 justified itself with a clause the rewrite still contained word for word. One cited a dropped condition where the only difference between the two texts was a removed full stop. Deciding the same question by rule refused 106 non-degradations at zero model calls and admitted 39 planted defects, of which the judge then caught 35.

void_unless: a verdict is void until its stated reason survives a rule re-run over the artifact, and the record distinguishes no rule was registered for this reason from the rule ran and the reason failed it.

The eight lines

from four_asserts import assert_ran, Hold, witness, void_unless

ran = assert_ran(discovered, ran, unimported=modules)     # exit_code 2 if it did not say
with Hold("gpu.hold", holder="lane-a", purpose="a suite"): ...   # waits while the holder lives
kept = witness("rows.jsonl")                              # sha256 + size, outlives the file
rulings = void_unless(verdicts)                           # void until a rule re-derives the reason

Measured on the MCP Python SDK, on a server with three tools and a runner, code lines only:

Against the survey’s own finding that adding a tool is about four lines, adding all four asserts to a server and its runner is eight. On a three-node graph in the other popular framework it was eight again.

What those lines bought on the same server in the same run: a tool that answered 120 rows now answers counted 120 rows; sha256 fd61574c… over all 1328 bytes; a grading tool returned void on a reason claiming a dropped clause where the two texts differ by a full stop, which is the exact fabrication the module was extracted for, caught through a live tool call; and a training tool waited for a planted foreign holder instead of running, and left no lock file behind, checked by planting one, because a context manager that never blocks is decoration.

What it does not do

• It does not decide whether your work is good. It decides whether the record of it says what it appears to say. A run where everything ran and everything failed is a valid record of a bad result.
• void_unless does not decide whether a verdict was right. A reason that survives its rule can still be a wrong call; it says only that the stated reason is about the text in front of it.
• hold is not a distributed lock. One file created atomically, one machine, named holders.
• assert_ran does not explain a mismatch. It is red whenever discovery and execution disagree, whatever the cause, because a guard that needs a diagnosis first fails on the next cause nobody diagnosed.
• The survey is a reading of documentation and published source, not of behaviour. Two projects’ documentation was unreachable and is recorded as not found rather than as absent.

One thing the package found in itself while being lifted out: the harness’s own classifier kept trailing punctuation on a word, so where two texts differed only by a removed full stop it reported a dropped clause for a dropped period. That is the very case the module exists for. Fixed in the package, left in the harness, and the changelog says which.

Since publishing: three real traces

Added 2026-09-06. The survey above is a reading of documentation and published source, and it is unchanged. What it could not tell me is what a working system actually records, so I read three, from three vendors, all already on this machine: two coding agents at 186,546 and 402,769 records, and the local model server that owns this machine’s graphics card, at 27,838 requests. I am not adding those up, because two of them count records and one counts requests, and a total across two units is not a number.

The identity is absent from all three, and that is the strongest thing here. Across three vendors and every record I have read, not one content hash appears as a field value. Artifacts are named by path everywhere, and a path is a location. When the bytes at that path change, nothing in any of these records notices.

The holder result got sharper, not repeated. Neither coding agent has a field for who holds a shared resource. The model server does: it is populated on all 27,838 requests, and it has exactly one distinct value, the loopback address. So the server arbitrating the card that four of my agents contend for cannot tell them apart, and could not name the holder of a long generation if you asked it. A missing field is legible. A populated field that cannot discriminate looks like an answer, which is worse.

One of my four is ordinary practice, and it has a limit I did not see at first. All three pair or count, and both agents survive the check from either direction — one of them while carrying thousands of duplicate blocks across compaction, because the id carries the meaning rather than the position. So a quarter of what I called discipline here is simply how records are built. But the third shows what that buys: its log records completions, not arrivals, so a request that never returns writes no line at all. Against thirteen hangs measured on that same server, at most six appear in its log, and the correspondence is unverified. A record of completions is a record of the successes and the fast failures.

And the fourth belongs to a harness rather than to records in general. Two of the three express failure. The other reports success on all 507 of its patch applications, has no failure event type in its schema at all, and records non-zero exit codes faithfully everywhere else. A gate that cannot fail, in somebody else’s system, and the two later traces are what stop that being a claim about traces.

Three is three, and two of them were chosen for being on the disk.

Where it is

github.com/naidx0/four-asserts — MIT, no dependencies, Python 3.10 and up. Its own suite runs 38 of 38 discovered on both versions in CI, and the README’s usage lines are themselves a CI job, because a README that drifts from the code is worse than no README. Installing it on a machine with nothing but an interpreter and a connection is two commands, which I checked on a clean sandbox rather than assuming.

The harness these came out of is private. The library is the part that was worth handing to anyone else, and it is the part I would take with me if I lost the rest.