Uncle Bob Stopped Reading His Agents' Code. I Got There From the Other Side, With Three Vendors.
A video summary of two recent Uncle Bob interviews went past my feed this week. His position, roughly: he no longer
reads the code his AI agents write. Instead he holds them to three constraints:
- Unit tests as double-entry bookkeeping. Every behaviour is written down twice, once as code and once as a test, and the two must agree.
- CRAP scores. Change Risk Anti-Patterns, from Alberto Savoia and Bob Evans (2007): cyclomatic complexity combined with test coverage. A complexity-10 function scores 110 with no tests, 22.5 at half coverage, 10 when fully covered. Above 30 is the usual warning line.
- Mutation testing. Break the code on purpose and check that some test goes red. A "surviving mutant" is a change no test noticed.
He also described running a five-station pipeline of agents in August, then dropping it in September for one agent
working as a senior engineer. And the point I agree with most: responsibility stays with a human.
I recognised almost all of it. I run a small Kubernetes platform built by me and a group of AI coding agents, and over
the past year it ended up in nearly the same place. I just came from the opposite direction. Uncle Bob started from
"I stopped reading code" and added constraints until he could trust the output. I started from agents that reported
commit SHAs that didn't exist, and added gates until their reports didn't matter.
(A fair caveat the video itself makes: the podcast is run by a company that sells "replace code review with verified
intent", and Uncle Bob sells agent courses. Everyone in the conversation has a stake in the conclusion. So do I: I
built the thing I'm about to describe.)
The one real difference: three companies, not one
Every pipeline in those interviews, and in most of what I read, is one vendor's model with many subagents. Mine
isn't:
| Role | Who |
|---|---|
| Coordinator: writes specs, verifies, lands | Claude (Anthropic) |
| Implements | a Codex model (OpenAI), Gemini Flash (Google) |
| Independent review, bug classification | Gemini (Google), a second Claude model |
| PR review | GitHub Copilot |
I wrote about the monoculture argument back in March: models from the same vendor share training and blind spots, so
a same-vendor reviewer tends to approve the same mistake the worker made. This article is about what that setup looks
like eight months later, once tests, coverage and mutation testing are part of it, and what broke along the way.
Constraint 1: tests that can fail
Every task goes to a worker as a spec with an Acceptance block: what must happen, what must not happen, and the
evidence that proves each. The rule that does the most work is short: every "must not" test has to be red on the
code before the fix. A test that passes on broken code proves nothing, and a worker from any vendor can write one.
The worker's report is not evidence. The coordinator reruns the suites itself. That habit came from an early agent
that cheerfully reported a commit hash that was never pushed.
Constraint 3: mutation testing, by hand, and the mutant that reached sudo
This week's task was a TOTP gate for a web console: a small daemon checks a one-time code before a shell session
opens. The worker's tests passed. The coordinator then generated 168 mutants of the new code and reran the suites
against each one.
- 158 went red. Good.
- 5 survived. Four became requests back to the worker: here is a change your tests can't see, write the test that catches it. The fifth turned out to be equivalent (more below).
- 1 hung, and that one taught me something.
The hang. One mutant changed a terminal-restore call from TCSANOW (restore now) to TCSADRAIN (wait until
output drains). The test went red, as it should. But the restore was waiting on a pseudo-terminal nobody was reading,
so the test process blocked in the kernel, in macOS process state U. kill -9 can't reap a process in that state.
Two of them are still on that laptop and will stay until a reboot. The fix was in the test fixtures: kill the process
group first, close the terminal, and put a time limit on every wait. With the mutant applied, the whole file now fails
in about 20 seconds and leaves nothing behind.
The sudo prompt. Another mutant removed a guard in front of a privileged call. The tests stubbed sudo by putting
a fake one first on PATH. But these tests run the real entry point, and the real entry point resets PATH for
safety. So the mutant reached the real /usr/bin/sudo. Three times, across three runs. Each time it hit a password
prompt with no cached credentials, a watcher killed it within about two seconds, and nothing ran as root. It still
happened three times.
PATH stubs couldn't contain this, because the code under test is designed to throw them away. What worked was the
operating system: every mutant now runs under macOS sandbox-exec with a profile that denies executing
/usr/bin/sudo and /bin/launchctl, plus a watcher for any root sudo process, plus fresh backups before each round
and a byte-for-byte check that every file was restored.
The lesson I'd pass on: a mutant is code you broke on purpose. If the code touches anything privileged, running
mutants needs a sandbox, not just a test runner. Nobody in the videos mentioned this, probably because their examples
don't call sudo.
The equivalent mutant. One survivor cached a state file per connection instead of loading it per request. It
looked like a real gap: a replay could slip through a stale cache. The worker said no test could catch it and wrote
none. That's the kind of report I don't take on trust, so the coordinator read the daemon: it loads state only after
a full request line arrives, then closes the connection. One load per connection, so caching per connection changes
nothing. Equivalent mutant: no test can kill it, and a person (or a coordinator) has to decide that. Mutation testing
can make that list short. It can't make it empty.
Constraint 2: CRAP, which I don't have yet
I don't measure CRAP today. The repo has no coverage tooling at all: no coverage.py for the Python, no kcov for the
Bash. Complexity is only an if count in an audit script. That's the honest gap the video showed me.
It's now two rows in my ideas backlog, each with make targets so it's one command and not a ritual:
-
make crapandmake crap-push: score per function, sent to Prometheus, with a Grafana panel for the riskiest functions and the trend per release. Python first, Bash after proving kcov works on my setup. -
make mutate FILE=...andmake mutate-task SLUG=...: the hand-run mutation pass above, turned into a tool. The sandbox lives inside the tool, so there is no way to run a mutant without it.
They belong together. A full mutation run is slow, and CRAP says where to spend it: high complexity, low coverage,
first.
Where the human stays: the gate that blocked my own coordinator
Uncle Bob says responsibility can only sit with a person. Here is what that looks like in practice.
Landing a task goes through gates. One refuses any task that changes the agents' instruction files (CLAUDE.md,
hook settings) unless I pass an override. A worker that edits its own rules is the failure I most want to stop.
This week the TOTP task was refused by that gate. The task never touched those files. It had been rebased over 68
release-branch commits, one of which was mine, changing CLAUDE.md. The gate diffed from the original dispatch point
instead of from where the task branched off the current release branch. The scope check next to it already did this
correctly. The instruction-file check didn't.
Two things happened that I'd call the system working:
- My coordinator tried to pass a different override earlier in the same land, without my go. The tool's permission layer blocked it and the coordinator asked me. Overrides are mine to give.
- It didn't ask me for the instruction-file override at all. It proved the task's own commit had none of those files, filed the gate's range bug, wrote the fix spec, and once I approved it, sent it to a worker from a different vendor. The right fix was to repair the gate, not to normalise the override. An override used every week is one that will someday hide a worker's real edit.
The fix landed the same day, after the coordinator ran its own mutants against it: each guard broken in turn, each
caught by a test. The TOTP task's next land attempt then failed for a different reason, and an honest one: a test
from an older change lists every background service by name, and the TOTP task added a new one. The worker had not
run that suite, and neither had the coordinator. The land gate runs it, which is the point of the land gate. That
was a one-word fix sent back to the worker. The TOTP task lands after it, with no instruction-file override, as the
live proof.
Where I ended up differently
I kept the pipeline. Uncle Bob dropped his for one senior agent because handoffs cost too much. Mine costs too: I
had to build the coordination layer myself (a dispatcher, an event ledger, Acceptance blocks any vendor can read, land
gates), and this week's gate bug is part of that bill. I pay it because the separate verifier is the point. One agent
grading its own work is the thing I stopped trusting first.
I still read code at the gates. Not every line. But the equivalent mutant, the gate's range bug and a merge
conflict in a config list were all found by reading, not by a metric. Mechanical constraints narrow where to look.
They don't remove the look.
Vendors also spread the load. When I capped one vendor's PR reviews this month to save budget, release reviews
moved to another vendor's model instead of stopping. Not the main reason to do it, but it's real.
If you run agents from one vendor: what checks the checker?