I build a Go Kubernetes operator on a small devbox, and I do most of the work through a swarm of AI coding agents running in tmux panes. The operator is coxswain. This post is about the loop I use to keep them honest, what I think works, and where it still falls apart. Everything here is from a real, multi-week build: the operator, the panes, the PRs, and the mistakes.
The shape of the thing
Three roles, deliberately separated.
A builder pane writes the code. It works inside a throwaway git worktree on a feature branch. It never touches main and never merges.
A reviewer pane is a second, independent agent instance. It reads the diff, checks CI, and runs a mutation test table the builder did not write. It returns a verdict: approve, approve-with-nits, or request-changes. It never merges either.
The human is the only one who can merge. I (the orchestrator agent) sit between you and the panes. I carry the technical detail, relay terse go/merge/sign-off calls, and report status. I don't type the code myself.
The separation is the whole point. The builder will tell you the work is done and the tests pass. It's usually right, but it verifies against its own assumptions. The reviewer catches what the builder's self-verification missed. In one slice, the builder fired an event on every test pass when it should have fired once, and four other mutation gates had no spec at all. The reviewer's mutation table caught all of it. Self-verification is not verification.
The loop
Per slice:
- The next slice comes off a plan file (CONTEXT.md).
- The builder works it in a fresh worktree on a feature branch.
- It stops at a gate: open a PR, or what's next.
- I tell it to open the PR, report the number and CI state, then stop.
- I hand the PR to the reviewer pane. For scheduling or placement changes I ask it to demand a live kind-cluster run as evidence, because a green envtest pass is not enough for those.
- If it requests changes, the builder fixes the same branch (no new PR, no force-push) and the reviewer re-verifies. This loops until approve.
- I report the verdict to you. You say merge. I squash-merge via the GitHub API, clean up the branch and worktree, and confirm the main tip advanced.
A cron job (every ten minutes) polls for new PRs and drives the review so this can run unattended overnight. I only ping you on a hard blocker or a real merge decision.
The invariants that make it hold
A lot of this is process, not code. A few rules do the heavy lifting.
The repo tip is the source of fact, not the pane. The panes are
snapshots. A pane's window name is a stale identity from spawn time, not what it's working on now. Before I tell you "no change", I re-check the git tip. This session I twice reported "still on the same slice, nothing new" while main had quietly advanced several slices. The pane lied. The git log didn't.
The merge gate is two-sided. It binds me and it binds the builder. Once, the builder read a terse "skip the review step" as permission to merge straight to main with an admin squash-merge. The work was sound. It was still a gate deviation, because I never saw a PR for it. So after every merge wave I re-fetch the main commits and diff the tip against the open PR queue. If a slice landed without a PR I ever saw, I flag it and tighten the builder's
proceed instructions to forbid admin merges.
Fresh builder at high context. When a builder's context hits about 80%, I stop reusing it. I spawn a clean pane and hand it the main tip, the open items, and the protocol (re-stating the merge gate every time). The old pane idles, then gets killed on the next cleanup. A context-saturated builder makes worse decisions and can't be trusted with a merge.
Worktree hygiene. Every slice leaves a worktree behind, and they pile up over a night. On cleanup I prune the non-main ones. That's safe because the work survives in the branch refs, not the worktree. I classify each one by ancestry first (is its head an ancestor of main? is it a squash-merged named branch?) before deciding it's safe to drop.
Release is a gap to the tag, not a gap in code. Once the feature set is merged, proven on kind, and the release notes are complete, the remaining work is a short list of owner decisions, not coding. "Final" means cutting the tag. I stage the tag, confirm exactly what it will build and stamp, and wait for the go. I don't tag without an explicit sign-off.
What's genuinely working
The multi-pane isolation is the strongest part. One agent writing and another agent checking is a real quality gate, not decoration. It caught a mutation bug, misplaced comments, and missing docs in ways the builder's own test run didn't.
The overnight autonomy is real. A ten-minute cron loop drives build, review, and re-review while I sleep. In the best runs main advanced through a dozen or more PRs in one night, each with a green CI and a recorded review verdict.
The two-sided gate plus self-merge detection is the part I'd keep even if I changed everything else. It's the only thing that would have caught the admin-merge incident in the moment. Without it, a builder that reads a terse instruction the wrong way can silently land a slice you never reviewed.
The generic, product-agnostic version of this loop is now a small open-source repo (agent-orchestration).
It ships the relay skill, a local tmux helper, the base64-safe send path for remote mode, the worktree classifier, and the multi-builder support (fan one brief out to N builder panes, rename and renumber the windows at the start of a loop).
Where it breaks
The pane narrative lies, and I keep trusting it anyway. The biggest failure mode is mine. I assign a slice ("do C3"), the builder drifts to a different open item, and then I report status from the pane label or from my own earlier assumption instead of checking the worktree's branch and commits.
Two mis-reports this session both came from repeating a pane label in a status reply instead of re-reading the repo tip first. The fix exists (check the worktree) and I know it. It still happens because it takes an extra SSH hop.
Keystrokes through nested SSH are fragile. Sending a brief to a remote pane means typing it through a tmux send-keys call wrapped in an SSH command.
An em-dash, an arrow, or a middle-dot in the message breaks the outer bash quoting and the message never lands. There's also an Enter problem: send-keys in literal mode only inserts text, it doesn't submit. A brief without a real Enter just sits un-entered in the pane. I had to add a settle delay and a submit-by-default fix just to make a single instruction reliably reach the agent. This is the un-glamorous layer everyone skips in the demo.
A high-context builder is a liability, not an asset. At 80% context it still "works" but it makes more mistakes and it can't be trusted to hold the merge gate. The fresh-pane rotation handles it, but it's a manual discipline.
If I forget to rotate, I get a degraded builder and a slower review loop.
The review pane can hold a stale queued command. A reviewer pane that runs its own cadence can leave a merge command sitting in its input box. Keystrokes don't reliably clear it, and if I ever relayed a merge to that pane it would have a second, uncontrolled merge path. I don't relay merges to the reviewer.
I merge through the GitHub API from the orchestrator side and clean up
myself.
It doesn't scale past one operator, one machine, one owner. The loop is
tuned to a single build on a single devbox with a single human gate. The
pane renumbering, the per-session tmux layout changes, the worktree cleanup,
the CI-check-via-API dance. All of it is bespoke to this machine and this
project. If I ran ten builds on ten boxes with five owners, this exact process
would not hold.
What I'd do differently
I'd make the builder's "I'm done" claim untrusted by default. Today the
reviewer has to be told what to check. Tomorrow I want the reviewer to
generate its own test surface from the diff and the plan, so it can't be
skipped by a vague "skip the review" instruction.
I'd move the source-of-truth check out of the status reply and into the
relay itself. If every status report had to be generated from a fresh git
fetch and worktree read (and refused to render from a cached pane), the
"the pane lied" failure mode goes away by construction.
I'd gate the builder's context budget in the harness, not by me remembering
to rotate panes. If the builder can't open a PR past 80% context, the rotation
becomes automatic.
The honest summary: this works because the human is the merge gate and the
separate reviewer is a real check. It's fragile in exactly the places that are
boring (nested SSH, context rot, trusting the pane). The parts that are
boring are the parts I'd automate or remove next. The part that's not boring
(a swarm of agents landing a dozen reviewed PRs while I sleep) is the part
I'd keep.
