My coding agent ships a page in minutes. It also breaks the same unwritten rules every time. No theme switch. A sticky header that covers the section you linked to. A menu that stays open after Escape.

I wrote those rules down and made them executable. The project is called Plumb. A rule is a short markdown file. Plumb turns it into a Kane CLI flow, runs it in real Chrome and hands the step that failed back to the agent. Then it checks again.

I built it for the Kane CLI Online Hackathon. Kiro wrote most of the code. The demo site and its fixes came from opencode running gpt-6.1-sol. The runs below are real.

What a rule looks like

A rule has a statement, the browser steps and the usual fix. The steps are plain English. This is the theme rule:

  1. Look for a control that switches the page between a light and a dark theme. Verify such a control is visible.
  2. Click the theme control. Verify the page background colour changed.
  3. Reload the page. Verify the page still shows the theme that was chosen before the reload.

Plumb ships 12 house rules. You can add your own. A rule with the same id as a house rule replaces it.

The run

I asked the agent to build a four page tool lending site called Lantern. I did not mention any rules. Its first draft stopped with two pages and a script missing. Then Plumb took over.

Check Rules passing Kane credits Agent
First check 6 of 12 387 none yet
After attempt 1 11 of 12 195 11 minutes, 8 files changed
After attempt 2 12 of 12 43 2.5 minutes, 1 file changed

The loop prints one line per check and one line per agent attempt:

End of the plumb loop output: iteration 0 has 6 of 12 rules passing, iteration 1 has 11 of 12 after 8 files changed, iteration 2 has 12 of 12 after 1 file changed, 624.6 credits in all

Plumb also ships a web app, the Studio, that replays the loop with every result and screenshot:

The Studio loop page: 6 of 12 rules passing at the start, 12 of 12 at the end, 2 agent attempts, 625 Kane credits

What Kane found

The missing script was a 404 in the browser console. Plumb reads Kane's console log for that rule, so it failed with no model judgement involved. Kane's own steps passed. The console did not.

The PL-CONSOLE rule in the Studio: both steps passed but the console reported two 404 responses, so the rule needs work

After attempt 1 one rule was still red. The Browse submenu stayed open after Escape. I had not looked for that. Attempt 2 fixed it.

For each failing rule Plumb writes a brief for the agent. This is the theme rule from the first check:

The fix brief for PL-THEME: the rule, the step that failed, what the browser saw, Kane's suggested change and the usual fix

Why the last check was cheap

Kane writes a flow once and replays it later. On a three step rule, writing and running it cost 47.7 credits and 92 seconds. A replay cost 1.9 credits and 16 seconds. In the last check 11 rules replayed and one healed, so it cost 43 credits against 387 for the first. Plumb also skips a rule when neither the page nor the rule changed.

The agent cannot change the judge

Before each attempt Plumb hashes the rules, the generated tests, the Kane policy and the config. If the agent edits any of them Plumb restores them and says so in the next prompt. A Kane policy also fails the run if a rule has no check, so a rule cannot pass by asserting nothing.

Plumb checks itself

I pointed Plumb at its own web app. Seven of eight rules passed. The eighth failed because my rule said same footer but only stored the navigation labels. The app was fine and the rule was wrong. I fixed the rule and it passed.

What I did not check

A model drives the browser, so rules about judgement can vary from run to run. Every number here is one run on a small page, not a benchmark. I tested one site with one agent. Once, three rules started together and Kane failed to pick a project for one of them. That rule was not judged. Plumb now runs such a rule again on its own. Kane saves each run to your TestMu AI account by default.

Try it 

git clone https://github.com/zkasuran/plumb cd plumb && npm run demo

The demo opens the Studio with the recorded runs and needs no Kane account. Plumb is Apache-2.0 and runs on Kane CLI by TestMu AI. The code is at github.com/zkasuran/plumb.