I run a small side project that publishes short social posts, in Japanese, about keeping household pests out of your home. I don't write those posts by hand. A set of AI agents drafts them, a two-stage checker reviews them, and the finished drafts land in an outbox/ folder where I read them and post them myself. Nothing goes out automatically.
One day the agent that summarizes the day for me reported that four drafts in the outbox had passed review and were ready to post.
They had not passed review. One of them contained two numbers with no source, one claim about effectiveness that the checker is specifically supposed to block, and a fact that the model had copied from the wrong chapter of its own knowledge base. A second agent, the one that was supposed to catch exactly this, had never looked at any of the four.
Nothing was posted, because I re-checked before posting. But the fact that I had to re-check is the whole problem, so this is a write-up of what went wrong, why each failure got through, and the two small checks I now use. The checks are short. Getting to them took a while.
The setup
The review step is two agents in sequence, modeled on how I split gates in code review:
- scan — a small, cheap model. It compares the draft against a list of banned and borderline phrases ("guaranteed", "100%", numbers about effectiveness with no source, and so on). It only reports. It never edits.
- judge — a larger model. It only looks at the borderline items that scan flagged, and decides: allow, or fix with specific instructions.
draft → scan: compare against the phrase list → nothing found → draft is final → borderline found → judge rules → fix or allow → clearly banned → sent back to the writer Each draft is a Markdown file with frontmatter, and the result of the review is written into it:
--- status: draft platform: x compliance: passed created: 2026-07-10 --- The summarizing agent reads compliance: passed and tells me the draft is ready. That is a reasonable thing for it to do. The field says passed.
What "passed" actually meant
When I went back through the records, here is what I found.
The four drafts were created on the day I first set up this group of agents. That first run was a dry run. I was checking that the pieces connected, not producing real output. During the dry run, scan ran. Judge did not. The drafts were marked compliance: passed anyway, and they stayed in the outbox with that flag for seven days.
A different file, the pipeline log, said plainly that review had not been carried out during the dry run. So the system contained a flag saying "passed" and a log saying "not checked", and nothing ever compared them. The summarizing agent read the flag, because the flag is in the file it was told to read.
So the first failure was simple. A flag that says "passed" recorded that something had passed, but not which check had run, when it ran, or who ran it. With two stages, "passed" after stage one and "passed" after both stages look exactly the same.
What the drafts actually contained
I re-ran the review by hand on all four. Three were fine. The third post was not: it had two numbers with no source, one effectiveness claim, and a main hook that contradicted the source. And the review of it produced one more error of its own.
One of the two unsourced numbers was a claim that pest complaints rise toward a particular month. The reference document has no seasonal data at all, so there was nothing to check it against. The other one is more interesting.
1. A fact copied from the neighboring chapter
The post said, roughly, "cockroaches are said to become more active once the temperature goes above" a specific value.
The agents write from a single reference document, a long Markdown file that summarizes and cites public sources. It is organized by pest: cockroaches, ants, mosquitoes, dust mites, termites, flies, and so on. Every factual claim in a post is supposed to come from it.
I opened the file and read the cockroach chapter. It contains no temperature figure at all. The temperature figure is real, and it is in the document, but it is in the mosquito chapter, in a line about where mosquitoes breed.
This kind of error is hard to catch, and I think it is worth spelling out why:
- The fact exists in the source. A text search for the number finds it. If your check is "does this claim appear in the reference document?", the answer is yes, and the check passes.
- It is plausible. Temperature affecting insect activity is not a strange idea. Nobody reading the post would stop on it.
- It was wrapped in "it is said that". My phrase list blocked unsourced numbers stated as fact. It did not block unsourced numbers stated as hearsay. "It is said that" turned out to work as a way around the rule. It sounds careful, and it gives no source.
The model did not make the number up. It took a true sentence from the document and used it for the wrong subject. The source existed. It was attached to the wrong subject.
2. The fix had the same problem
When judge reviewed the draft during the re-check, it proposed a replacement sentence about when one species lays eggs, and added "(according to manufacturer data)".
The egg-laying season is in the cockroach chapter, so the fact was fine. But the reference document does not attribute it to a manufacturer. A manufacturer is cited a few lines above, for a different fact (how many eggs are in each egg case). The judge took the citation from that line and attached it to a different line further down.
So the agent I had set up to catch unsupported attributions produced one in its own fix. It was not a dramatic error. It is the same kind of error as the original, made by the reviewer while fixing it.
3. The main hook ignored a warning in the source
The post's main hook was that a particular household fixture is a common way for cockroaches to get inside in summer. The reference document doesn't support that framing. Its section on cockroach entry routes does list that fixture as one possible way in, but the next item adds a professional's view that it is unlikely to be one. The document also ends with a Caveats section that lists a closely related entry-route claim as disputed.
Neither scan nor judge caught this. Scan compares phrases, and nothing in the hook was a banned phrase. Judge only sees what scan flags. The disagreement was between the draft and a sentence in the source that argues against it, and no check compared those two things.
Fixing the wording would not have helped, because the whole post was built on that claim. The post was rebuilt around a different, uncontested fact from the cockroach chapter.
Check 1: a flag has to carry its evidence
The first fix is to stop accepting a bare passed. Every draft now has to carry a compliance_log that records each stage with a date:
compliance: passed compliance_log: "scan 2026-07-17 ⚠️ 1 borderline item → judge 2026-07-17 ✅ allowed" or, when scan finds nothing borderline:
compliance_log: "scan 2026-07-17 ✅ 0 banned / 0 borderline / judge不要" (judge不要 means "judge not needed". The real logs are in Japanese, and the check below matches that exact token.)
A rule in a README does nothing unless something reads it, so I wrote a check that runs before I post. This is the whole thing:
// Gate check: a "passed" flag only counts if it carries the evidence of each stage. import { readFileSync, readdirSync } from "node:fs"; import { join } from "node:path"; const DATE = String.raw`\d{4}-\d{2}-\d{2}`; const SCAN = new RegExp(String.raw`scan (${DATE})`); const JUDGE = new RegExp(String.raw`judge (${DATE})|judge不要`); function frontmatter(text) { const m = text.match(/^---\r?\n([\s\S]*?)\r?\n---/); if (!m) return null; const fm = {}; for (const line of m[1].split(/\r?\n/)) { const i = line.indexOf(":"); if (i > 0) fm[line.slice(0, i).trim()] = line.slice(i + 1).trim().replace(/^"|"$/g, ""); } return fm; } export function checkDraft(fm) { if (fm.compliance !== "passed") return { ok: false, why: "not marked passed" }; const log = fm.compliance_log; if (!log) return { ok: false, why: "passed, but no compliance_log" }; const scan = log.match(SCAN); if (!scan) return { ok: false, why: "no dated scan record" }; const judge = log.match(JUDGE); if (!judge) return { ok: false, why: "scan recorded, judge stage missing" }; if (judge[1] && judge[1] < scan[1]) return { ok: false, why: "judge dated before scan" }; return { ok: true, why: `scan ${scan[1]} / ${judge[1] ? "judge " + judge[1] : "judge not needed"}` }; } (Comparing ISO dates as strings works because they sort lexically.)
Fed the two states the drafts were in before the fix, it holds both:
{ ok: false, why: 'passed, but no compliance_log' } { ok: false, why: 'scan recorded, judge stage missing' } The second one is the failure that actually happened: stage one recorded, stage two missing, still marked passed.
The check found something I didn't expect
When I ran it over the real outbox for this article, the six drafts that were re-reviewed under the new rule that week passed. Three drafts written for the following week did not:
HOLD 2026-07-24-post-1.md — passed, but no compliance_log HOLD 2026-07-24-post-2.md — passed, but no compliance_log HOLD 2026-07-24-post-3.md — passed, but no compliance_log At first this looked like the same bug again. It wasn't quite. Those three drafts had been reviewed properly. The writing agent's daily log has the full record: scan found five borderline items, judge allowed two and gave fix instructions for three, the drafts were revised (the writing agent declined two of judge's instructions and wrote down why), and a second scan came back clean. The review happened. It was written down in a log file, not in the draft.
I'm keeping the HOLD anyway. The original incident started because the "passed" flag was in one file and the "not checked" note was in another, and nothing compared them. If I accept "the evidence is somewhere else", I'm back to that. Evidence has to be stored in the same file as the flag. That's less convenient for the writing agent, but the gate only has to read one file to decide.
It also showed me something about the rule. I added compliance_log as a requirement on the day of the incident. On that same day, a different agent produced three drafts without it. A written rule did nothing until a script checked for it.
Check 2: is the fact in the right chapter?
The neighboring-chapter error needs a different check. "Does this text appear in the source?" is the question that failed, so the check has to ask "which section does it appear in?"
The reference document is Markdown with a heading hierarchy, so a search can report the heading path where each match sits:
// Attribution check: is the fact in the section about the subject, or just somewhere in the file? export function sections(markdown) { const lines = markdown.split(/\r?\n/); const out = []; const stack = []; lines.forEach((line, i) => { const h = line.match(/^(#{1,6})\s+(.*)/); if (h) { const depth = h[1].length; while (stack.length && stack.at(-1).depth >= depth) out.push({ ...stack.pop(), end: i }); stack.push({ depth, title: h[2], start: i + 1 }); } }); while (stack.length) out.push({ ...stack.pop(), end: lines.length }); return { lines, list: out }; } export function locate(markdown, needle) { const { lines, list } = sections(markdown); const hits = []; lines.forEach((line, i) => { if (!line.includes(needle)) return; const owners = list .filter((s) => s.start <= i + 1 && i + 1 <= s.end) .sort((a, b) => b.depth - a.depth); hits.push({ line: i + 1, path: owners.reverse().map((s) => s.title).join(" > ") }); }); return hits; } The CLI wrapper prints OK when the heading path contains the subject and MISATTRIBUTED when it doesn't. Run against the real reference document with the subject set to "cockroach", the temperature claim from the incident comes back as:
MISATTRIBUTED L89 Guide > Details > 3. Mosquitoes and a fact the rebuilt post kept from the cockroach chapter (how long egg cases take to hatch) comes back as:
OK L66 Guide > Details > 1. Cockroaches > Egg-laying sites (I've translated the headings. The real ones are in Japanese.)
This is a rough check and I want to be clear about what it misses:
- It needs a searchable string. It works on numbers and distinctive phrases. It does nothing for a paraphrased claim.
- It doesn't check citations at the line level. The judge's mistake, putting a manufacturer citation on a different line from the one it belonged to, happened inside the right chapter. Both lines are under "Cockroaches", so this check would pass it. Catching that needs the citation and the fact to be linked in the source document itself, which mine aren't yet.
- "Wrong section" is not always wrong. The Key Findings summary at the top repeats facts from the detail sections, and a match in Caveats is a reason to read more carefully, not an automatic failure. It flags things for a person to look at.
Even so, it would have caught the original error. The original error would also have passed every existing check, because it was a true fact in the wrong place.
The rules I added
Beyond the two scripts, three rules went into the reviewer's definition:
- Every number or factual claim must cite either a line number in the reference document or a primary-source URL. If you can't point to one, it is a clear failure. Wrapping it in "it is said that" doesn't change that. Hearsay phrasing no longer gets around the rule. The URL option exists because seasonal posts are supposed to use current news, which can't be in a static reference file. Requiring only line numbers would have rejected every seasonal post.
- Check the draft against the source's Caveats section every time. If the main hook depends on something listed as disputed, the fix is to rebuild the post, not reword it.
- Notes in the planning log are not facts. The planning log had a note saying "next time, angle the seasonal post toward" a particular month being the peak. The writing agent researched it and found no statistic that supported it. The only figures it found showed a different month on top, and the "peak" claim only appeared in unsourced SEO articles from pest-control companies. That note had been written as a plan, and the next run was about to treat it as a fact. A log of what to avoid repeating doesn't tell you what's true.
What I'd tell myself before building this
A status flag is a claim, not evidence. passed, verified, approved, done are all claims. A flag that doesn't say who set it, when, and based on which check should be treated as if no check happened. With several stages, the flag has to show which stages ran, or "stage one passed" looks identical to "everything passed".
Look for leftover dry-run output. The day you first wire up a pipeline is the day it's most likely to produce output that looks real and isn't. I had test output sitting in the real outbox with the real flag on it for a week. Anything created on a setup day should be reviewed before anyone trusts it.
"Is it in the source?" is weaker than it sounds. A large source document contains plenty of true facts about the wrong subject. The question you need is "is it in the source, in the section about this subject, with this citation?" Each part of that question catches a different error.
The reviewer's fixes need checking too. The larger model making the final call didn't make it reliable. It repeated the original mistake while fixing it. Whatever the last automated step produces is still something I have to verify.
Store evidence with the artifact. If the gate has to look in another file to decide, it will eventually read a file that's out of date.
Rules in prose don't run. I wrote the compliance_log requirement, and on the same day another agent produced drafts without it. Nobody noticed until a script checked for it.
None of this is specific to pest-control posts. If you have an LLM pipeline with a "reviewed" flag somewhere, whether it's content, code, or data labeling, I'd suggest a quick test. Pick one item marked as passed and try to find the record of each check that was supposed to run on it, without leaving that item's file. If you can't, your flag is in the state mine was in.
How are you recording which checks ran on AI-generated output? I'm interested in whether anyone has a good way to link a citation to a specific fact in the source, because that's the part my section check still misses.
