I asked for one more way AI code is "almost right". You gave me seven.

Last week I wrote about 7 ways AI code is "almost right". At the end I asked you for an 8th way and promised to share the best answers here, with credit.

I asked for one. You gave me seven: five brand new ones, and two that make checks from my list sharper. Mudassir also shared a simple rule from his team, it's near the end. Honestly, a few of your replies taught me more than my own post did. Thank you to everyone who took the time to write them out, some of these clearly came from a bad day in production.

Quick confession before we start. The cover for that post came from an image generator, and on the first try it dropped one word from my headline. The picture looked great. The sentence was wrong. I caught it just before saving. Kind of embarrassing, considering what the post was about.

8. Right shape, wrong value

Maxwell's comment is the one that worries me most. No exception, nothing to catch, the field is filled in. It's just not what the source said. In his words, "a parser can read annual as monthly across a whole layout and never once fail."

30-second check (his): pick 5 rows at random and compare them with the source by hand. Random matters. If you only check the rows that already looked odd, you catch nothing.

I'd add two things: check the labels and units in those rows too, and if the source has a total you can trust (a column sum, a row count), compare it. A matching total doesn't prove every field is right, but a wrong conversion usually shows up there first.

9. You asked for one line. It tidied three files.

Aliza Ali calls this silent scope creep. You ask for a one-line fix, and the diff also renames a variable, "cleans up" an import and touches files you never mentioned. On their own, none of those changes looks worth worrying about. Together they change a lot more than you asked for.

Her follow-up made a point I hadn't thought about: auto-accept makes this much more likely, but step-by-step review doesn't fully save you either, because each edit looks reasonable in isolation. The final diff is the only view that shows the whole footprint at once.

30-second check: run git diff --stat before reading a single line (or git diff --stat main...HEAD if the agent already committed). More files isn't automatically bad, since a real fix can need its test and a type too. But ask about every file you didn't expect before reviewing anything else.

10. State that drifts

This one is harder to spot, because the code reads clean. Khali Sollis described a custom hook or async routine that passes the first render test but quietly misses a reactive dependency, or closes over stale state. It "works fine during a quick manual check, but drifts out of sync the second real users start interacting with the UI sequentially."

30-second check: don't test it once after a fresh load. Change the state a few times in a row, and trigger anything that runs later (timers, async callbacks). Also look for a newly added eslint-disable for react-hooks/exhaustive-deps in the diff. That rule catches many of these, and silencing it is the quickest way to make the warning go away.

11. The test that would have caught it was mocked away

mp3sss sees this in Electron work. An AI-written IPC handler looks correct but never validates what the renderer sends, and the integration test that should catch that is mocked. Same story around auto-update and crash reporting: the happy path is easy to generate, and the failure path (network down, partial download, a stale crash dump) is exactly what gets skipped.

Mudassir Khan hit the same wall from another side: a catch that returns an empty array passes every test, because the same agent wrote the tests.

30-second check: for each AI change, force one failure on purpose. Cut the network, send a bad payload, break the dependency, and make sure you get an error instead of a quiet empty result. As mp3sss put it, "pairing each AI change with a forced failure-path test has saved us more than the happy-path tests did."

12. The flaky test that got a longer timeout

Max Kaya added this one to the test checks from #2. Besides toBe quietly turning into toBeTruthy, watch for a timeout extended "to fix flakiness". In his words, it "usually means the test now passes by construction."

Sometimes the old timeout really was too tight for CI. Often, though, the slow call or the race is still there, and you've just stopped hearing about it.

30-second check: search the diff for bigger timeout numbers and new retries (retries, jest.retryTimes, --retries). Each one should point to a timing problem someone actually understands.

13. A real package, but far too much of it

Artur Pokładek called this a cousin of #3. The package exists and is popular, it just pulls in a pile of dependencies for something that was fifteen lines of code. He accepted one from an agent without looking.

30-second check (his): git diff --stat on the lockfile. If it grew by hundreds of lines for a small feature, ask why. A big lockfile diff can have boring causes too, so treat it as a reason to look, not as proof.

I asked him what he does then, and his answer is a better rule than anything I had. A small helper, he writes himself and tests. Dates, parsing and crypto stay with a well-known library, because there his own fifteen lines would be the bug. The small lookalike package is the option he trusts least.

14. It passes everything, and never ships

Guillermo Baigorria had this happen the day before he commented. The AI fixed a bug with a new patch folder, and every test passed on his machine. But the deploy script packs a fixed list of folders, and the new one wasn't on it. No test could see that, because tests run where the file exists.

Paul Lovelace called it the gap between where tests run and what actually ships, and said this is the check he'd steal first.

The list is hard-coded on purpose, by the way. A fixed list is what keeps a stray .env from shipping. So Guillermo's team checks the built package instead of the script, flags new files, and a person decides.

30-second check (his): list the files the change added, then look for each one in what you ship.

git diff --name-status main | grep '^A'

Compare that with your Dockerfile COPY lines or the files list in your package.json.

A rule worth copying

Mudassir also sent a defence against #2 from last time. His point: when the test file changed in the same diff, a green run tells you very little. It shows the new test passes, not that the old behaviour is still checked. So his team's rule is that any diff touching a test file needs a one-line reason in the PR, and their review bot fails a test that got skipped or loosened to toBeDefined.

What I like is how cheap it is. Whoever changed the test just says why. And if nobody can explain why a test changed, that's a good reason to stop and look closer.

Your turn again

If I had to pick the worst, it's #8. A fix that never shipped usually gets noticed once someone hits the bug again. A wrong number in a report can quietly survive for months.

Which of these have you actually run into? And is anyone catching any of them automatically in CI, without someone reading every diff? If the comments grow like last time, maybe there's a part three in here.

Small note: Mudassir's rule reminded me of my free Claude Code plugin, proof-of-done. It flags test changes that make checks pass without a fix (skipped tests, weaker assertions, deleted tests, a new eslint-disable or @ts-ignore) and won't let the agent say "done" until its report explains them. Since the last post it also checks each package in a monorepo on its own, which came from an earlier comment by Mudassir, and points out changed lines your tests never ran. Watch the 30-second demo.

Everything I've built (all free and open source)

which-ai-to-use: picks the best AI and model for each task, and hands your work to another AI when you hit a usage limit

Try it: npx which-ai-to-use
npm · VS Code Marketplace · Open VSX (Cursor, Windsurf) · GitHub · Website

proof-of-done: a Claude Code plugin that won't let the AI say "done" until the tests pass

Install: /plugin marketplace add WST2002/claude-code-skills then /plugin install proof-of-done@claude-code-skills
GitHub · Featured on the HOL blog

security-audit: a Claude Code skill that finds security holes before attackers do