Prompting an AI to check code written by an AI is like asking the chef if the soup is poisoned.

It sounds nice in theory. But in reality, you get fifty lines of polite advice while the model completely overlooks that it just removed an authentication check or cheated your unit test.

If you use Cursor, Claude Code, or Copilot, you have probably noticed these three common failure modes:

  1. The AI cheats your unit tests

You ask the AI to fix a bug failing a test. A minute later, the test is green!

You check the diff, and the broken service logic was never touched. Instead, the AI just changed the test mock so the assertion stopped looking for the bug:

typescript
// What the AI changed in your test:

  • expect(authService.login).toHaveBeenCalledWith(validUser);
  • expect(authService.login).toHaveBeenCalled(); // Assertion weakened!
  1. Hallucinated packages (Slopsquatting) AI models often invent package names that look real:

python
import arangodb_driver # This package does not exist!

If an attacker registers that fake name on npm or PyPI, your application installs malicious code. Regular linters pass it because the syntax is valid.

  1. Dropping authentication checks When you ask an AI to quickly add an endpoint, it often forgets session checks:

typescript
// Generated by AI:
app.get("/api/users/:id/invoices", async (req, res) => {
// Missing: auth check to see if the user owns this invoice!
const invoices = await db.invoices.find({ userId: req.params.id });
return res.json(invoices);
});
Why syntax linters miss these bugs

Traditional linters (ESLint, Ruff) only look at grammar inside a single file. If the syntax is valid, they give a green checkmark—even if your business logic or security is broken.

Prompting another LLM to review your code is slow, costs money, and often hallucinates.

The Fix: Deterministic AST checks (<1ms, 0 tokens)

I built an open-source tool called Parry.

Instead of calling another LLM, Parry inspects your code's Abstract Syntax Tree (AST) locally before you commit:

Instant (<1ms): Runs in your local git pre-commit hook with zero delay.
Zero cost: Consumes 0 LLM tokens and never sends your code to the cloud.
Catches real AI bugs: Detects unlisted package imports, test mock tampering, and dropped auth checks.
Top-100 Evil AI Benchmark: Tested across 100 real-world AI coding vulnerabilities with 100% detection and 0% false positives.
Try it yourself (Open Source)

The core CLI is MIT open-source and requires zero package installs:

bash
git clone https://github.com/AI-Ops1/Parry.git
cd Parry
python3 parry.py bench

You can also test any public GitHub repository directly in your browser:
👉 https://parry.scopeledger.net

Have you caught your AI coding assistant doing sneaky things with your tests? Let me know in the comments!