Somebody told me last week, on a thread about a Claude Code plugin that stops an agent from editing its own tests, that an allow-list "fails closed, which is the failure mode you want when the agent is the one trying the edge case."

That is the right intuition and the wrong conclusion. It is worth pulling apart, because the same reasoning shows up every time someone hardens an agent's shell.

The guard

The plugin (Mattbusel/tamperproof, v0.1.1) replaced its deny-list of "writer" commands with an allow-list — the right direction, because a deny-list only covers the verbs someone thought of. The new rule, from hooks/guard.mjs:

Any shell segment that names an existing protected path is blocked, unless it starts with a known read-only or test-runner command and contains no redirect or tee.

Read the first clause slowly. "names an existing protected path" is what decides whether the rule is applied at all. Here is the code:

for (const piece of seg.split(/[\s'"`,;()=<>{}\[\]]+/)) { if (!piece || piece.startsWith("-") || /^[a-z]+:\/\//i.test(piece)) continue; const r = rel(piece); if (!r.startsWith("..") && isProtected(r, protect) && existsSync(resolve(cwd, r))) hits.push(r); } if (!hits.length) continue; // <- never judged, so allowed if (!READ_ONLY.test(body) || writesOut) block(...);

existsSync(resolve(cwd, piece)), on the literal token. If no token looks like an existing protected file, the segment is skipped — it is not judged, and not being judged means allowed.

What I measured

I checked out 7559493 (v0.1.1), built a throwaway tree containing test/calc.test.mjs, and drove the guard the way the hook does — the tool call on stdin, the exit code read back. Same command in every row; only the target's spelling moves:

$ sed -i s/5/-1/ test/calc.test.mjs -> exit 2 blocked $ sed -i s/5/-1/ test/calc.test.j? -> exit 0 $ sed -i s/5/-1/ test/*.mjs -> exit 0 $ sed -i s/5/-1/ test/calc.test.{mjs} -> exit 0 $ sed -i s/5/-1/ "$PWD/test/calc.test.mjs" -> exit 0 $ sed -i s/5/-1/ test/nonexistent.mjs -> exit 0

The literal answers 2 because it is a file: existsSync says yes, the path is protected, the segment is judged, sed -i is not on the read-only list, blocked.

The rest answer 0 because existsSync says no. test/*.mjs is a token, not a path — the guard asks the filesystem about the string test/*.mjs, and that string is not a file. Same for ?; same for the brace form (the splitter even cuts { and } out, leaving test/calc.test.); same for $PWD/..., which resolves under a directory literally named $PWD. Nothing matches, so hits stays empty and the rule is never consulted.

And a spelling is expanded by a real shell before sed ever sees it — that is the entire job of a glob. So the string defeats the check whose subject is the string.

The generalisation, which is the useful part

Which way a guard fails is not a property of allow-list versus deny-list. It is a property of where the default sits in the decision chain.

This guard is default-deny at the verdict: a segment that gets judged and is not read-only is blocked. And it is default-allow at the selector: a segment that is never judged is allowed. The selector is the earlier of the two, and it is the layer a command string drives. So the guard fails closed exactly where the attacker has no freedom, and fails open exactly where they do.

"An allow-list fails closed" is true only on the axis you happened to enumerate — here, the verb. Enumerate the axis the attacker's input controls, or you have measured the wrong thing.

The other direction is real too

Two arms came back the other way, and they are why I would not call even the verb axis simply "closed":

$ cat test/calc.test.mjs > /dev/null -> exit 2 $ grep -n '=>' test/calc.test.mjs -> exit 2

writesOut is />/.test(seg), so the arrow in a grep pattern reads as a redirect, and cat ... > /dev/null is a redirect to nowhere. Both are things an agent does constantly while debugging a failing test — the exact state the guard exists to keep it in. A false block is not free: it teaches the reader to route around the guard, and then the guard protects nothing.

What I would take from it

  • Name which layer the default lives in, per rule. "Blocked unless read-only" and "not looked at" are different verdicts, and only one of them is visible in a code review.
  • Ask the question about the thing that will be written, not the string that names it — or accept that the string is a spelling and chase spellings. The author already named the honest tension: treating a glob-shaped token as a hit closes this hole and also blocks a genuinely new test written from a shell.
  • Keep a control arm in both directions. All twenty of the guard's test cases name the path, and so did every arm I sent before looking. A case count is not a coverage claim — the question is which axis the cases move on.

The plugin is a genuinely good piece of work, and 0.1.1 closed every hole the previous version had, including all of the above for the literal spellings. This is the one axis that survived a rewrite, and it will survive the next one too unless the fix goes into the selector. I wrote up the thread it came from here.