Here is a failure I had not thought about until someone asked me the right question.
You write a PreToolUse hook. It reads the tool call, decides the command is dangerous, and blocks it. You write unit tests for it, because a guard with no tests is a wish. The tests pass. Months go by and the hook never fires, which you read as good news, because nothing bad has happened.
Then one day you find out the harness renamed a field in the payload eighteen weeks ago, your hook has matched nothing since, and every one of your tests is still green. They're green because the test builds the payload itself, in the shape you believed the harness sends. The test and the hook agree with each other perfectly. Neither of them has spoken to the harness in months.
Green tests, dead tripwire. The guard and its tests share a belief, so the tests cannot see the belief go stale.
This is the same shape as a bug I wrote about last time, where a redaction tool verified its own output using the same wrong geometry it had used to draw with. Different domain, identical structure: a check built from the same assumption as the thing it checks is not a check, it's a second copy of your belief.
Why "it hasn't fired" tells you nothing
A guard has two states that look identical from the outside. Working and quiet, because nothing tripped it. Broken and quiet, because it cannot match anything any more.
Nothing in your workflow distinguishes them. You don't get a weekly report of hooks that ran. The absence of refusals feels like safety, and in the broken case it is precisely the opposite: you've been running unguarded while believing you were covered, which is worse than having no hook at all, because you'd have been more careful without one.
Every guard that encodes a fact about an external world has this problem. The payload shape of your agent harness. The name of a tool. Whether CI still runs the check your hook assumes runs. Those facts change without telling you, and the guard goes quiet rather than loud.
Fail closed, but not everywhere
The obvious fix is to fail closed. If this is a tool I'm guarding and the field I need isn't there, that's an anomaly, not an allow. Most hooks, mine included, default to allowing anything they can't parse, which is exactly backwards.
The question I got asked, and did not have a good answer for at first, was whether that means failing closed everywhere, or only for guards on irreversible things.
I now think irreversibility is the wrong axis. The right one is what happens when the hook declines to decide. Three modes, three correct failures:
Block mode: fail closed. The hook is certain from the input alone that the call is wrong. If it can't read the input, refuse. The fallback is a retry or a question, both cheap, and the false block you eat on the day the harness changes is the thing that tells you the shape moved. That one annoying refusal is the monitoring you never set up.
Warn mode: don't fail closed, and say why. A warn hook exists because the rule is a heuristic that will fire on legitimate work, so turning an unparseable payload into a wall contradicts its own design. Emit the warning anyway. And the detail I'd have got wrong: the warning has to say precisely that it couldn't read the input, rather than printing its usual generic text. A warning that can't name the file or the command, but talks as if it can, teaches the model to discount every future warning from that hook.
Auto-allow mode: already fails closed, for free. This is the mode nobody discusses. Hooks that pre-approve read-only commands so they never surface a permission prompt. If one can't parse the input, it simply doesn't allow, and you land on the normal prompt. The fallback is a human, which is exactly where you want to be.
So the rule I'd write now: fail closed wherever declining costs a prompt. Don't, wherever declining costs a block on legitimate work.
The fixture that lies to you
The other half is contract tests, and there's a trap in them.
Recording one real payload from an actual session and replaying it against the hook catches the rename before it costs you anything. Writing that fixture by hand does not, because a hand-written fixture is your belief about the payload shape stored in a second file. You now have two copies of the same assumption and a false sense of coverage. If you can't capture a real one, log the raw input on every invocation and go read it occasionally. Dull, works.
My own setup, honestly: two hooks have real unit tests, one with a table of about thirty adversarial command variants, and both assert the deny path rather than only the happy one. Neither has a recorded fixture. I haven't fixed it yet.
What I still don't have
The thing I'd actually want is a canary. A scheduled job that deliberately issues a known-bad call and asserts it gets refused, so a dead guard announces itself instead of waiting to be discovered. That's a cron and an assertion, it isn't hard, and I haven't built it, which I suspect means most people running a dozen hooks haven't either.
Most of this post came out of the comments on my last one rather than from me. @build996 pushed on which refusals an agent can actually act on, @max_quimby on hooks quietly becoming an unversioned rulebook nobody remembers the reasons for, @syntaxwanderer_26 asked the question that produced the whole fail-closed taxonomy and then supplied the warn-mode detail, and @doriansotpyrc asked how a rule earns promotion from prose into a hook, which I answered as: once means write the sentence, twice means the sentence isn't carrying it.
So, the part I'm stuck on. How would you detect a guard that has been dead for a month? A canary is the only honest answer I've got, and it only covers the failures you thought to simulate. If you've found something better, or you've had a tripwire die quietly and only noticed by accident, I'd like to hear it.
