Security Regression Gate — block the rise, not the presence

How Sentinel decides which merge requests get stopped in CI: not "this file can run shell commands" (that's been true for months), but "this change just gave the file a power it did not have yesterday".

Where the numbers come from. Everything here was measured on 2026-10-12, on the master branch of the Sentinel repository, using Node 24.19.0. I used three sources and I name the source at each number:

1. Unit tests, run for real with node --test. I did not count test( lines with grep.
2. A replay of today's gate over all 118 merge requests ever merged into Sentinel's master. This is deterministic, offline and takes one command (see Captured output).
3. An earlier evaluation on 16 public GitHub repositories and 5 GitLab ones, written up in the repo's design-partner notes. That one was produced by a throwaway script that lives outside the repo, so nobody has reproduced it independently yet. I say this again wherever I use those numbers.

There are no CI pipeline statistics in this article (how often the job actually failed on real MRs). I didn't have read access to the pipeline API while writing, so the article doesn't claim any.

Part 1 — For humans

The smoke detector that screams at candles

Imagine a smoke detector that goes off every time it sees a candle. Not smoke. Not fire. A candle that's been on the same table for three years.

The first day, you jump. The second day, you sigh. By the end of the week you've taken the battery out, and now you have no smoke detector at all.

Most "security scanners" in code review work this way. They read a file, see something dangerous-sounding ("this file can run other programs!"), and raise a flag. They raise it again the next time someone fixes a typo in that file, and the time after that. The file was always able to run other programs. The flag carries no new information, so people learn to ignore it. Once a team ignores the warnings, the scanner has stopped doing anything useful.

Sentinel's merge-request gate is built around the opposite question. It doesn't ask "is this file dangerous?" It asks:

"Did this change make the file more dangerous than it was before?"

That's the whole idea of this article. The rest is detail: how "more dangerous" is measured, what happens when the answer is yes, and the cases where the gate still doesn't see the problem. I list those too.

What Sentinel is, in one paragraph

Sentinel is an autonomous agent that watches a JavaScript codebase (its own, mostly) and decides, file by file, whether to leave it alone or propose a change. An earlier article in this series covered how it avoids re-thinking the same thought (the Novelty Gate). This one leaves the agent's head and goes to the front door of the codebase: the merge request. That's the moment a human (or an agent) asks "please add my change to the main branch". The gate lives in CI, the automated checklist that runs on every merge request. If the gate says no, the merge button stays grey.

The gate uses no AI. No language model is asked whether the code "looks risky". It's a plain program that reads code structure, and it gives the same answer every time you run it on the same input. That matters for a reason I'll come back to: you can argue with a deterministic gate, and you can reproduce its behaviour. You can't reproduce a mood.

A ladder with four rungs

To say "more dangerous", you need a scale. Sentinel uses four levels, decided purely by what the code can do:

That last row is the interesting one. A file that can start another program (a shell command, git, ffmpeg) or turn a string into running code (eval) has no meaningful upper limit on what it can do. So the ladder stops there. You can't be "more critical than critical".

Each changed file gets two readings: one before the merge request, one after. The gate compares the two.

The rule, in plain words

A merge request gets blocked when one of its files ends up at critical and at least one of these is true:

1. It climbed. It was low, medium or high before, and now it's critical. "This file couldn't run programs yesterday. Today it can."
2. It was already critical and gained a new kind of power. Say it could already run programs, and now it can also turn text into code. Same rung on the ladder, new trick. "This file could already run programs. Now it can also execute arbitrary strings."

A merge request does not get blocked when:

• a critical file is edited without gaining any new dangerous power (fixing a typo, renaming a variable, adding a log line),
• a dangerous power is removed (going down the ladder is always fine),
• a file moves up the ladder but stays below critical. This one is a deliberate choice, explained under The honest part.

Put another way, the gate reacts to the candle being lit. It doesn't react to a candle that has been sitting on the table for three years.

The permission slip

Some files should run programs. Sentinel's own report script has to ask git for the old and new version of each file, and there's no way to do that without running git. Blocking that file forever would be silly.

For these files the repository keeps a small permission-slip file called .sentinel-gate.json. Each entry says which file, why, and exactly which powers are accepted:

{ "acknowledged": { "scripts/mr-report.js": { "reason": "Reads git objects through execFileSync('git', ...) — the report has no other way to see both revisions of a file, and the arguments are never user input.", "facts": ["child_process", "process_spawn"] } } } 

That's the real file from the repository (one of its two entries). It says: this file may import the process-spawning module and may spawn processes. It does not say "this file may do anything". If someone later adds eval to that file, the slip doesn't cover it and the gate blocks again.

The slip also sits in the repository, next to the code. Someone has to write the reason down, a reviewer can read it, and the version history shows who added it and when.

The honest part: what the gate does not see

A gate is only as useful as its known weaknesses are known. These are written down in the repo, and each one is pinned by a test that passes (more on that in Part 3):

1. Climbing slowly below the top rung. If one merge request takes a file from low to medium (it starts writing files), and the next takes it from medium to high (it starts deleting files), a gate set to "critical" says nothing. Neither step reached the top. The report still shows both climbs, and the team can set the gate stricter (high instead of critical) to catch exactly this. It's a dial, not a hidden bug. The CI job in Sentinel's own repo is set to critical, so in that repo deleting files is reported but not blocked.

2. Two files working together. The gate reads each file on its own. If file A gets a function that runs a program (blocked, or explicitly allowed once), and a later merge request makes file B call that function, file B looks harmless. File B didn't change its own powers, it just borrowed A's. Seeing that needs a map of who calls whom across the whole codebase. That's a different, bigger tool, and the project's own planning document explicitly lists it under "what not to build yet".

3. The old-style permission slip. Older slips just say "this file is fine, here's why", without a list of powers. They still work, so old setups don't suddenly break. The catch is that a new power on such a file is accepted too. The gate now prints that acceptance in the merge request comment, along with the power it absorbed, so nobody can say they weren't told. Sentinel's own slips use the precise form.

4. Code the parser can't read. If a file uses syntax the JavaScript parser doesn't understand (TypeScript-style type annotations in a .js file, for example), the report says "not analysed" for that file. It doesn't guess. The flip side is that an unreadable file is never blocked. I checked while writing this: a file that gains both child_process and a type annotation comes out as parsed: false and doesn't trip the gate. It is clearly marked in the report, but it isn't stopped.

There used to be two more items on this list. An audit in September found that (a) adding a second dangerous power to an already-critical file slipped through, because "critical → critical" counted as "unchanged". It also found that (b) a permission slip for "runs programs" quietly covered a later eval too. Both were turned into passing tests that documented the hole, and then both were fixed in a follow-up merge request. That's the rule from point 2 above ("already critical and gained a new kind of power") and the precise permission slip. That order — first prove the hole with a test, then close it — is how this project likes to work.

Does it actually catch anything?

Three different measurements, in increasing order of "you can check this yourself":

Other people's code (not independently reproduced yet). The detector was run over the last 15 merged pull requests of 16 well-known public JavaScript projects (express, fastify, axios, Ghost, strapi, and others). That's 140 pull requests with JavaScript changes. On the first run, it would have blocked 10 of them, and when someone read all 10 by hand, only 3 were real. The other 7 were false alarms. Precision: 30%. That was embarrassing, and it was useful. Every one of the false alarms had the same cause, which I'll explain in Part 2. After fixing that cause, the same 140 pull requests produced 3 blocks (2%), all 3 genuine, and every changed file that really runs commands was flagged: 28 of 28. On five GitLab-hosted projects it blocked 5 merge requests, all 5 genuine when read by hand.

Sentinel's own history (reproducible with one command). I replayed today's gate over all 118 merge requests ever merged into Sentinel's master. 93 of them changed JavaScript. The gate would have blocked 3. All three are real "this file can now run programs" moments: a capture tool that shells out to an external binary, a benchmark sample that spawns workers, and a test helper that runs a child process. Two more raises were waived by the permission slip (the report script itself and a CLI test). Not one of the 118 merge requests had the "already critical, gained a new power" pattern, so the fix from the audit has never fired on real history. It's pinned by tests, not by luck.

Live production. The gate runs in CI, not inside the running agent, so the production dashboard doesn't count gate blocks at all. I won't invent a number for it.

The other gate with the same name

The name of this article is a little unfair to a module that already exists. In Sentinel's source code, the class actually called SecurityRegressionGate isn't the CI gate. It's a sibling that lives inside the agent. When Sentinel proposes a change to its own code (say, "make this login check faster"), that sibling compares the old and new version and refuses the change if a security safeguard got weaker. Examples: a constant-time comparison disappeared, there are fewer input checks than before, or a "remember the answer" shortcut now skips the real password check.

It's the same philosophy from the other side. The CI gate blocks when danger goes up. The runtime gate blocks when protection goes down. Neither one cares that danger or protection exists. Both compare a before with an after. I cover it briefly in Part 2, because it shares the idea but not the code.

The 30-second version

• The gate compares each changed file before vs after the merge request.
• It blocks when a file reaches critical (can run programs or execute text as code), or gains a new dangerous power while already critical.
• Editing a dangerous file without making it more dangerous is free. Removing danger is always free.
• Intended risk is accepted per file and per power, with a written reason, in the repo.
• Known blind spots, all written down and tested: slow climbing below the threshold, two files cooperating, old-style slips, and files the parser can't read.
• No AI is involved and the result is deterministic: on Sentinel's own 118 merge requests it would have blocked 3, all genuine.

Part 2 — For developers

Where the gate sits

flowchart TD MR["Merge request pipeline (GitLab CI, job: mr_report)"] --> FETCH["git fetch origin target branch (GIT_DEPTH: 0)"] FETCH --> CHANGED["changedJsFiles(base, head): git merge-base + git diff --name-only --diff-filter=ACMRD"] CHANGED --> BLOBS["blob(mergeBase, file) and blob(head, file) - missing revision = empty string"] BLOBS --> EXTRACT["AstFacts.extract(before) / AstFacts.extract(after) - tree-sitter, per file"] EXTRACT --> DIFF["diffFacts: set difference per fact family + riskDelta(riskOf(before), riskOf(after))"] DIFF --> REPORT["report(): findings, factCount, riskRaised, highestRisk"] REPORT --> GATED{"parsed AND risk.after >= level AND (direction == raised OR dangerous.added non-empty)?"} GATED -- "no" --> PASS["not a candidate"] GATED -- "yes" --> ACK{"covered by .sentinel-gate.json entry for this path?"} ACK -- "facts list includes every added family, or legacy string" --> WAIVED["waived[] - printed in the MR comment"] ACK -- "no entry, or a family outside facts" --> OFFENDER["offenders[]"] OFFENDER --> EXIT["exit code 1 (unless MR_REPORT_GATE=off) - pipeline fails, allow_failure: false"] PASS --> MD["render(): markdown report, tee mr-report.md artifact"] WAIVED --> MD EXIT --> MD

Two properties are worth stating before the code:

• The pure layer (libs/mr-report/index.js) does no I/O. No network, no LLM, no filesystem, no git. The caller hands it both revisions of every file as strings. There's a test that enforces this (Part 3).
• The only side effect of the CLI (scripts/mr-report.js) is reading git objects and the optional acknowledgement file. It needs no credentials, so it runs on any plain checkout.

Step 1 — what changed, against what

// scripts/mr-report.js:78-88 function changedJsFiles(baseRev, headRev) { const mergeBase = git(["merge-base", baseRev, headRev]).trim(); const names = git(["diff", "--name-only", "--diff-filter=ACMRD", mergeBase, headRev]) .split("\n") .map(name => name.trim()) .filter(Boolean) .filter(name => name.endsWith(".js")) .filter(name => !name.startsWith("node_modules/")); return { mergeBase, files: names.map(file => ({ path: file, before: blob(mergeBase, file), after: blob(headRev, file) })) }; } 

Three details are doing real work here:

• git merge-base, not the target branch tip. If master moved on after the branch was cut, diffing against the tip would credit (or blame) the merge request for other people's commits. The merge base is "what this branch actually started from".
• --diff-filter=ACMRD means Added, Copied, Modified, Renamed, Deleted. Deleted files go through the diff too, so removing a dangerous file shows up as a lowered risk instead of vanishing silently.
• A missing revision is the empty string:

// scripts/mr-report.js:70-76 function blob(rev, file) { try { return git(["show", `${rev}:${file}`], "ignore"); } catch { return ""; } } 

An empty source parses to empty facts, so an added file is simply "low → whatever it is now". That's the right baseline: a brand-new file that spawns processes did raise the risk of the repository.

Only .js is in scope. .ts, .mjs and .cjs aren't analysed today, and that's a deliberate, stated scope, not an oversight.

Step 2 — facts from syntax, not from substrings

Each revision goes through AstFacts.extract, a tree-sitter walk that records operations (booleans like processSpawn, dynamicEval, diskDelete) and derives two things from them:

// libs/core/ast-facts.js:553-561 function dangerousOf(operations) { const dangerous = []; if (operations.dynamicEval) dangerous.push("eval"); if (operations.childProcess) dangerous.push("child_process"); if (operations.processSpawn) dangerous.push("process_spawn"); if (operations.processExit) dangerous.push("process.exit"); if (operations.diskDelete) dangerous.push("destructive_fs"); return dangerous; } // libs/core/ast-facts.js:570-579 function riskOf(facts) { const operations = facts?.operations || {}; if (operations.dynamicEval || operations.processSpawn || operations.childProcess) return "critical"; if (operations.diskDelete) return "high"; if (operations.diskWrite) return "medium"; if (operations.outboundHttp || operations.inboundHttp) return "medium"; if (operations.cryptoUse || operations.secretComparison) return "medium"; return "low"; } 

dangerous is the list of fact families: eval, child_process, process_spawn, process.exit, destructive_fs. Acknowledgements use exactly these names, and they're what the gate compares when a file is already critical.

Note the third condition in the critical line: childProcess. It's set by the import alone, before anything is called:

// libs/core/ast-facts.js:348-362 (visitCall) // require("child_process") / import("fs") / require("fs") -> imports const importer = !object && (property === "require" || property === "import"); if (importer) { const source = stringLiteral(callArgs[0]); if (source) { imports.add(source); if (CHILD_PROCESS_MODULES.has(source)) { const evidence = `${property}(child_process)`; facts.operations.childProcess = true; calls.process.add(evidence); signal("child_process_import", evidence, node); } } return; } 

with CHILD_PROCESS_MODULES = new Set(["child_process", "node:child_process"]). A bare const { exec } = require("child_process") is critical by itself. This is what makes the "import in one MR, call in the next" trick fail on the first MR (scenario 1a in the tests).

The bug that made precision 30%

The first version decided processSpawn by the callee's name. exec, spawn, fork → spawning. Then it met real code:

• schemaAndHostRegExp.exec(url): RegExp.prototype.exec, probably one of the most-called methods in JavaScript,
• cluster.fork(),
• a locally defined function exec(script, stds) in pm2 that only wires up logging,
• bundled editor code full of regex .exec.

Seven out of ten blocks on public repositories were regular expressions and local helpers. If you've ever been paged because a regex was "executing", you know the feeling.

The fix was binding resolution: a call is process spawning when it's bound to a spawning module, not when it happens to share a name with one.

// libs/core/ast-facts.js:48-60 const SPAWN_FUNCTIONS = new Set([ "exec", "execSync", "execFile", "execFileSync", "spawn", "spawnSync", "fork", "execFileAsync" ]); // `regex.exec(input)` and `cluster.fork()` share their names with the // child_process API, so these two are only process spawning when the callee is // bound to a spawning module. The rest of SPAWN_FUNCTIONS is unambiguous enough // to flag on the bare name. const SPAWN_ONLY_WHEN_BOUND = new Set(["exec", "fork"]); const SPAWN_MODULES = new Set([ "child_process", "node:child_process", "cross-spawn", "execa", "shelljs", "zx" ]); 

// libs/core/ast-facts.js:326-341 function isSpawnCall({ object, property, raw }, host, spawnBindings) { if (!property) return false; // Anything imported from a spawning module spawns, whatever it is called. if (object ? spawnBindings.modules.has(host) : spawnBindings.functions.has(property)) return true; if (!SPAWN_FUNCTIONS.has(property)) return false; if (object) { // require("child_process").exec(...), childProcess.spawn(...) if (/child_process/.test(raw)) return true; return !SPAWN_ONLY_WHEN_BOUND.has(property) && /^(child_?process|cp)$/i.test(host); } return !SPAWN_ONLY_WHEN_BOUND.has(property); } 

spawnBindings is collected in a pre-pass over require / destructuring (including renames like const { exec: run } = require("child_process")) / import statements. After the fix the same 140 public PRs blocked 3 instead of 10, and the "really executes commands" recall check (a file whose text both imports a spawning module and calls one of its functions) stayed at 28/28. Again: those numbers come from the design-partner write-up, produced by a script outside the repo, and haven't been reproduced independently.

Step 3 — diff the facts, then the risk

// libs/mr-report/index.js:62-69 function riskDelta(before, after) { const direction = RISK_ORDER.indexOf(after) - RISK_ORDER.indexOf(before); return { before, after, direction: direction > 0 ? "raised" : direction < 0 ? "lowered" : "unchanged" }; } 

with RISK_ORDER = ["low", "medium", "high", "critical"]. Every fact family is diffed as a set (added = after \ before, removed = before \ after), never as a text diff. That's why a formatting-only change reports nothing: same sets on both sides, factCount: 0.

// libs/mr-report/index.js:76-91 (abridged) function diffFacts({ path, before = "", after = "" }) { const factsBefore = AstFacts.extract(before); const factsAfter = AstFacts.extract(after); const parsed = Boolean(factsBefore.ast && factsAfter.ast); const finding = { path, parsed, // routes, env, exports, imports: { added, removed } ... dangerous: { added: added(factsBefore.dangerous, factsAfter.dangerous), removed: added(factsAfter.dangerous, factsBefore.dangerous) }, operations: operationDelta(factsBefore.operations, factsAfter.operations), risk: riskDelta(AstFacts.riskOf(factsBefore), AstFacts.riskOf(factsAfter)) }; // ... } 

Notice that risk alone saturates: once a file is critical, direction can only be unchanged or lowered. That's exactly why the September audit found a hole, and why dangerous.added had to join the trigger.

Step 4 — the decision

// libs/mr-report/index.js:144-150 function gated(result, threshold) { return result.findings.filter(f => f.parsed && RISK_ORDER.indexOf(f.risk.after) >= threshold && (f.risk.direction === "raised" || f.dangerous.added.length > 0) ); } 

Read it as a truth table, at --gate critical:

Then acknowledgements:

// libs/mr-report/index.js:152-182 function acknowledgement(entry) { if (typeof entry === "string") return { reason: entry, facts: null }; if (entry && typeof entry === "object" Source：https://dev.to/jackymencz/security-regression-gate-sentinel-block-the-rise-not-the-presence-42ka
My favorites

Related posts

Document Review With Unsiloed and TypeSafe Jev

Originally published on Unsiloed AI . An extraction pipeline can read a value correctly and still pass bad data downstre

0dev.to Machine Learning

AmeriHealth Caritas Files 61-Worker WARN Notice in Orlando, Effective December 31, 2026

AmeriHealth Caritas has filed a WARN notice covering 61 workers at its Orlando, Florida location, with an effective date

0dev.to Career

Remote SWE Jobs Paying Up to $220k (Nov 2)

Here are the 8 highest paying fully remote software engineering roles this week. Solutions Engineer Onebrief Salary: $18

0dev.to Career

What are your goals for the week? #203

What are your goals for the week? What are you building this week? What do you want to learn? What events are you attend

0dev.to Career

Why Your Mind Goes Blank the Moment a Stranger Talks to You

Someone at the coffee line turns and asks how your day is going. A new coworker says hi in the kitchen. A friend of a fr

0dev.to Beginners

Google Wants Your Share Links Noindexed by Default: A 5-Step Checklist for Devs

Your share feature is working. But Google keeps indexing every link your users create. Here are the practical fixes to k

0dev.to Beginners