Notifio is two codebases: a Next.js site that sells and supports the product, and an Electron app that does the work on your own machine. I measured the website's test suite a few days ago: five files on node:test, mostly about money and metadata.
This post is the other half, and the other half does not have a test suite. It has three scripts.
$ ls app/test-*.ts test-backoff.ts test-baseline.ts test-scraper.ts $ wc -l app/test-*.ts 118 test-backoff.ts 88 test-baseline.ts 258 test-scraper.ts 464 total No __tests__ directory, no framework in devDependencies, no config file, and no runner except tsx. They sit in the package root next to package.json, which is exactly where a file lands when somebody needed to check something that afternoon.
What is in them
file checks local modules imported deterministic -------------------------------------------------------------------- test-baseline.ts 15 1 yes test-backoff.ts 21 1 yes test-scraper.ts 17 2 no (browser + network) -- 53 Fifty-three assertions over four of the app's 28 top-level modules, counted as calls to the check helper in each file. The two deterministic scripts cover pure bookkeeping logic: one is about when a search starts over instead of being compared, which is the trickiest state rule in the product, and the other covers the timing bookkeeping next to it. Both run in milliseconds and touch nothing.
The third is a smoke test. It launches a real browser against a real page and checks that what comes back is usable. I am not going to detail what it asserts, because that part is the product rather than the engineering around it, but its shape matters here: it is not a unit test, it cannot pass offline, and a failure in it is at least as likely to be news about the outside world as news about our code.
The harness is four lines, written twice
let failures = 0; function check(label: string, ok: boolean, detail = ''): void { console.log(` ${ok ? 'ok ' : 'FAIL'} ${label}${detail ? ` ${detail}` : ''}`); if (!ok) failures++; } That block appears identically in two of the three files, and the third has a variant of it with a different name and the counter scoped inside a function. Then each script ends the same way:
console.log(failures === 0 ? '\nAll checks passed\n' : `\n${failures} check(s) failed\n`); process.exitCode = failures === 0 ? 0 : 1; process.exitCode rather than process.exit(1), which is the detail I am most pleased about in the whole suite. process.exit terminates immediately and can truncate buffered stdout, so a failing script can exit non-zero while losing the line that says which check failed. Setting the code and letting the process end naturally keeps the output and still fails the shell.
It is a fifteen-line test framework with a readable output format, and for the two deterministic files it genuinely is enough. The checks read like sentences:
check('a fresh start re-bases a search left over from a previous run', checkSearch(afterRestart, KAMERNET)); check('the next check of that search is a real comparison', !checkSearch(afterRestart, KAMERNET)); That is a specification of a behaviour nobody can hold in their head, written in the words we use to talk about it, and no framework would have improved those two lines.
What pnpm test actually does
"test": "pnpm test:baseline && pnpm test:backoff && pnpm test:scraper", "test:baseline": "tsx test-baseline.ts", "test:backoff": "tsx test-backoff.ts", "test:scraper": "tsx test-scraper.ts", Two consequences of that &&, and neither was chosen.
The first is that a failure in the earliest script means the other two never run, so the output tells you about one broken thing at a time. For a suite of three that is survivable, and it is the opposite of what you want from a test command: the point of running everything is to learn everything.
The second is that pnpm test cannot run without a browser and a network, because the third script needs both. So the command named after the ordinary developer habit is the one command you cannot use on a train, and the two scripts that would run anywhere are hidden behind the one that will not. Anybody who tries pnpm test with no connection concludes the suite is broken.
$ ls -a ../.github 2>/dev/null || echo "no .github anywhere in the repo" no .github anywhere in the repo And there is no CI, which the website half does not have either. These 55 checks have only ever run on this laptop, on purpose, because a smoke test that drives a browser against somebody else's page is not something I want firing from a data centre on every push.
The 24 modules with no checks at all
Four modules of 28 are imported by a test. The 24 that are not include every module that writes to your disk: the config, the settings, the find history, the reply ledger, the per-search runtime state. I have written about the caps on all of that and about which of those stores is allowed to survive a restart, and not one of those rules is enforced by anything except the code that implements it.
That is the real finding, and it is not "write more tests". It is that the two modules which got tests are the two that are pure functions, and everything else in this half of the product is a long-running process that reads disk, drives a browser and talks to a server. The testable surface and the tested surface are the same set, which means the suite grew to exactly the size that was easy and then stopped.
What I am changing
Three things, none of them a framework.
The two deterministic scripts move to node:test, which the website half already uses and which ships inside Node, so this costs zero dependencies. It buys parallel execution, a real exit code per file, and a --test-reporter that other tools can read. The 36 deterministic checks keep their sentences, because it('a fresh start re-bases a search left over from a previous run') is the same string in a better socket.
The smoke test loses the name test. It becomes pnpm smoke, documented as a thing you run by hand when something smells wrong, and pnpm test becomes a command that runs in 200 milliseconds with the wifi off. A test command you can trust to be fast and offline is a test command people actually run.
And the disk modules get the first checks they have ever had, starting with the ones whose rules I have already written down in public. A rule that is interesting enough to blog about is interesting enough to be a line in a file that fails.
You can see the half of the product these scripts cover by grabbing the app from the download page, and the help page describes the behaviour those 15 baseline checks are protecting: the first check after a restart does not email you yesterday's listings.
![Multi Robot and Swarm Physical AI — Ch.00: Introduction [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F3kjs92ynywy72p5gq7o2.png)

