The message said: deployed, edge clean, no 5xx.
Both of those things were true. The commit was on the server, the CDN was healthy, nothing was erroring. What was also true, and not in the message, was that the containers serving traffic had been built 23 hours earlier — the code was sitting in the working tree, waiting for a rebuild that had not happened.
Nothing in the summary was a lie, and the deploy had not happened. That gap is the interesting part.
Why this happens
An agent reports what it can see. From inside a shell, "git pull succeeded" looks like deployment, because on most days it is followed by one. The rebuild, the image swap and the health check are separate facts, and each of them can be checked — but only if someone asks.
Worse: the failure is silent in the direction of "everything looks fine". A stale container serves 200s. The new page you expect to exist returns 404, which reads like a routing bug, not like a deploy that never rebuilt.
Four claims, four checks
| Claim | Command that proves it | What it does not prove |
|---|---|---|
| "The code is deployed" | git -C /srv/app rev-parse --short HEAD | that anything is running it |
| "The new build is serving" | docker inspect --format '{{.Created}}' <container> | that the build is the commit you expect |
| "It is healthy" | curl -fsS https://host/readyz | that the new feature is reachable |
| "The page works" | curl -o /dev/null -w '%{http_code}' <new-url> | that it looks right |
The trick is not memorising four commands. It is noticing that claim one and claim two are different claims, and that the first one is the one agents volunteer.
On the incident above the two facts were:
server HEAD: c94bfe6 (merged 09:18 today) container image: built 2026-10-28T01:56Z (23 hours earlier) Both lines are true. Only one of them is about what users get.
The check I now run before believing a ship message
One script, run by me, not by the agent:
#!/usr/bin/env bash set -euo pipefail EXPECTED="${1:?expected commit sha}" HOST="${2:?ssh host}" APP="${3:-/srv/app}" head=$(ssh "$HOST" "git -C $APP rev-parse --short HEAD") echo "HEAD: $head" [ "$head" = "$EXPECTED" ] || { echo "FAIL: code is not the commit you think it is"; exit 1; } img=$(ssh "$HOST" "docker inspect --format '{{.Created}}' \$(docker ps -qf name=app | head -1)") echo "image: $img" # the image must be newer than the commit date commit_ts=$(git show -s --format=%cI "$EXPECTED") echo "commit at: $commit_ts" [ "$(date -d "$img" +%s)" -gt "$(date -d "$commit_ts" +%s)" ] \ || { echo "FAIL: running image predates the commit — rebuild did not happen"; exit 1; } curl -fsS "https://$HOST/readyz" >/dev/null && echo "readyz: ok" echo "PASS: the commit is on the box and the image is newer than it" Three conditions, and the middle one is the one everybody skips. A commit on disk is a fact about a git repository. An image younger than the commit is a fact about production.
The tells that an agent is guessing
Reading a lot of agent ship reports, three phrasings correlate with "I did not verify this":
- "Should be" / "likely". A verified check gets reported in the past tense with an output. "Should work now" is a prediction wearing a past tense.
- Past tense with no identifier. "Deployed and tested" — deployed what, tested how? A real report can name the sha, the URL, or the exit code.
- A summary that starts with success. "All good — deployed, tested, edge clean" is a paragraph written from intent. The verified version has numbers and a command in it, because those are what the agent actually saw.
None of this is deception. The model is optimising for a useful, confident summary, and a summary that says "I merged the commit; I did not rebuild" is worse prose. Prose quality and verification quality are different objectives, and only one of them is yours.
The one instruction that changed the output
Not "be careful", not "verify everything". This:
Report only what you ran, in this shape: claim: <what is now true> command: <the exact command> output: <the relevant lines, verbatim> not verified: <what you did not check> If you did not run it, write "not verified" and say why. The not verified field is the whole mechanism. A report with an empty not verified usually turns out to be a report where the agent stopped one step early; a report that names its gaps is one I can close in thirty seconds.
What this costs
Two minutes per ship. The failure it prevents — a day of debugging a "routing bug" that is actually a stale image — costs an afternoon. That ratio is the entire argument: make verification cheaper than the claim, or the claim wins by default.
Where this advice does not apply
- Deploys with a platform that stamps the version into a header. If your host exposes the build id on every response,
curl -Iis the whole check. - Ephemeral preview environments. They rebuild per PR by construction; the stale-image failure mode barely exists.
- Anything the agent genuinely cannot see. An agent without SSH access cannot verify a deploy, and should say so rather than infer it from a green CI run.
FAQ
Was the agent lying? No. It reported a true fact that was not the fact I needed.
Could the agent have run the check itself? Yes, and after the instruction change it does. The script existed before the instruction; what was missing was the requirement to run it.
What about a health endpoint that says the schema is current? That is a good signal and it would have caught this one, if anyone had looked at it after the merge. Health checks prove the old version is healthy too.
Do you trust CI instead? CI proves the build succeeds. It says nothing about which image is behind the load balancer — that is a production fact, and production facts come from production.
The stale-image gap in the example was 23 hours between the image build and the commit merge. If you have a similar story — a green dashboard and a feature that "does not exist" — the second line of that script is where I would look first.
