From prose to assertions: making 'done' checkable when an AI agent writes the code

The hardest part of an AI coding loop is not getting code written. It is the moment after the agent says "done".

When one agent writes the code and another agent reviews it, you have not removed the trust problem — you have just moved it. The reviewer reads the diff, reads the tests, reads the ticket, and produces a verdict. If the ticket says "spacing should be 12vp" and the screenshot looks about right, the verdict is "pass". Two auditors can look at the same delivery and disagree, and nothing in the tooling forces that disagreement to resolve.

That is the gap I tried to close in DwGuard-Dsh-Hmos, a plugin for DeepSeek Harness that orchestrates HarmonyOS app delivery: DSH freezes a four-dimension acceptance pack (function / aesthetic / performance / accessibility), runs an adversarial pre-review before dispatch, hands the work to DevEco Code in Goal mode, then audits the delivery with independently gathered evidence.

The change I want to write about is small in surface area and large in consequence: acceptance criteria can carry a machine-checkable assertion, and the audit gate recomputes the verdict instead of trusting prose.

Three failures of prose criteria

My criteria used to be a sentence plus a verification method:

statement: 冷启动到首帧 ≤ 800ms (cold start to first frame ≤ 800ms) verify: Profiler 启动分析毫秒数 (milliseconds from the profiler) 

And an audit check was a result plus free-text evidence:

result: pass evidence: 3 runs, 700/720/760ms, screenshot artifacts/perf.png 

Three problems showed up immediately:

1. Criteria are not executable. "12vp" in a sentence means nothing to a machine. Two auditors reading the same criterion can reach opposite conclusions and both feel right.
2. Evidence is not reviewable. evidence is free text. No tool can tell whether "1200ms" satisfies "≤ 800ms" without a human eyeballing it.
3. The report is not consumable. The verdict lives inside markdown. CI, regression scripts, and the next agent cannot read a field; they re-parse prose.

So the audit was still a person voting, with extra steps.

The assertion: criteria that can fail by themselves

The fix is an optional assertion on each criterion:

{ "id": "P-01", "dimension": "performance", "statement": "Cold start to first frame ≤ 800ms", "verify": "ms reported by the startup profiler", "weight": "must", "ruleId": "DW-PERF-01", "assertion": { "matcher": "lte", "expected": 800, "unit": "ms", "source": "startup profiler" } } 

Eight matchers cover the criteria I actually write:

The audit side then must report an actual value next to the verdict:

{ "criterionId": "P-01", "result": "pass", "evidence": "three cold starts: 700/720/760ms; screenshot artifacts/perf.png", "actual": "720ms" } 

One detail matters more than it looks: numeric matchers take the first number inside actual. That means measured values can keep their units — "12vp" parses as 12, "720ms" as 720, "4.9:1" as 4.9. The assertion compares numbers; the report stays human-readable.

The rules that make it a gate instead of a suggestion

An optional field that nobody enforces is decoration. So the gate has opinions:

Plus a nudge at authoring time: a must criterion whose statement contains numbers but carries no assertion gets W_CRITERION_NO_ASSERTION, because that is exactly where two auditors will disagree later.

The subtle rule is the third one. Saying "fail" when the number passes is not always wrong — a screenshot can show a layout bug that a timing assertion cannot see. But the gate should make that mismatch visible instead of silently accepting whatever the reviewer wrote.

What it changes in the artefacts

Because the verdict is now recomputed, the reports gain a column each:

• spec.md and the Goal brief gain an assertion column, so the executor knows which criteria will be measured rather than judged;
• audit-<n>.md gains assertion / actual / verdict columns;
• the acceptance report's evidence index gains assertion / actual;
• rework orders gain a gate problems section — previously a contradicted assertion produced a refusal with nowhere to record why.

And audit-<n>.json now contains checks[].assertionVerdict, which means CI can fail a build on verdict: fail without parsing markdown. That was the whole point of making the report consumable.

What stays human

I want to be careful here, because "machine-checkable acceptance" is easy to oversell.

• Assertions only cover what is measurable. "The empty state feels calm" cannot be asserted. It stays a judgement call, and I still write those criteria — they just do not claim to be objective.
• The assertion does not collect evidence. Somebody still gathers the actual value: a screenshot with a known device, a dumpsys reading, a layout tree. Evidence files get copied, hashed, and recorded, but the assertion only decides pass/fail from the number.
• Thresholds are taste wearing a number. "≤ 800ms" is a decision I made, not a law of physics. Turning it into an assertion makes the decision visible and debatable, which is an improvement, but the number itself is still a human judgement.
• Values can be gamed. If the process that produces actual is the same process being audited, an agent can satisfy the letter. That is why the audit playbook insists on externally observable sources — hidumper render stats and hitrace for performance, layout bounds for spacing — and treats in-app timing instrumentation as auxiliary evidence, never the sole basis.

The honest result

The change shipped as 0.5.0 and the suite went from 49 to 55 tests, all green (six new ones specifically about assertions: accepting valid, rejecting invalid, recomputing, refusing contradictions, refusing a pass without actual, and keeping old packs unchanged).

The real payoff is not "the AI is now trustworthy". It is that disagreement became resolvable. When two auditors disagree now, there is a file with a number and a rule that says whether it passes. Either the number is wrong, or the threshold is wrong, or the measurement is wrong — and all three are fixable. Before, the disagreement was just two paragraphs.

If you are building an agent loop where one model checks another, the question to ask is not "is the reviewer good". It is: can the reviewer's verdict be recomputed by something that did not write it?

Plugin: https://github.com/Dogwind221/dwguard-dsh-hmos · npm: https://www.npmjs.com/package/dwguard-dsh-hmos · listed in awesome-dsh-plugin.