Agent safety discussions often collapse three different questions:
- Role preflight — Is this task inside the declared authority of this role?
- Diff verification — Did the actual changes satisfy acceptance criteria without weakening tests?
- Runtime authorization — Is this exact side effect allowed now?
They catch different failures.
A role contract can reject a read-only reviewer being assigned an implementation task. It cannot prove the runtime obeyed the contract.
A diff check can catch an ordinary write that disables a test. It cannot decide whether the writer was authorized.
A runtime broker can stop a send or deploy. It may still permit the wrong actor if the policy never modeled the role.
I'm maintaining Agent Role Contracts (ARC), an open-source, offline preflight for explicit role/task declarations.
Important limit: current stable ARC evaluates supplied declarations. It does not observe runtime behavior and does not prove that a platform enforced anything.
I want to turn this into public comparison receipts instead of another abstract framework. For each volunteered workflow, the evaluation is:
- use one sanitized completed task
- record the incumbent controls
- run a baseline PASS
- introduce one real contradiction
- repair and rerun
- publish setup time, false positives, missed controls, and a keep/remove decision
The evaluation template is public.
Three fixtures I especially want:
- the tools stay the same, but a role crosses from read to send
- an in-scope write weakens a test or acceptance condition
- a read-only reviewer is asked to implement
If you have a workflow, reply with one completed task or repository and the controls you already use. I'll do the mapping and publish the receipt—including where ARC adds nothing.
Which layer is your real stop condition today, and what failure does it still miss?
