A Show HN went up yesterday for plimsoll, a Go daemon you run yourself that executes code an agent wrote. The sandbox part is ordinary. The interesting part is that every result carries the isolation tier the run actually got, as a value, and a request can demand a minimum tier before anything executes.
That second half is the rare bit. plimsoll's own docs say they read the current documentation of six hosted sandbox services in September 2026 (E2B, Modal, Daytona, Vercel Sandbox, Cloudflare and Northflank) and found that none of them returns the wall a run actually got, and none refuses a run that would land below a minimum the caller stated.
I checked two of those myself instead of taking the claim on faith.
Cloudflare's sandbox docs are explicit about the wall: "Each instance is a microVM with its own kernel and network, so no other workload shares it." That's a real statement about the boundary. It's also a statement about the product. Nothing on a run's result repeats it back per call, and there's no per-request floor to set.
Modal comes at it from the other side. Its Sandbox.create reference takes experimental_options, and passing {"vm_runtime": True} asks for a full virtual machine instead of the default gVisor layer. Nothing in what comes back tells you which of the two served a particular call. You asked. You don't get told which one you got.
Four tiers, as a field on the result
plimsoll's tiers are process, container, kernel, vm, weakest first. The tier is a value on every result rather than a sentence in a datasheet. A request sets MinimumIsolation, and if the daemon can't meet it you get ErrInsufficientIsolation and nothing ran.
That last clause is the part I care about more than the tier names. "Nothing ran" and "may have run" are different facts, and a plain error code can't tell them apart. InvalidArgument is usually a request that failed validation, but a worker can also refuse after its container started. A capacity refusal can come from the daemon's limiter or from the provider. So refusals raised before dispatch are marked with a reason (request, permission, protocol, unsupported, isolation, environment, capacity), and a refusal without the mark reads as "may have executed." That's the conservative default: a code path that forgot to mark its error can never be mistaken for a safe retry.
There's a detail under that which is easy to miss. The mark only counts if the answer carries your request ID back. Otherwise a confused proxy serving one request's answer for another would make a call that ran read as "nothing ran," and the retry runs it twice. The client sends a fresh 32-hex-digit ID with each request and the daemon echoes it.
The record is the part you can hand to someone else
Every answered run carries SHA-256 digests of the request as sent and the result as returned, plus the provider, the tier, the image identity and the timings. The daemon holds no signing key, on purpose, because it executes hostile code and one engine escape would otherwise be a key away from forging every record. A harness outside the daemon recomputes the digests from its own copy and signs the ones that match. For a deterministic workload you can replay the stored request and compare digests.
Their run report for the cart-pole example shows what that buys you. An agent-written controller, run twice, same fingerprint 63cfd676924a3896127d60c97333677f3b3c804dd764dba001121f6c234a937f. The agent's first draft, same structure with two gains of the wrong sign, different fingerprint, pole down at 5.96 s. The same run on five different providers, four of them different backends, hashed to that same value.
The page also says plainly what it doesn't show. The controller can read the plant and the runner, since they sit on the same read-only root. A fingerprint is identity, not a score, and recompiling the plant moves every fingerprint on the page.
The credential story is where this gets practical
Most of what I read about agent sandboxes stops at containment. The capability grants doc goes further into the thing that actually bites teams: the agent's code needs to call your API, and the usual fix is to hand the sandbox a token.
A grant here is permission for one run to call listed routes of one API. Without a grant, the run has no network at all. Docker runs keep --network none and send calls over a per-run Unix socket. The code calls a small client plimsoll places in the sandbox, and the broker outside makes the real call, checks the caller and the exact route, then attaches a credential minted per run. The token never enters the sandbox, and code that skips the client and calls out by hand gets nowhere because the broker, not the client, enforces the rules.
The route matching is where they've clearly been bitten. A path is accepted only if it's already decoded, in simplest form, and byte-identical to an approved route. Queries, .. traversal and percent-encoding tricks are refused before anything is sent. So are ; parameters inside a path segment, because Tomcat and Spring read /orgs/..;/repos/x as /repos/x. A * matches one segment of unreserved characters only. On an API following Google's custom-method convention, POST /v1/items/a:setIamPolicy is a different operation than POST /v1/items/a, and on OData POST /odata/$batch carries a batch of other requests, so a grant of POST /v1/items/* doesn't reach either.
There's a smaller piece I liked more than I expected. Three things describe the same set of routes (the allow list the broker enforces, the JavaScript client the agent calls, and the tool description the model reads), and they drift apart whenever a human keeps them in sync. plimsoll-specgen builds all three from one OpenAPI document, fails closed on ambiguous input, and warns instead of silently dropping what it can't express. It also emits every operation in the document as allowed, because a spec describes what an API has, not what an agent should reach. Pasting that unedited grants the whole API. The doc says so out loud.
Why I'm writing about this one
The docs say the tier is configuration plus startup smoke tests, and that it is not attestation, meaning cryptographic proof from the hardware of what software is actually running. They say the project is pre-1.0, has no outside security audit, and the bus factor is one. gVisor is Linux only, so a Mac gets the container tier. The wasm provider runs inside the daemon's own process and the docs say not to point it at hostile code. Those are the sentences that make the rest of it checkable, and they're the reason I'd rather read this than another launch post.
For my beat it's the same shape as every accuracy claim I've torn apart. "Isolated" is a claim like "96% accurate." The version you can falsify has a field on the result, a floor on the request, and a record somebody outside the box can recompute. This is also where the patch-versus-execution gap lands. If review reads the diff and never the run, then the execution record is the thing you have to trust, and most of the options today hand you a sentence instead.
Practical version, since I'm not asking anyone to take my word for it either: next time you're choosing where an agent's code runs, ask for one result object from a real run. If it doesn't state the boundary that run got, you're still reading marketing copy.
