The worst thing a tool can return to an agent is not an error. It is the word "failed".

A person reading "failed" understands that something was attempted and did not work. An agent reading "failed" has no way to distinguish three completely different situations that all arrive looking like the same sentence:

  1. The call never reached the service. Nothing happened. Retrying is free.
  2. The call reached the service, the service refused, and nothing changed. Retrying will fail again, and should be fixed instead.
  3. The call reached the service, the effect landed, and the response was lost. Retrying repeats the action.

Only the first wants a retry. The third is why a customer gets two identical messages, a booking gets charged twice, or a support ticket gets filed in triplicate by an assistant that was being careful.

This is the part of agent engineering that looks like plumbing and is actually the whole safety story: a tool call has to fail in a way the caller can act on. After a year of building long-running assistants on top of brittle real-world tools — browsers, payment APIs, mail, forms behind someone else's login — the design I keep coming back to is small. It is a return contract, an idempotency key, a deadline budget, and one rule about what to do when the answer is unknown.

Error strings are for humans; outcomes are for callers

Most tool wrappers today return something like:

{"ok": False, "error": "HTTP 500 from /orders"}

Then the agent, being a language model, reads the string and improvises. Sometimes it retries. Sometimes it apologises to the user and gives up. Sometimes it invents a reason: "the service is down" when the service returned a validation error, or "your order is confirmed" because it saw the substring "order". The variability is not a model defect; it is what happens when the contract leaves the decision in prose.

The fix is to make the outcome a small closed vocabulary, and to require every tool to fill it in:

{ "outcome": "retryable", # ok | retryable | permanent | unknown "effect_applied": "no", # yes | no | unknown "attempt_id": "ord-7741-a2", # ours, stable across retries "retry_after_s": 30, # from the service if it said; else null "deadline_left_s": 42, "message": "503 from /orders after 8.1s; no response body", }

Four values, and each has one meaning:

  • ok — the call completed and the effect (if any) landed. The agent may continue.
  • retryable — the call did not complete for a reason that is likely to change. Safe to retry only under the idempotency rule below.
  • permanent — retrying the same call will produce the same answer. The agent must change something: ask a human, correct a field, choose another route.
  • unknown — we cannot tell whether the effect landed. This is not an error state; it is a state that has to be resolved by reading the world, and the agent must not be allowed to paper over it.

effect_applied is separate on purpose. A 200 that says "queued" has not applied anything yet. A timeout on a POST may have applied everything. A bulk edit may have applied 39 of 40 rows.

The rule I enforce on every wrapper: never return a bare boolean, and never let the model be the component that maps an exception to a retry decision. That mapping is deterministic and belongs in code, where it can be reviewed and tested. The model's job is to decide what to do with permanent or unknown — escalate, explain, choose another path — which is a judgement, not a lookup.

Idempotency is a property of the intent, not the payload

Retries are safe exactly to the degree that the receiver can recognise a repeat. If you cannot change the receiver, you carry the recognition yourself with a key.

The mistake I made first was to hash the request body. It seems principled — same bytes, same key — until the payload contains a timestamp, a nonce, or a freshly fetched price, and the "same" retry produces a different body and therefore a different key. Then the key provides no protection at all, at exactly the moment it is needed.

The key has to be derived from the intent: who is acting, on what object, to do what, in which task.

def idem_key(task_id, actor, verb, target, slot=None): raw = f"{task_id}|{actor}|{verb}|{target}|{slot or ''}" return hashlib.sha256(raw.encode()).hexdigest()[:32]

task_id is the durable identity of the work (the order the user asked for), not the run, not the retry loop. slot exists for genuinely repeated acts in one task — the second and third reminder message — so that "send notice #2" is a different intent from "send notice #1" and a retry of each is still a repeat.

With the key in hand, the retry semantics become readable:

Kind of call Duplicate is Policy
Read (price, schedule, status) harmless retry freely with backoff
Create with a key deduplicated by receiver retry with the same key, never a new one
Send (message, mail, notice) visible to a human at-most-once: key + probe before second attempt
Money movement expensive to reverse at-most-once, and ask before the second attempt
Destructive (delete, cancel) possibly irreversible no automatic retry; resolve the state first

That last row is where a lot of "helpful" agents do damage. A delete that timed out is not a delete that failed. If the design has no way to ask "does it still exist?", then the correct behaviour after a timeout is to stop and report — not to try again and not to declare success.

Deadlines belong to the task, not the attempt

A retry loop with a per-attempt timeout silently multiplies the promise you made. Tell a user "I'll check and come back in a minute", then retry a 30-second call three times, and the minute is gone before the first attempt has even given up twice. Worse, the abandoned attempts are still running in the background, doing whatever they were doing.

Two things fix this cheaply.

Carry one budget. The task gets a wall-clock budget. Each call receives the remaining budget, and a wrapper refuses to start an attempt with less time left than the operation's known floor. When the budget is out, the return is retryable with deadline_left_s: 0 — which the caller must treat as "stop and report", not "try once more".

Cancel, don't orphan. A timed-out call is a request that may still complete. If the tool supports cancellation, use it; if it doesn't, the attempt must be recorded as unknown rather than failed, because the effect may yet land. This is the same distinction as before, applied to time instead of to response codes.

A boolean is a lie for bulk work

Anything that touches many rows at once — a spreadsheet range, forty product listings, a batch of appointments — has four possible results, not two: all applied, none applied, some applied, and unknown. Returning true/false throws away the only information that matters for recovery.

{ "outcome": "permanent", "effect_applied": "partial", "items": [ {"id": "row_12", "applied": "yes"}, {"id": "row_13", "applied": "no", "reason": "merged cell"}, {"id": "row_14", "applied": "unknown"} ], "message": "38 of 40 rows written; 1 blocked, 1 unresolved" }

With per-item outcomes, the retry is scoped to three rows instead of forty, and the report to the user can be true ("38 done, 2 not") instead of flattering ("done"). The same shape works for paginated reads that fail halfway: a partial list is usable, an empty list is a lie, and a truncated list presented as complete is worse than either.

When the answer is unknown, read before you write

The single most useful line of policy in my tool layer is not about retries at all:

A retry is a guess. A probe is a measurement.

Before any second attempt on a call with a side effect, the agent must attempt a read that would reveal whether the first attempt landed: fetch the order by its idempotency key, list the last three messages in the thread, read the sheet range back, check the booking slot. If the probe says the effect is present, the outcome is ok and the work continues. If the probe says it is absent, the retry is safe. If the probe itself fails, the outcome stays unknown, and unknown escalates to a human.

The ordering matters: probe → decide → act. Agents that act → discover → apologise produce the duplicate messages that make people stop trusting automation. And the escalation path has to exist before it is needed; an agent that can only succeed or silently fail will eventually do the second while claiming the first.

None of this requires a larger model. It requires the tool layer to stop lying about what happened.

Six failures worth injecting on purpose

Every design above is cheap to test, and untested retry logic is a rumour. The fixture list I use:

  1. Kill mid-call. The service applies the effect, then the connection drops. Expect: unknown, then a probe, then ok without a second effect.
  2. Duplicate delivery. Send the same idempotency key twice. Expect: one effect, two ok returns.
  3. 200 with an error body. The transport succeeded, the operation did not. Expect: permanent or retryable by body, never ok on status alone.
  4. Slow success. The effect lands at 29 seconds on a 30-second budget. Expect: no orphaned retry racing the original.
  5. Partial batch. One item permanently invalid. Expect: per-item outcomes and a scoped retry of the remainder.
  6. Rate limit with Retry-After. Expect: honour the header, and do not hammer while guessing.

If a wrapper cannot pass these six against a fake, it will not pass them against a real bank, a real marketplace, or a real person's inbox.

The logging that makes a post-mortem possible

One line per attempt, written before the call and updated after it, is enough:

{"task":"ord-7741","call":"create_order","intent_key":"9c1f…","attempt":2, "started":"05:12:04","budget_left_s":42,"outcome":"unknown","probe":"found"}

Written before the call, it survives a crash — the same reason databases write the intent to a log before acting. Without it, the post-mortem reads "the agent retried and something happened twice", which is not an explanation, it is a confession.

What I'd tell someone starting today

Treat the tool return value as an interface you will have to reason about at 3 a.m., because you will. Four outcomes, an explicit effect flag, an intent-derived key, one deadline per task, per-item results for bulk work, and a probe before every second attempt. That is the entire design, and it fits in a page.

The interesting part is not the retry. It is that this contract is what lets an agent be wrong safely: it can stop, say what it does not know, and leave the world in a state a person can inspect. An assistant that never does anything twice and never claims more than it can see is worth far more than one that is fast.

If you want the concrete version of this on a phone-side assistant — what it does at a CAPTCHA, how a daily usage limit behaves, what a reset ticket actually restores — I keep two public notes:

Disclosure: I use Pickle, a phone-side AI assistant app, and I am not its vendor. The links above are mine; if a sign-up completes through the invite link, both sides receive reset tickets, and the app's own notices govern any benefit. This is not a paid placement. The tool contract described here is the one I use in my own agent work, written from failures I paid for.