Most writing about assistants stops at the model. If you actually operate one for a few weeks, the model turns out to be the least interesting component. The thing that decides whether the assistant is useful is a much older shape of software: a schedule, a check, a run log — and a model that is only allowed to start when the check says so.

A chat window answers when spoken to. An assistant you delegate to wakes up on its own, looks at something, and usually says nothing. That "usually says nothing" is the hard part, and it is not a model property. It is an architecture property.

Here is the design I ended up with, the failure modes that forced each part of it, and the numbers that tell you whether it is working.

Three artefacts, one model

Split the thing into three files, because they fail independently:

  • A schedule — when this is due. A cron-like expression, a timezone, and an end date.
  • A check — a program that answers one yes/no question before anything expensive runs: has anything changed that this action cares about?
  • A run log — what the run saw, what it decided, what it did, and the evidence for each claim.

The model runs only after the check exits 0. Everything before that is cheap, deterministic, and debuggable without reading a transcript.

The check: exit codes are the entire API

The tempting design is to wake the model every fifteen minutes and let it decide whether anything changed. It fails for two reasons that compound.

First, cost. If a poll costs a model call, you cannot poll at the cadence the world changes. A page that updates once a day does not need to be read 96 times.

Second, and worse: nondeterminism in the decision itself. The same unchanged page will sometimes read as "changed" and sometimes as "unchanged", because that judgement is a generation, not an assertion. You end up with an assistant that reports the same parcel status twice on Tuesday and misses a delivery on Thursday.

So the check is a small script, and its whole interface is an exit code:

  • 0 — wake the model.
  • 1 — nothing new, stay asleep.
  • 2 and above — the check itself failed. This is a different kind of silence, and it is the only one that deserves an alert.

That distinction is worth more than it looks. 1 is the normal case and should be invisible: no log line that requires reading, no notification, no tokens. 2 means you have lost the ability to see the thing you claim to be watching — an expired session, a changed HTML shape, a carrier API that answers 500 — and if you fold it into "nothing new" you will discover it weeks later, in the middle of the thing it was supposed to catch.

The check must also be boring on purpose. What it reads should be a fact with a tiny surface: an HTTP status, a hash of one element rather than a whole page, a count of rows, a status string from an API, a file's mtime, a search-results position. And it must have a timeout short enough that a hung socket cannot outlive the scheduled run.

A parcel tracker makes the shape obvious:

check(): status = GET carrier tracking page # 10s timeout if status.http != 200: exit 2 # cannot see, not "nothing new" if login wall detected: exit 3 # needs the human; alert once, then back off h = hash(status.last_event + status.timestamp) if h == stored_h: exit 1 store(h); print(status.last_event); exit 0

Now the model can be woken with a fact instead of a question. Its brief starts with "the status changed from X to Y at 14:02", so it is drafting a message, not rediscovering the world.

The run: brief, intent, effect, receipt

A wake should hand the model three things: what was due, what the check saw, and what the person originally asked for — verbatim. Without the third, a run that fires three days late confidently does the wrong thing in the right format.

Then the run follows a fixed order, and the order is what makes it safe:

  1. Write the intent first — a record keyed by (action, due_instant, occurrence) saying "about to do this".
  2. Perform the effect.
  3. Write the receipt with the identifiers and evidence: confirmation number, order id, the URL it acted on, the screenshot path, the timestamp.

If the process dies between 2 and 3, the next run sees the intent key and does not repeat the effect. That is the whole idempotency story, and it needs to be in place before the first run that spends money or sends a message, because "the run died but the post went out" is not a rare coin flip — it is an ordinary Tuesday on a flaky network.

Two rules I keep having to re-learn:

  • A 200 is not proof. Outside state persists after a response is lost. The receipt has to come from reading the account back, not from the call returning.
  • Never restart a run that already committed. If a run got half way — submitted, then failed to fetch the confirmation — the correct next step is to read the state and report the partial result, naming what completed and what did not. Re-running is how you get two bookings, two posts, two charges.

The window: due time is not delivery time

A schedule has two times: when the action is due, and when it is allowed to speak. Conflating them is how a background assistant becomes something people mute.

Quiet hours constrain output, not checking. The check can run at 03:40 and find exactly what it needed to find; the message queues until 08:00. What matters is that the queue is explicit, with a rule for what may break it:

  • Urgent — time-critical and cheap to act on (a booking slot expiring in twenty minutes). May break quiet hours; must be rare enough that nobody learns to ignore it.
  • Normal — queued to the next allowed window, batched into one message if several are due.
  • Silent — log only. Most finding falls here, and saying so out loud at design time stops the log from being read as a to-do list.

Timezones need to be stored, not inferred. "8am" belongs to a place, and a run that resolves today from the host clock at the wrong offset will happily fire an hour early across a DST boundary. Store the zone with the schedule, resolve both the due time and the message window in it at every run, and treat the phrase "today" as ambiguous until it has been given a place.

A per-action and per-day message budget is the cheapest insurance here. The failure mode of a proactive agent is noise, and a notification cannot be un-sent. When the budget is hit, compress: one digest line per action beats five separate pings.

The log: the only defence against "it said it did it"

The log is not diagnostics. It is the product. Every run writes:

due_at, started_at, finished_at check: result (0/1/2/3), what it read, hash before/after decision: act | queue | skip, and why, in one line actions: attempted list with outcome per item evidence: ids, urls, screenshot paths model: name, tokens, cost

And it writes the no-op runs too, at least as a counter. A scheduled thing you cannot see run is a scheduled thing that has died. If silent runs leave no trace, "no news" and "broken for nine days" look identical from the outside — and they look identical from the inside too, which is the actual problem.

That is also why the noise floor has to be low: a log you can read in ten seconds a week is a log you will read. A log that fires a notification per check is one you filter, and once you filter it, it stops existing.

Expiry and the dead-man switch

Every scheduled action needs a way to die. Two numbers:

  • Consecutive failures — after N checks exit 2, stop running and say what broke. A check that has been failing for a day is not a check, it is a rumour.
  • Relevance horizon — the trip is over, the parcel arrived, the trial converted. An action past its purpose keeps waking, keeps finding nothing, and quietly costs money. Zombie jobs are the most common background failure I have seen, and the least visible, because a zombie produces no errors at all.

Stopping should be a logged, explained state, not an absence. "Stopped on 12 November: the tracking number was delivered on the 11th" is a result. Silence is not.

The boundary: what runs unattended

Architecture decides how well a background agent works. The boundary decides whether it should exist.

The line I use is about what lands on the person and stays there. A lookup, a draft, a diff, a check — those cost the assistant's time and can be thrown away. Money, a booking, an account, who gets to see them — those land on the person and cannot be given back. A notification is on that second list too: it spends someone's attention, at an hour they did not choose.

So unattended work is read-only and reversible; commitments get prepared until one yes is missing. And a message sent in someone's name — a call, an email that speaks for them — is not a lookup, because the person on the other end now remembers them, not the assistant.

That split also makes the design cheaper: the model is asked to produce the smallest correct artefact (a fact, a draft, a one-tap decision), not to behave well in a space where being wrong is expensive and unfixable.

Failure table

Symptom Cause Fix
Tokens burn, nothing useful check always exits 0 make the predicate concrete and hashable
Wakes are inconsistent model decides "changed" exit code decides; model only acts
Silent death, discovered late errors folded into "nothing new" exit 2+ is an error class; alert and back off
Double message, double booking effect before intent key write intent, then effect, then receipt
"Done" with no artefact receipt written from the 200 read the account back; store ids and paths
Muted by the user no message budget daily cap, digest, quiet hours, escalation classes
Runs forever no expiry failure count + relevance horizon, logged stop

Numbers worth watching

  • Wake rate = runs / checks. Well under 1, or the check is too loose.
  • Useful-run ratio = runs whose receipt the person acted on / runs. Low but non-zero is healthy; zero means the action should be deleted.
  • Duplicate-effect rate = repeated effects per key. Must be exactly 0.
  • Time to notice a dead check — measured by sending yourself a test wake once a week. A monitoring system that has never proven it can alert is a decoration.
  • Noise ratio = messages the person ignored or disabled / messages sent. Above roughly a third, the channel is dying.

Checklist

  1. Schedule stores a timezone, a due time and an end date.
  2. Check is a program with an exit-code contract, a timeout, and a stored predicate value.
  3. Exit 2+ is an error class, alerted once, then backed off.
  4. Intent key written before any effect that cannot be undone.
  5. Receipt written from a re-read of the target, with ids and evidence paths.
  6. Quiet hours are applied to output, with an explicit urgent class.
  7. Per-action and per-day message budget, with digesting.
  8. Expiry on failures and on relevance; stopping is logged and explained.
  9. No-op runs leave a counter you can see.
  10. Anything that spends, commits, or speaks in someone's name waits for that person.

None of this needs a better model. A background agent is a scheduler that has learned to stay quiet, and the parts that make it trustworthy are the parts that were already true of batch jobs forty years ago: a cheap predicate, an idempotent effect, and a log you can audit against the outside world.

Disclosure: I am a user of Pickle, a phone-side AI assistant app that runs scheduled actions of the kind described above. I am not its vendor. The links below are mine: a page on its daily limits and reset tickets, a comparison of where conversational, browser and phone-side assistants stop, and an invite link that gives both sides reset tickets if a sign-up completes through it. Nothing in this post depends on them — the schedule, check and log design stands on its own.