Last quarter we finally stopped trusting the LLM bill. We wired real per-step token attribution into 11 production agents — not the default "total tokens" the provider hands you, but a tracer that tags every call, every tool, every cache lookup with the trajectory it belonged to.

The number that broke our mental model:

Across those 11 agents, the single most expensive step — measured per resolved task, not per API call — was the model in exactly 2 of them. In the other 9, something we'd filed under "negligible overhead" was quietly eating more than the LLM.

Here is the trap, in one picture. Same agent, same steps — two ways of ranking them.

WHAT THE DASHBOARD SHOWS (per-call mean) WHAT THE BILL ACTUALLY IS (per-trajectory total) Step $/call rank Step $/task rank LLM completion $0.0028 #1 PII / guardrail $0.0042 #1 PII / guardrail $0.00035 #4 Embedding loop $0.0036 #2 Embedding (in loop) $0.00002 #5 LLM completion $0.0028 #3 Semantic-cache chk $0.00090 #3 Semantic-cache chk $0.0009 #4 Tool call $0.00010 #2 Tool call $0.0008 #5

Left column: what every dashboard reports. Right column: what one task actually costs. The model drops from #1 to #3. A step you pay two-hundredths of a cent per call becomes the #2 cost driver. The ranking is not the same list.

And across all 11 agents, the #1 cost step by trajectory-total:

#1 cost driver (per trajectory), across 11 production agents Embedding / retrieval loop ██████████ 5 / 11 PII / guardrail model ██████ 3 / 11 Model (LLM) ████ 2 / 11 Cache-verify model ██ 1 / 11

Nine of eleven. The model isn't even in the lead most of the time.

The hidden axis: mean-vs-trajectory

The reason dashboards mislead is structural. They report mean cost per call, because that's what one request easily exposes. But an agent step isn't called once — it's called inside loops, per chunk, per re-plan.

The real cost of a step is:

step_cost_per_task = calls_per_task × cost_per_call

A step at $0.00002/call looks free on the mean chart. Call it 180 times in a retrieval loop and it's $0.0036/task — more than the LLM, which runs once. The "free" step wins by 28%, on exactly the hard tasks you care about.

The invisible cost axis is trajectory-total vs call-mean. Every dashboard I've seen only draws one of them.

Three cases we actually shipped

These aren't hypotheticals — they're line items from the 11 agents.

Case 1 — The "free" embedding step that wasn't.
We had a retrieval step the finance team literally labeled free in the cost model (embeddings at $0.00002/call). It lives inside the agent's re-planning loop, which on hard tasks re-retrieves up to 180 times. 180 × $0.00002 = $0.0036/task. The LLM completion, called once, is $0.0028. So the step we scored as zero out-costs the model by 28% — on every hard task, which is the only tasks that matter.

Case 2 — The cache hit that cost more than the miss.
Our semantic cache doesn't just return the cached answer on a hit. It runs a reranker plus a 70B "is this still relevant?" check, to avoid serving stale advice. That verification costs $0.0009. A genuine miss regenerates from the 8B model for $0.0006. So a hit — the thing caches exist to save you — costs 50% more than a miss, across the 40% of traffic that hits. We were paying a "savings" tax and calling it a win.

Case 3 — The guardrail that 4×'d the bill.
A PII-scrubber runs a 7B model on every message chunk before the LLM sees it: 12 chunks × $0.00035 = $0.0042/task. We added it for a compliance ask and never costed it per-trajectory. It became the #1 line item on 3 of the 11 agents. Removing it wasn't on the table — but routing it to a 0.5B model dropped it to $0.0003, a 14× cut, with no detectable quality change on our eval set.

How you should actually measure

The fix is almost never "pick a cheaper model." It's accounting per trajectory so the loop multiplier can't hide, then routing each step to the right-sized model. We sit in front of 25+ models behind one OpenAI-compatible gateway; the interesting savings came from moving the scrubber and the cache-verify down, not from swapping the completion.

Three changes that took an afternoon:

  1. Attribute every token to a trajectory_id, not a request. If you can't say what one task cost, you don't know your cost.
  2. Rank by total, sample real tasks. Pull N resolved tasks, sum per step, sort by total. The mean chart will lie to you; the total won't.
  3. Put a cost_per_resolved_task SLO in place. Alert on the trajectory, not the call.

A minimal version of what we run:

cost_attribution: unit: trajectory # not 'request' and not 'call' trace_id: trajectory_id # tag every step, tool, cache lookup rank_by: total_cost # never mean_cost alert: metric: cost_per_resolved_task threshold_usd: 0.02 exclude_steps: [] # never exclude the 'cheap' steps

And the five lines that surface the buried step:

from collections import defaultdict def rank_steps_by_task(events): per_step = defaultdict(float) for e in events: per_step[e.step] += e.cost_usd # already trajectory-attributed  return sorted(per_step.items(), key=lambda kv: -kv[1]) # prints the REAL ranking — usually not what your dashboard shows for step, cost in rank_steps_by_task(events): print(f"{step:24} ${cost:.5f}")

The question I'll leave you with

The dashboards aren't lying. They're reporting the axis that's easiest to measure. We spent a year optimizing model choice while a loop multiplier and a "free" safety step ran away from us — invisible the whole time, because nothing on the screen was drawing the trajectory-total axis.

So: when you last looked at your agent's cost, were you ranking by what one call costs — or by what one task actually costs? Because those are not the same list.

What's the step you've been treating as free?