Coding agents have an expensive habit. Every turn, they send the whole conversation back to the model, with the system prompt, the tool definitions, every file read and every command run. A session that starts at 6,000 tokens reaches 30,000 a few dozen turns later, and the agent pays for all of it again on every call.
Prompt caching is supposed to fix this. If the model saw the first 29,000 tokens a second ago, it can reuse them, and Bedrock bills those tokens at a lower "cache read" price.
On November 16, AWS announced explicit prompt caching for open-weight models on Bedrock. The post describes an agent session where "each later request reads everything up to the end of the previous request". Its numbers are marked as illustrative. This article measures the same thing with real agent traffic.
This follows earlier research on how inference engines keep an agent's cache between turns, on vLLM, SGLang and a small custom engine (preprint). Here the same workload goes to a managed service.
The test
The sessions come from the SWE-rebench-OpenHands dataset (CC-BY-4.0). They are recordings of an agent fixing bugs in real GitHub repositories, reading files, running tests and editing code.
A small Python script replays them turn by turn against Bedrock, exactly the way an agent sends them.
- Turn 1 sends the system prompt, the tools and the task (about 6,500 tokens).
- Each next turn sends everything before it plus the latest tool result.
- Between turns the script waits 0.5 seconds, close to the median gap measured for real coding agents.
- Each session stops when its prompt passes 30,000 tokens.
Every response gives two numbers, the prompt size and how many tokens Bedrock served from cache (prompt_tokens_details.cached_tokens).
The runs used eu-north-1 (Stockholm), the InvokeModel API with an OpenAI-style chat body and two sessions at the same time, on November 21, 2026. Two coding models were available to the account.
- Kimi K3 (
global.moonshotai.kimi-k3), through the global cross-Region profile, the only way to reach it from Stockholm. Its model card lists implicit and explicit caching.reasoning_effortset tonone. - GLM 5 (
zai.glm-5), in-Region. Its model card does not mention prompt caching at all.
The requests carry no cache markers. This is the default, implicit caching an agent gets without code changes.
Result 1. Kimi K3 kept almost the whole history
| Kimi K3 | |
|---|---|
| Turns | 87 |
| Prompt tokens sent | 1,601,931 |
| Read from cache | 1,540,701 (96.2%) |
| History the model had already seen on the previous turn | 1,541,296 |
| Turns after the first with at least 90% of their history cached | 85 of 85 |
Out of 1,541,296 tokens the model had already seen, it reused all but about 600. In practice, only the new part of each turn was billed at the full input price.
The cache also lasted. The same sessions sent again 7 minutes later came back with 6,696 of 6,703 tokens cached on the very first turn. The model card says Kimi K3 keeps cache entries for at least 30 minutes.
Result 2. GLM 5 cached, but mostly the wrong part
GLM 5 had two surprises. First, it reports cached tokens even though its model card does not list prompt caching. Second, the hits were very uneven.
| GLM 5 | Run 1 | Run 2 (new sessions) |
|---|---|---|
| Turns | 88 | 35 |
| Prompt tokens sent | 1,620,724 | 632,524 |
| Read from cache | 984,192 (60.7%) | 302,272 (47.8%) |
| Share of already-seen history that was cached | 63% | 53% |
Turn by turn in run 1 (86 turns after the first)
- 26 turns reused at least 90% of their history.
- 33 reused part of it.
- 12 reused exactly 5,632 tokens.
- 15 reused nothing.
5,632 tokens is almost certainly the beginning every session shares, the system prompt and the tool definitions. In run 2, most turns hit exactly that number and nothing more. GLM 5 reliably kept the part that is the same for everyone, and only sometimes the agent's own history.
Every cached count was a multiple of 64, so the cache seems to work in blocks of 64 tokens. The AWS caching guide gives one likely reason for the misses, "A cache entry is stored on the server that processed the request, and a later request can read it only if that server handles it before the entry expires." When consecutive turns of one agent land on different servers, the history is computed again. How Bedrock routes GLM 5 requests isn't visible from outside, so this is an interpretation, not a measurement.
Cache hits did help speed. GLM 5 turns that reused at least 90% of their history took 0.99 seconds (median). Turns that reused less than half took 2.37 seconds.
Result 3. The bill
Cost Explorer, grouped by usage type, the next day.
Kimi K3 has three lines, input tokens, cache-read tokens and output tokens. The cache-read line is 2,283,928 tokens, exactly the sum of the cached tokens the script logged across all Kimi runs. The uncached input matches too. There is no cache-write line, so writing to the cache was not charged.
GLM 5 has two lines, input tokens (2,414,839) and output tokens. There is no cache-read line. Every token GLM 5 reported as cached was billed as normal input.
This matches the GLM 5 model card, which doesn't list prompt caching. The cached count in the response just isn't reflected in the price. For GLM 5, the cache makes turns faster but not cheaper.
In money, for the first run of each model (87 and 88 turns of the same two sessions)
| Kimi K3 | GLM 5 | |
|---|---|---|
| Input price per 1M tokens | $3.00 ($0.30 if cached) | $1.20 (no cache discount) |
| Cost of the run | about $0.80 | about $1.97 |
| Same run if nothing were cached | about $4.96 | about $1.97 |
Kimi K3 lists 2.5 times the input price of GLM 5, and still cost less than half as much for this agent workload, because 96% of its input was billed at the cache price.
The Bedrock pricing page has no Stockholm row for GLM 5. The total bill ($4.05 for all runs) only adds up with GLM 5 at $1.20 input and $3.84 output per 1M tokens, the page's "Asia Pacific & South America" row. That figure comes from the arithmetic, not from AWS.
Result 4. A new account hits a wall quickly
Kimi K3 started throttling during the first run (17 retries), and on a repeat run it stopped answering entirely after about 100 requests, with this error.
ThrottlingException: Too many requests, please wait before trying again. You have sent too many requests. The account's quota for Kimi K3 is 10,000,000 tokens per minute, far above what two sessions use, and the quota list shows no requests-per-minute limit for it. So the limit that hit isn't one that can be seen or raised through a quota request. The account is new and on the Free plan, which may matter. On a new account, test the throughput before depending on it.
GLM 5 never throttled.
Strands Agents
Strands is AWS's open-source agent SDK. Its source (sdk-python, November 20) shows the following for these two models.
- The Bedrock provider uses the Converse API.
- With
cache_configset to"auto", Strands adds cache points only for models its model list marks as supporting them, or whose id contains "claude" or "anthropic" (bedrock.py,_cache_strategy). GLM 5 and Kimi K3 are not marked, so it adds none. The Strands docs say "Prompt caching is supported for Anthropic Claude and Amazon Nova models on Bedrock." - The Kimi K3 model card warns that Converse fails with
InternalServerExceptionwhen reasoning content from earlier turns is included, "which affects frameworks such as LangChain and Strands Agents in their default configurations". It recommends the Chat Completions or Responses APIs instead.
So a Strands agent on Kimi K3 or GLM 5 relies on the same implicit caching measured here. These runs used InvokeModel, not Converse, so Converse caching was not verified.
Why it matters
The same problem shows up on self-hosted engines. In the earlier research, the same agent sessions replayed against vLLM and SGLang on one GPU spent more than half of all prompt tokens on history computed once and then thrown away. Keeping each agent's history in memory between turns made the whole workload 1.31x to 1.45x faster (preprint, code).
A managed service faces the same choice. When an agent comes back half a second later, its history has to still be on the server that answers. For Kimi K3 on Bedrock it almost always was. For GLM 5 it often wasn't, and even when it was, the bill did not reflect it.
Checklist for agents on Bedrock
- Pick a model whose model card lists prompt caching for agent workloads. Kimi K3 lists it, and it delivered 96% of the input billed at the cache price.
- Look at
cached_tokensin the responses. It is in the usage block of every response. If it is far below the prompt size on later turns, the history is being reprocessed. - Confirm the savings in Cost Explorer. Grouped by usage type, it shows a cache-read line for models where caching is billed. That is the real saving.
- Compare models on the actual workload, not only the price sheet. The model with the higher list price was the cheaper one here.
- Test throughput early on a new account. The limit that stopped these runs wasn't in the quota list.
Limits of this test
- Two sessions per run, one Region, one tool wait (0.5 s). Kimi K3 was measured in one full run. The repeat on new sessions was blocked by throttling after two turns.
- The replay ignores what the model answers, and each turn sends the recorded history. That measures caching, not agent quality.
- Prices are list prices from the pricing page and model cards on November 21, 2026, and the GLM 5 Stockholm price is inferred from the bill.
The replay script and the raw results are in this gist. It runs from CloudShell with the account's existing credentials, and stops by itself at a set budget.

