Anthropic's prompt caching promises a 90% discount on cached input tokens. Teams add cache_control: {"type": "ephemeral"} to their system prompts, deploy to production, and expect their API bills to collapse.
Then the invoice arrives, and costs went up by a quarter.
Here is the mechanical trap inside exact-prefix caching.
The Primary Source: Anthropic Prompt Caching Specification
Anthropic's official Claude documentation defines prompt caching mechanics with mathematical precision:
-
Hierarchical Prefix Order:
"Prompt caching is evaluated strictly from the beginning of the prompt. Tools are evaluated first, then system messages, then user and assistant messages."
-
Exact Byte-for-Byte Prefix Match:
"Prompt caching requires an exact prefix match. Any change to tools, system instructions, or message order creates a new prefix."
-
Pricing Multipliers Relative to Base Input Tokens:
"5-minute cache writes are priced at 1.25x the base input token price, while cache reads are priced at 0.1x (a 90% discount)."
-
Minimum Token Threshold:
"The minimum cacheable prompt length is 1,024 tokens." (Below this threshold, caching directives fail silently without writing or reading from cache).
Where the Silent Penalty Happens
Consider a standard production agent with a 4,000-token system prompt containing instructions, policy rules, and schema definitions.
# The common architectural trap import datetime system_prompt = f""" You are a reliable enterprise financial agent. Current Server Time: {datetime.datetime.utcnow().isoformat()} Request ID: {request.id} [... 4,000 tokens of static policy and guidelines ...] """ Here is the step-by-step failure mode:
- Prefix Divergence at Token Zero: Anthropic evaluates tokens in strict sequence:
tools$\longrightarrow$system$\longrightarrow$messages. Because{datetime.datetime.utcnow()}changes on every request, the prefix diverges within the first 20 tokens. - 0% Cache Hit Rate: Caching does not use semantic similarity. Because the prefix changed, zero subsequent tokens match the cache.
- The 1.25x Write Surcharge: Because
cache_controlwas explicitly marked on the block, the API attempts a cache write on every single request. - The Invoice Shock: A cache read costs 0.10x. But a cache write costs 1.25x—a 25% premium over standard input pricing.
- The Net Outcome: Every request pays 1.25x base price. You paid a 25% penalty across your entire workload to populate disposable cache entries that expired five minutes later without ever being read once.
Expected: [4,000 tokens @ 0.10x] --> 90% Cost Reduction Reality: [4,000 tokens @ 1.25x] --> 25% Cost Surcharge The Second Failure Mode: Non-Deterministic Tool Ordering
Even if your system prompt is completely static, Python dictionary or set iteration can silently destroy cache hits:
# Unsorted tool registration tools = [search_tool, calculator_tool, sql_query_tool] # If order flips across worker processes: # Worker A sends: [search, calc, sql] # Worker B sends: [calc, search, sql] Because tools precede system in Anthropic's prefix evaluation, flipping the order of two tools invalidates the entire cache for both the tools AND the system prompt downstream.
The Engineering Discipline
To guarantee 90% cache savings instead of a 25% write penalty:
- Strict Prefix Segregation: Keep
toolsand foundational system instructions 100% immutable. Move timestamps, user session IDs, and dynamic environment state into the trailing user message:
{ "role": "user", "content": [ {"type": "text", "text": "Context: Current time is 2026-11-20T10:15:00Z. Session: sess_94b18."}, {"type": "text", "text": "Analyze the Q3 ledger."} ] } - Deterministic Tool Serialization: Always sort tool definitions alphabetically by name before serializing them into the request payload.
- Assert on Cache Telemetry: Instrument your API middleware to alert when
cache_creation_input_tokens > 0on steady-state traffic or whencache_read_input_tokens == 0. - Respect the 1,024 Token Floor: Static prefixes under 1,024 tokens will not cache. Ensure static documentation blocks meet the threshold before adding cache breakpoints.
Two Hard Rules
- A dynamic variable in your static prefix turns a 90% discount into a 25% write penalty.
- In exact-prefix caching, order is architecture. If token zero changes, token ten thousand does not exist.
Honest limitation: I am reading Anthropic's official prompt caching documentation and pricing schedule. Realized cost impacts depend on prompt token length, call frequency, and cache hit distributions in your workload.
Checkable references: Anthropic Claude Platform Documentation ("Prompt Caching") and Claude API Pricing Schedule.
What this series is: one primary source a day — a spec, a changelog, a status page — and what actually changed, with the quote. Follow if you prefer the change over someone's summary.
