Cut LLM Costs: Token Budgets, Caching and Cheaper Models

An AI automation that runs once an hour costs almost nothing. Then someone pastes in a long document, a retry loop goes wrong, or the volume triples, and the bill grows without anyone noticing.

The good news is that most of the cost of a language model comes from a few things you can control. This guide explains how you're charged, then builds a small set of tools: a cost calculator, a daily spending limit, a cache for repeated questions, and a way to find the cheapest model that still does the job. It also covers Anthropic's prompt caching, with the numbers worked out.

The code uses only what Python already includes, apart from the optional Claude example.

How you're charged

Language model APIs charge by the token, a small piece of text, roughly a short word or part of one. You pay separately for what you send (input) and what the model writes (output), and the output costs more.

Here are the prices for some of Anthropic's models, per million tokens, as listed on Anthropic's pricing page in October 2026:

Prices change often, so check the pricing page before you rely on these. A few other facts from that page matter:

• Output costs five times as much as input for every model in the table.
• Haiku 5.5 charges more for very long prompts. Prompts over 100,000 tokens cost $0.50 input and $2.50 output per million.
• The Batch API gives 50% off both input and output, for requests that don't need an immediate answer.
• Newer models can use more tokens. Claude 4.7 and later models use a newer tokenizer that produces about 30% more tokens for the same text, so the same prompt can cost more tokens on a newer model. Count tokens with the model you plan to use.
• Tools add tokens. Tool definitions are sent with every request, and there's a fixed overhead too. On Sonnet 5.5, that overhead is 286 tokens before your own definitions.

Where the money goes

In small automations, the cost usually comes from a handful of places:

• Long instructions repeated on every call. A 6,000-token set of rules, sent 200 times a day.
• Big inputs. A whole document where one section would do.
• Long conversations. Each turn resends the history.
• Chatty outputs. Output tokens cost five times as much as input tokens.
• Loops and retries. An agent that keeps asking for tools, or a retry with no limit.
• The wrong model. A top-tier model doing work a small one handles fine.
• Asking the same question again and again.

Before reaching for a model at all, it's worth asking whether you need one. A rule or a regular expression is free, and it never has a bad day. I compare rule-based and AI-based automation in AI agents: how they work vs RPA.

Six ways to spend less

Step 1: Know what you spend

You can't cut what you can't see. Create costs.py. It turns the usage numbers from a model response into dollars, and keeps a running total for the day.

"""Work out what language model calls cost, and stop before a budget is spent.""" import json from datetime import date from pathlib import Path # Dollars per million tokens, from Anthropic's pricing page in October 2026. Prices change, so check the page. # "cache_read" is the fraction of the input price charged for tokens read from the prompt cache. PRICES = { "claude-sonnet-5-5": {"input": 2.00, "output": 10.00, "cache_read": 0.05}, "claude-opus-5-5": {"input": 4.00, "output": 20.00, "cache_read": 0.05}, "claude-haiku-4-5-20251001": {"input": 1.00, "output": 5.00, "cache_read": 0.10}, # Confirm the exact model ID on the models page. Prompts over 100,000 tokens cost more on this model.  "claude-haiku-5-5": {"input": 0.10, "output": 0.50, "cache_read": 0.10, "long": {"input": 0.50, "output": 2.50}}, } CACHE_WRITE = {"5m": 1.25, "1h": 2.0} # multiples of the input price  def field(thing, name): value = thing.get(name) if isinstance(thing, dict) else getattr(thing, name, None) return value or 0 def cost_of(model, usage, batch=False): """Dollars for one response, worked out from its usage numbers.""" price = PRICES[model] fresh, read, output = field(usage, "input_tokens"), field(usage, "cache_read_input_tokens"), field(usage, "output_tokens") written = field(usage, "cache_creation_input_tokens") split = usage.get("cache_creation") if isinstance(usage, dict) else getattr(usage, "cache_creation", None) written_1h = field(split, "ephemeral_1h_input_tokens") if split else 0 written_5m = written - written_1h # without a breakdown, assume the shorter cache  rates = price["long"] if "long" in price and fresh + read + written > 100_000 else price per_token = rates["input"] / 1_000_000 dollars = (fresh + written_5m * CACHE_WRITE["5m"] + written_1h * CACHE_WRITE["1h"] + read * price["cache_read"]) * per_token dollars += output * rates["output"] / 1_000_000 return dollars * (0.5 if batch else 1) class BudgetExceeded(Exception): pass class Spend: """Keeps a running total for each day in a small file, and refuses new calls once the daily limit is reached.""" def __init__(self, daily_limit, path="spend.json", today=date.today): self.limit, self.path, self.today = daily_limit, Path(path), today def _load(self): try: return json.loads(self.path.read_text()) except FileNotFoundError: return {} except ValueError: # a damaged file must not quietly reset the total to zero  raise BudgetExceeded(f"{self.path} is damaged, so spending can't be checked") def total(self): return self._load().get(str(self.today()), {}).get("dollars", 0.0) def check(self): if self.total() >= self.limit: raise BudgetExceeded(f"today's limit of ${self.limit:.2f} has been reached (${self.total():.2f} spent)") def record(self, model, usage, batch=False): data = self._load() day = data.setdefault(str(self.today()), {"dollars": 0.0, "calls": 0}) day["dollars"] += cost_of(model, usage, batch) day["calls"] += 1 temporary = self.path.with_suffix(".tmp") temporary.write_text(json.dumps(data, indent=2)) temporary.replace(self.path) # a crash can't leave a half-written file 

A few things to notice:

• PRICES is the table from above, in code. Update it when prices change.
• cost_of() reads the usage numbers that come back with every response: fresh input, cache writes, cache reads and output. It follows the documented rules: a 5-minute cache write costs 1.25 times the input price, a 1-hour write costs 2 times, and a cache read costs a fraction of it.
• Spend keeps a daily total in a small file and refuses new calls once your limit is reached. If the file gets damaged, it stops calls instead of quietly starting from zero, because a budget that resets by accident isn't a budget.

To check the calculator against Anthropic's own worked example, save this as example_usage.py:

from costs import cost_of # Anthropic's own worked example for Claude Opus 5.5: 50,000 input and 15,000 output tokens print(f"No caching: ${cost_of('claude-opus-5-5', {'input_tokens': 50_000, 'output_tokens': 15_000}):.3f}") # Same request, with 40,000 of the input tokens read from the prompt cache print(f"With cache reads: ${cost_of('claude-opus-5-5', {'input_tokens': 10_000, 'cache_read_input_tokens': 40_000, 'output_tokens': 15_000}):.3f}") 

It prints:

No caching: $0.500 With cache reads: $0.348 

Those match the figures on Anthropic's pricing page: $0.20 for input and $0.30 for output without caching, and $0.04 plus $0.008 plus $0.30 with 40,000 tokens read from the cache.

Step 2: Put limits in front of every call

Create guarded_model.py. It wraps your model call with four protections.

import hashlib import json import time from pathlib import Path MAX_INPUT_TOKENS = 20_000 # refuse to send a bigger prompt than this MAX_OUTPUT_TOKENS = 500 # a hard cap on how much the model may write  def rough_tokens(text): """A cautious estimate. For an exact count, use your provider's token counting.""" return len(text) // 3 class GuardedModel: """Wraps a model call with a size limit, a spending limit, and a cache for repeated questions.""" def __init__(self, send, spend, cache_dir=".reply-cache", max_age=86400): self.send, self.spend, self.dir, self.max_age = send, spend, Path(cache_dir), max_age self.calls = self.cache_hits = 0 def _key(self, model, system, prompt, max_tokens): return hashlib.sha256(json.dumps([model, system, prompt, max_tokens]).encode()).hexdigest() def ask(self, model, system, prompt, max_tokens=MAX_OUTPUT_TOKENS, use_cache=True): if rough_tokens(system + prompt) > MAX_INPUT_TOKENS: raise ValueError(f"that prompt is over {MAX_INPUT_TOKENS} tokens, so it wasn't sent") max_tokens = min(max_tokens, MAX_OUTPUT_TOKENS) file = self.dir / (self._key(model, system, prompt, max_tokens) + ".json") if use_cache: try: saved = json.loads(file.read_text()) if time.time() - saved["saved"] < self.max_age: self.cache_hits += 1 return saved["text"] # the same question as before, so there's nothing to pay for  except (OSError, ValueError, KeyError): pass self.spend.check() # raises BudgetExceeded before anything is sent  text, usage = self.send(model, system, prompt, max_tokens) self.calls += 1 self.spend.record(model, usage) if use_cache: self.dir.mkdir(exist_ok=True) file.write_text(json.dumps({"saved": time.time(), "text": text})) return text 

• An input limit. A prompt over about 20,000 tokens is refused before it's sent. The check uses a cautious estimate of one token per three characters. For exact counts, use your provider's token counting.
• An output limit. Whatever max_tokens the caller asks for, the model can never write more than 500 tokens.
• A daily budget. Once the day's limit is reached, new calls are refused before anything is sent.
• A memory for repeated questions. An identical question gets the saved answer for free, even after the budget runs out.

Any change in the model, instructions, question or output limit counts as a new question, so a saved answer is only reused when it really is the same request.

A saved answer is only right if the same answer is still acceptable. That's true for something like "classify this email" and false for "what's the latest on this order?". Saved answers also sit in a folder on disk, so don't use this for private data unless that folder is protected. And pass use_cache=False when you want a fresh answer.

I use the same idea in my own tools. My Repurpose Bot has a per-run limit on how many posts it handles, so a bad day can't produce a surprise bill, and you bring your own AI key, so you can see usage on your provider's page. A limit that works without anyone watching it is the best kind.

Step 3: Use prompt caching

If every call starts with the same long instructions, you're paying full price to send them each time. Prompt caching lets the provider remember them.

Here's a send function for Claude, using Anthropic's Python library (pip install anthropic). It plugs into GuardedModel above:

import anthropic client = anthropic.Anthropic() # reads the key from the ANTHROPIC_API_KEY environment variable  def send(model, system, prompt, max_tokens): response = client.messages.create( model=model, max_tokens=max_tokens, system=[{"type": "text", "text": system, "cache_control": {"type": "ephemeral"}}], # cache the fixed instructions  messages=[{"role": "user", "content": prompt}]) return response.content[0].text, response.usage 

The cache_control marker on the system block tells Claude to cache everything up to and including it. Anthropic's documentation gives these rules:

• Put the fixed content first, and the marker at the end of it. Caching reads a prefix that is identical across requests. Put the marker on the last block that doesn't change, which here is the system prompt.
• Don't put changing content before the marker. A timestamp or a user's message ahead of the cached part changes the prefix every time, so you pay to write the cache on every request and never get a read.
• Automatic caching suits conversations, not this. There's also a single top-level cache_control setting that caches up to the last block. It's meant for growing multi-turn conversations. With a fixed prefix and a changing final message, it would place the cache point on the part that changes.
• There's a minimum length. For Sonnet 5.5, Opus 5.5 and Haiku 5.5, the prompt must be at least 512 tokens to be cached. For Haiku 4.5 it's 4,096. A shorter prompt is processed without caching, and there's no error.
• The cache lasts 5 minutes, and each use refreshes it at no extra cost. A 1-hour cache is available too, with "ttl": "1h" in the marker, at 2 times the input price to write.
• Check that it works. The usage in each response shows cache_creation_input_tokens (written) and cache_read_input_tokens (read). If both are zero, nothing was cached.

Does it pay off? Here's the arithmetic from cost_of() for a 6,000-token set of instructions on Sonnet 5.5, with a 300-token question and a 200-token answer. Save this as savings.py:

from costs import cost_of MODEL = "claude-sonnet-5-5" PREFIX, QUESTION, ANSWER = 6000, 300, 200 # tokens: the fixed instructions, the changing question, the reply  def day(calls, usage_for): return sum(cost_of(MODEL, usage_for(i)) for i in range(calls)) no_cache = lambda i: {"input_tokens": PREFIX + QUESTION, "output_tokens": ANSWER} cached = lambda i: ({"input_tokens": QUESTION, "cache_creation_input_tokens": PREFIX, "output_tokens": ANSWER} if i == 0 else {"input_tokens": QUESTION, "cache_read_input_tokens": PREFIX, "output_tokens": ANSWER}) every_call_writes = lambda i: {"input_tokens": QUESTION, "cache_creation_input_tokens": PREFIX, "output_tokens": ANSWER} one_hour_cache = lambda i: ({"input_tokens": QUESTION, "cache_creation_input_tokens": PREFIX, "cache_creation": {"ephemeral_1h_input_tokens": PREFIX}, "output_tokens": ANSWER} if i == 0 else {"input_tokens": QUESTION, "cache_read_input_tokens": PREFIX, "output_tokens": ANSWER}) print(f"A {PREFIX}-token set of instructions, reused on every call ({MODEL})") print(f" 200 calls a day, no caching: ${day(200, no_cache):.2f}") print(f" 200 calls a day, 5-minute cache, calls close together: ${day(200, cachedSource：https://dev.to/procwire/cut-llm-costs-token-budgets-caching-and-cheaper-models-1l8d
My favorites

Related posts

Turn Today’s Messy Notes Into Tomorrow’s Checklist With AI

I run more than one thing at once. Most small business owners I know do the same. By late afternoon your head is full of

0dev.to AI

A 763B MoE at "2 Bits": Where the Bytes Really Go in the GSQ-RCO GGUFs, and Why 100% Recovery Sits Inside the Noise

TL;DR ISTA's DASLab published GGUF builds of DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model, at 1.7

0dev.to AI

How I ship fixes to open-source projects maintainers actually merge (with AI in the loop)

Last month six of my pull requests were merged into other people's projects: an Android keyboard, a Python OAuth library

0dev.to AI

Prompt Injection Through Documents: How to Harden an LLM Extraction Pipeline

A document is not a passive object once a language model reads it. Anything printed, hidden or embedded in a PDF becomes

0dev.to AI

Keeping noindex, the sitemap and the image sitemap in agreement for programmatic pages

Google's documentation for sitemaps says to "include the URLs in your sitemap that you want to see in Google's search re

0dev.to AI

3 Things in AI You Should Know: Character.AI Files for Bankruptcy

Disclosure: TechSifted has no affiliate relationship with Character.AI, Pika, or Mistral. This is editorial news coverag

0dev.to AI