Free Tokens Aren't Free Requests: Five Budget Myths and a Spend Ledger You Can Run

A free endpoint feels like a blank check. It is not. My first week of free-model experiments taught me that much.

I was calling an OpenAI-compatible gateway through MonkeyCode's free model access. Then I compared my assumed spend against what actually left the account.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The gap was ugly. Not because the endpoint lied. Because my model of "one request, one charge" was wrong.

So here are five myths I keep hearing, the evidence that breaks them, and a ledger you can run tonight.

Myth 1: "A free token is a free request"

The claim: tokens are free, so requests are free.

The evidence: a failed request still shipped a prompt. Retries re-ship it. Your budget drains on attempts, not on successes.

Ask yourself: do you count attempts or answers?

The corrected model: budget per completed task, not per HTTP call.

cost_per_task = sum(tokens across every attempt) / tasks that actually finished 

That denominator is smaller than you think.

Myth 2: "One request equals one prompt"

The claim: the user message is what gets billed.

The evidence: the wire payload carries more. System prompt. Tool schemas. Few-shot examples. Conversation history you forgot to trim.

Here is a payload I see in real repos:

{ "model": "<your-model-id>", "messages": [ {"role": "system", "content": "<1,800-token style guide>"}, {"role": "user", "content": "Fix this typo"} ], "tools": ["<schema A>", "<schema B>", "<schema C>"] } 

The typo is eight tokens. The envelope is thousands.

The corrected model: the envelope is the request. Measure it before you optimize prompts.

Quick probe:

curl -s "$LLM_BASE_URL/chat/completions" \ -H "Authorization: Bearer $LLM_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model":"'"$LLM_MODEL"'","messages":[{"role":"user","content":"ping"}]}' \ | jq '.usage' 

If usage comes back null, stop. Your ledger must estimate. That is a finding, not a nuisance.

Myth 3: "Retries are free because the call failed"

The claim: no answer, no charge.

The evidence: the input tokens left your machine either way. Three retries on a long context is a 3x prompt bill with zero output.

What does your client log after a 500? Usually just a stack trace.

The corrected model: every attempt is a line item. Log status codes next to token counts.

Myth 4: "Caching makes the second call cheap"

The claim: identical prefixes are cached, so repeat calls are near-free.

The evidence: caching depends on the provider, the prefix, the TTL, and whether your client sends a stable prefix at all. A timestamp at the top of your system prompt kills it.

Have you verified the cache, or assumed it?

The corrected model: treat caching as an optimization you measure, never as a budget assumption.

Myth 5: "The free server makes the experiment free"

The claim: free compute plus free tokens equals zero cost.

The evidence: the server removes host cost. It does not remove the request budget. And a cheap box that restarts mid-run loses your unsaved ledger.

The corrected model: free server = workbench, not vault. Stream your records out as you go.

This is where I run the worker, on MonkeyCode's free server option. But the log leaves the box immediately.

The artifact: a spend ledger in about 60 lines

Label: untested scaffolding. Run it against your own endpoint before trusting any number.

# ledger.py import json, os, time, uuid from dataclasses import dataclass, field from urllib import request ENDPOINT = os.environ["LLM_BASE_URL"].rstrip("/") + "/chat/completions" API_KEY = os.environ["LLM_API_KEY"] MODEL = os.environ.get("LLM_MODEL", "<your-model-id>") @dataclass class Attempt: ok: bool status: int prompt_tokens: int = 0 completion_tokens: int = 0 ms: int = 0 @dataclass class Task: task_id: str attempts: list = field(default_factory=list) @property def prompt_tokens(self): return sum(a.prompt_tokens for a in self.attempts) @property def ok(self): return any(a.ok for a in self.attempts) def estimate_tokens(messages): # crude, deliberately visible: ~4 chars per token  return sum(len(m.get("content", "")) for m in messages) // 4 def chat(messages, retries=3): task = Task(task_id=str(uuid.uuid4())[:8]) body = json.dumps({"model": MODEL, "messages": messages}).encode() for i in range(retries): t0 = time.monotonic() req = request.Request(ENDPOINT, data=body, headers={ "Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json", }) try: with request.urlopen(req, timeout=60) as r: payload = json.loads(r.read()) usage = payload.get("usage") or {} task.attempts.append(Attempt( True, r.status, usage.get("prompt_tokens", estimate_tokens(messages)), usage.get("completion_tokens", 0), int((time.monotonic() - t0) * 1000))) return payload, task except Exception as e: task.attempts.append(Attempt( False, getattr(e, "code", 0), estimate_tokens(messages), 0, int((time.monotonic() - t0) * 1000))) time.sleep(min(2 ** i, 8)) return None, task if __name__ == "__main__": msg = [{"role": "user", "content": "Reply with the word: ok"}] _, task = chat(msg) print(json.dumps({ "task_id": task.task_id, "attempts": len(task.attempts), "prompt_tokens": task.prompt_tokens, "tokens_per_success": task.prompt_tokens if task.ok else None, "ok": task.ok, })) 

Run it:

export LLM_BASE_URL="https://<your-gateway>" export LLM_API_KEY="<your-key>" export LLM_MODEL="<your-model-id>" for i in $(seq 1 20); do python ledger.py; done | tee ledger.jsonl 

Twenty runs. Then read the file, not your memory.

jq -s '{runs: length, attempts: map(.attempts) | add, ok: map(select(.ok)) | length}' ledger.jsonl 

The decision table I use

Thresholds are my defaults. They are not vendor limits. Tune them.

Limitations, honestly

• The estimator is 4 chars per token. That is wrong for code and non-English text.
• 20 samples tells you about plumbing, not about cost at scale.
• Token counts say nothing about answer quality. Pair this with an eval set.
• Free-tier terms change. Read the current project page instead of trusting a screenshot.

Who should not use this: teams already tracking spend in a real gateway. You have dashboards. Use them.

What I would change first

Not the prompt. The retry policy.

If half your budget vanishes before a model answers, prompt engineering is the wrong lever. Fix the transport.

Want to reproduce this? Open the MonkeyCode project pages, read the live free model and free server terms, then run the ledger against your own workload. Measure first. Optimize second.