Asking an LLM "are these two tickets duplicates?" costs $111 for 1,000 tickets and $11,130 for 10,000

You have a pile of support tickets and want to find the ones that describe the same problem. Models are good at this: show one two tickets, ask "same issue, yes or no?", and the answer is usually right. So you write a loop that compares each ticket with every other one. It costs a dollar on your 100 test tickets, and you ship it. Then it runs on the real backlog.

Why the bill grows faster than the data

Model APIs bill per token, a chunk of text that averages about three-quarters of an English word. Input tokens are what you send, and output tokens are what the model writes back. A yes/no comparison is almost all input: two tickets plus a short instruction in, one word out.

The catch is the number of calls. With N tickets, the number of distinct pairs (each ticket matched with each other ticket once) is N × (N − 1) / 2. That's 4,950 pairs for 100 tickets, 499,500 for 1,000, and almost 50 million for 10,000. Ten times the tickets means about a hundred times the calls. Doubling 1,000 tickets to 2,000 multiplies the cost by exactly 4.00. Growth like this is called quadratic, because the cost follows the square of the input size.

Measuring it

I generated 200 synthetic tickets (one to three sentences, like "Hello support, the export my data thing is broken for me again...") and counted tokens with tiktoken, OpenAI's open-source tokenizer library. Other vendors split text a little differently, but the call count doesn't depend on the tokenizer. A pair-comparison call came out at 105 input tokens and 1 output token.

The second approach labels first. One call per ticket asks the model to pick a category from a fixed list of 20 ("billing", "login", "app crash" and so on). That's 114 input tokens and about 2 output tokens. Then you compare pairs only inside each category, since a billing ticket is unlikely to duplicate a crash report.

def pairs(n): return n * (n - 1) // 2 PAIR_CALL = (105.313 * 2.00 + 1 * 12.00) / 1e6 # measured tokens x GPT-5.6 list price for n in (100, 1_000, 10_000): print(f"{n:>6,} tickets: {pairs(n):>10,} calls ${pairs(n) * PAIR_CALL:,.2f}") 

Prices are list prices, meaning the ones each vendor publishes, taken from the catalog of the LLM gateway I work on on 13 November 2026. On GPT-5.6 ($2 per million input tokens, $12 per million output), with tickets spread evenly over 20 categories:

The labeled version is 19.9x cheaper at 10,000 tickets. That's no accident: splitting into K equal groups divides the number of pairs by roughly K, and K is 20 here. The vendor doesn't change the ratio either. GPT-5.6-mini, Claude Haiku 5.5, Claude Sonnet 5.5 and Gemini 3.8 Flash all show the same 15.9x / 19.5x / 19.9x, because the savings come from the call count, not the price per token. A cheaper model only shrinks the absolute number: all pairs over 10,000 tickets is $552 on Claude Haiku 5.5 and $1,113 on GPT-5.6-mini. That's still a lot for a job that grows by itself every time the backlog does.

The same thing happens when a single new ticket arrives. Comparing it against 10,000 existing ones costs $2.23 on GPT-5.6. Labeling it and comparing it only with its category costs $0.11.

Real categories aren't even

The 20x assumes every category has the same number of tickets, and real data never does. A few categories get most of the traffic, and the pair count inside a big group is quadratic again. I reran 1,000 tickets with two lopsided splits. When category sizes fall off like 1/rank, so the top one has 278 tickets, the bill is $13.87, 8x cheaper than all pairs instead of 19.5x. When a catch-all "other" holds 40% of the tickets, it's $20.11, only 5.5x cheaper. Your biggest category sets your bill. If one grows large, split it further.

from collections import defaultdict def candidate_pairs(tickets, label_of): # label_of: one model call per ticket groups = defaultdict(list) for t in tickets: groups[label_of(t)].append(t) for group in groups.values(): # compare only inside a group for i in range(len(group)): for j in range(i + 1, len(group)): yield group[i], group[j] 

What this doesn't solve

Labeling first trades cost for recall. Recall is the share of real duplicates you actually find. If the model files one of two identical tickets under "login" and the other under "password reset", that pair is never compared, and you won't know you missed it. I measured cost, not accuracy, so I don't have a miss rate. Take a few hundred tickets where you already know the duplicates and compare both approaches on them before switching.

The label list is also something you have to maintain. New products bring new problem types, and an "other" category that quietly grows brings the quadratic bill back. At real scale, the usual tool for this is embeddings, which turn each text into a list of numbers so similar texts can be found with cheap math instead of a model call. The model then only checks the few closest candidates. The catalog I used doesn't list embedding models, so I didn't price that route here. And no gateway can fix this for you, including the one I work on: it sees each call, not the loop that makes half a million of them.

The takeaway

Before you ask a model about every pair, multiply N × (N − 1) / 2 by the price of one call, and do it for the size of your data next year, not today. If the answer scares you, put a cheap step in front that decides which pairs are worth asking about at all, and watch the size of your largest group.