Yesterday two Amazon engineers published a post with a headline built for Hacker News: Amazon saves millions by replacing Sonnet 4.6 with Strands Decider. By this morning it had hundreds of points and a very opinionated comment thread.

I've led teams for years and shipped a fair number of startups, some of which are no longer with us. This story touches a habit I recognize in myself, so I want to go through what they actually did, why it worked, and why I'm not fully convinced yet.

What happened

Amazon has a huge internal ticketing system covering customer escalations, capacity requests, alarms and security findings. Every ticket needs one of 20 labels. That job was done by Claude Sonnet 4.6, a large and capable language model.

They replaced it with Strands Decider, a "decision model" Amazon launched about a month ago. It can't generate text. You give it some state (the ticket), a question and a list of options. It returns a probability for each option plus a confidence score. That's all it does.

The numbers from their post:

  • Cost per ticket: $0.0222 with Sonnet vs about $0.0005 with Decider. That's roughly $22,200 vs $500 per million tickets, which they call "about 44 times cheaper" and a 98% reduction.
  • Speed: average call time went from 4,215 ms to 897 ms.
  • Tokens: input dropped from 6,554 to 2,009 per ticket, and output from 170 tokens to 1.
  • Rollback: a three-way config switch (old model, both in parallel, new model). They say going back takes 45 seconds.

The headline claim is that it "performed just as well" and will "save millions of dollars of token costs in a year."

The lawyer sorting your mail

Here's the plain version for anyone who doesn't write code. Using a big language model to pick one of 20 labels is like hiring a lawyer to sort incoming mail into 20 trays. The lawyer will do it well. The lawyer can also write you a memo about each letter, which nobody asked for, and bills by the hour.

The small model is a trained mail clerk. It can't write memos. It just puts letters in trays quickly and cheaply, and it tells you when it isn't sure.

Why I recognize this

As a team lead, I've watched the same pattern again and again, and I've done it myself. You need something working by Friday. A big model gets you there in an afternoon with a prompt and an API key, so you ship it.

Then it works. That's the trap. Nobody puts "check whether this is now a simple job" on the roadmap, so the lawyer keeps sorting mail for two years.

To me the good part of Amazon's post isn't the model. It's that someone went back and looked.

Why I'm not fully sold

Some fair skepticism, much of it from the HN thread:

  1. They sell this model. It's Amazon's own product on Amazon's own blog. That doesn't make it false, but it's a launch story as much as an engineering story.
  2. No accuracy numbers. The post says the new model "beat production Sonnet on a separate set of our tickets" but publishes no precision, recall or sample size. "Just as well" is doing a lot of work.
  3. "Millions" is a projection. No ticket volume is disclosed, so we can't check the math. One HN commenter estimated it would take tens of millions of tickets a year, which is plausible at Amazon's scale.
  4. Cheaper baselines exist. Commenters asked why not a smaller LLM, or a plain classifier like BERT, which people have used for this exact job for years.
  5. They had to bend the problem. The model has a 4,096-token window. They rewrote every label as a one- or two-sentence criterion and drop the oldest comments on long tickets.

The practical check

The authors list their own limits, and I find them the most useful part. It worked because "the task is a closed choice from a known catalog, the volume is high, and the latency of the old path mattered." If you need explanation text, your catalog changes weekly, or the useful context won't fit in a few thousand tokens, they say a decision model probably can't do the job.

So here's what I'd do this week on any product paying for AI:

  • List every place you call a big model.
  • For each one, ask: is the answer really one item from a fixed list? Labels, routing, yes/no, priority?
  • If yes, check the volume. At a few hundred calls a day, don't bother. Your time costs more than the tokens.
  • If the volume is high, run the cheap option in parallel first, like they did, and compare answers before you switch.

Big models are great for starting fast. They're just expensive to forget about.

Which would you choose for a high-volume job: a flexible model you never have to retrain, or a narrow one that's 40x cheaper but needs care? Why?