The demo cost four pence. You ran it thirty times while building it, glanced at the total, and stopped thinking about it, because four pence is not a number anybody worries about. Then it shipped, and the cost stopped being yours. It became a function of what people do. Somebody pastes in a two hundred page document. A nightly job decides to re summarise everything, including the things that have not changed. A page re asks the same question on every keystroke, because somebody wired it to the wrong event. None of that is abuse. That is a feature working, and the bill grows in a place nobody is watching, because the graph you watch is latency. Treat cost the way you treat latency. Log it per request, beside the duration, with the user and the endpoint attached. Then you can answer the question that is actually coming, which is not how much. It is who. Put a ceiling on one user. Per hour, per day, whatever suits the product. Decide today what happens when somebody reaches it, because if you do not decide, the answer is that everyone gets a degraded service funded by one person who wrote a loop. Look at what you are sending. Most prompts are mostly boilerplate, the same instructions on every single call, and half the context is there because somebody was unsure what mattered and included it all. Trimming that is not tidiness. It is the invoice. Cache the repeats. People ask the same question, in almost the same words, far more often than you expect. And stop sending everything to one model. Most of the traffic is easy. The expensive one is answering it only because that is what you happened to build with. None of this is premature. Cost is a property of the feature, exactly like response time and error rate, and it is the only one you learn about a month late, in a summary, from somebody who is not pleased. The meter is running now. Go and read it. – Serguey Asael Shinder