One of the biggest issues I've faced as a dev is actually knowing which LLM I need to use. And it's not just limited to the model accuracy, which is just one of the bazillion other factors. While I'm not dismissing it, you just can't use GPT-6 Astra for every single LLM call. It's kind of an untold rule if you use a lot of LLM calls in your application: you use the lighter models for interactions with the end user, while keeping the heavier ones for actual work.

With agentic workflows increasing every passing day, you can easily have 10s to 100s of LLM calls for just 1 part of the application, starting with data-processing to structural validation, from decision making to actual hallucination preventers and final responses. A good example is an art analysis application where the user inputs their artwork, and you point out issues and help them improve.

You will need LLM calls in every corner of the application. One call to actually find the gaps in the user's artwork, another to find its fixes, probably another to also find out the art style and offer suggestions, the list keeps on going...

So what actually stops someone from using the most-complex model for all of this? Is it the cost? Well, that is one of the reasons. But is it the only reason? Mostly no.

The most common reason is another important expense: time. The heavier a model is, the longer it takes to respond. It's understandable because there's obviously a lot of complex reasoning behind the scenes. As a developer, I thought time actually taken by models was the only important factor before deciding whether to use it in my applications or not, until I realized why we actually have multiple metrics to measure a stream, which prompted me to build Metricia as a public NPM package.

This post mainly details how I iterated through different versions, starting with the most naive one, and finally where I ended up, and also a quick working example so that other devs don't have to waste their time building another library from scratch.

The naive version

How Metricia parses a Gemini stream

Obviously, the simplest idea is to just find out how long it takes for an LLM call to come back with a response, right from the instant a request is sent, to the moment the response returns.

But there was a catch. If 2 models take the same total time, does it mean they are equal? Do they actually provide the same value to the user? Which is something most don't consider. You may think: well, they take roughly the same time to respond, so they're the same, right? But you see that your model takes 15 seconds to return a response, and then dumps 1k characters in your face, while another model starts to respond in just 1 second, and streams the same 1k characters in chunks over the next 14 seconds. Which one is the better choice?

And no, I'm not asking you to implement streaming in your application. It's already one of the basics of making LLM calls. And if your app doesn't use streaming, you're leaving UX on the table.

What if 1 model actually sends the first response at the 15th second? Then that's not your fault, right? Which is why we needed another metric along with total time - TTFT, the time-to-first-token, where you actually count the time it takes for the first carriage of the train to arrive, instead of the final compartment.

But does it solve the problem? Too bad it doesn't. What if a model sends the first token in the 500ms point, and then goes blank for the next 14 seconds, and sends you the rest of the response at the end of the 14th second? Which is why we needed another parameter - tokens per second.

This is the instant TPS: the tokens you have received in the past 1000ms. So, if the TPS is 0, you know something's off. An ideal LLM needs the TPS to be consistent throughout.

But this just gives us how many tokens you've received in the past second, what about where you actually stand at the present moment? Which is why I added the average TPS. It gives a clear picture of how the model is changing as it continues to stream.

I used a 1000ms sliding window to implement this approach, where metrics older than 1000ms get dropped, and the average TPS counts every token after the first chunk, divided by the time since the first token arrived.

So after bringing all of this together, and learning how to validate releases for an NPM package, and publishing it, here's how I use it now:

import { Metricia } from '@metricia/core'; const metricia = new Metricia(); metricia.on('metric', (metrics) => { console.log( `Tokens: ${metrics.totalTokens} | ` + `TPS: ${metrics.tokensPerSecondInstant.toFixed(1)}` ); }); metricia.on('complete', (summary) => { console.log(`Completed: ${summary.totalTokens} tokens`); }); metricia.on('error', ({ error, summary }) => { console.error(error); console.log(`Failed after ${summary.duration}ms`); }); const monitoredStream = metricia.monitor(stream); for await (const chunk of monitoredStream) { consumeChunk(chunk); }

Here, stream is the actual stream of the LLM, which can be Gemini, ChatGPT, Claude or anything you choose. The reason I separated the metric handling away from the actual chunks is because you can decide what to do with it: log it to the console, send it to your DB, find inconsistencies and ping you on Slack, or do everything at once. I isolated it so that it doesn't get in the way of your actual working.

Real Working Example

I used Metricia to track metrics for a simple prompt: "Explain JavaScript like I'm 5". I used the Gemini 2.5 Flash model for this. While testing, I encountered several bottlenecks and inconsistencies, which helped me to build the next version of the package (which is live rn). The issues are detailed in the next section. But, here is the final code snippet:

import { Metricia } from '@metricia/core'; import { GoogleGenAI } from '@google/genai'; const apiKey = process.env.GEMINI_API_KEY; if (!apiKey) { console.error('Set the GEMINI_API_KEY environment variable before running this script.'); process.exit(1); } const ai = new GoogleGenAI({ apiKey, httpOptions: { apiVersion: 'v1beta' }, }); const model = 'gemini-2.5-flash'; const prompt = "Explain JavaScript like I'm 5"; const metricia = new Metricia(); metricia.on('first-token', ({ ttft, ttftBasis }) => { console.log(`\n[first token] TTFT ${ttft.toFixed(0)}ms (${ttftBasis})`); }); metricia.on('metric', (metrics) => { console.log( `[metric] tokens=${metrics.totalTokens} ` + `tps(instant)=${metrics.tokensPerSecondInstant?.toFixed(1) ?? '-'} ` + `tps(avg)=${metrics.tokensPerSecondAverage?.toFixed(1) ?? '-'}` ); }); metricia.on('complete', (summary) => { console.log('\n[complete]', summary); }); metricia.on('abort', (summary) => { console.log('\n[abort]', summary); }); metricia.on('error', ({ error, summary }) => { console.error('\n[error]', error); console.log(summary); }); const stream = ai.models.generateContentStream({ model, contents: prompt }); const monitoredStream = metricia.monitor( stream ); console.log(`Prompt: ${prompt}\n`); console.log('Response: '); for await (const chunk of monitoredStream) { if (chunk.text) { console.log(chunk.text); } } console.log();

And here is the output:

Prompt: Explain JavaScript like I'm 5 Response: [first token] TTFT 6142ms (request) [metric] tokens=13 tps(instant)=- tps(avg)=- Okay, imagine a website is like a super cool coloring [metric] tokens=82 tps(instant)=230.3 tps(avg)=230.3 book on your computer. 1. **The Pictures:** When you first open the coloring book, you see all the drawings, right? Those are like the basic parts of a website (called HTML and CSS). They make things look pretty [metric] tokens=142 tps(instant)=212.7 tps(avg)=212.7 and stay in place. 2. **JavaScript is the Magic Pen!** Now, imagine you have a special magic pen. With this pen, you can make the drawings in your coloring book *do things*! * [metric] tokens=200 tps(instant)=204.4 tps(avg)=204.4 If you draw a button, the magic pen can make it so when you *tap* the button, a new picture suddenly **pops up!** * Or maybe you draw a little cat. The magic pen can make it so when you [metric] tokens=263 tps(instant)=209.5 tps(avg)=209.5 *touch* the cat, it **meows!** * You could even draw a car, and the magic pen makes it **drive across the page!** So, **JavaScript is the special magic that makes websites move [metric] tokens=305 tps(instant)=217.6 tps(avg)=220.5 , talk, change, and play with you when you click, tap, or move your mouse!** It makes the coloring book (website) come alive! [complete] { ttft: 6142.1739, ttftBasis: 'request', totalTokens: 305, bytesReceived: 0, textBytes: 957, duration: 7468.9962, avgTps: 220.07468520841115, averageTokensPerSecond: 220.07468520841115, status: 'completed' }

Here it is visualized:

Gemini 2.5 Flash TTFT with thinking on vs off: 6.1s vs 0.9s wait before the first token, 7.5s vs 2.3s total response time.

Some key observations which can be obtained from this example:

1. About 82% of the time was just waiting: TTFT was 6.14s, and all 305 tokens arrived in the last 1.33s. This was probably due to Gemini reasoning before it came up with an answer. So I turned reasoning off, and gave it another run, and here's the summary:

[complete] { ttft: 904.9793, ttftBasis: 'request', totalTokens: 312, bytesReceived: 0, textBytes: 957, duration: 2291.281, avgTps: 222.17386013448586, averageTokensPerSecond: 222.17386013448586, status: 'completed' }

The total time was cut by 5.2s, with the non-reasoning version returning the first token at 0.90s while the reasoning version returned at 6.14s. Now that's a lot of time wasted for a model called Flash!

2. Once the first token is received, rest of the chunks are superfast. The streaming rate is about 220 tokens per second, with or without thinking. So both have the same writing speed.

Tokens received over time for Gemini 2.5 Flash with thinking on and off, streamed in chunks after the first token arrives.

3. It streamed in just 6 chunks: Gemini sends its responses in major bursts rather than sending them token by token, so if you are planning to use this in an application, be prepared to have a turbulent outburst rather than a laminar flow.

I know this was a simple example, but this was enough to help me understand how usable the SDK was, and what changes I needed to implement in order to ensure that this is usable immediately after installing, and the end-user doesn't have to read through a pile of docs.

Once this was done, I went on to do what I actually built this SDK for: cross-vendor comparisons. So I took a single prompt and ran it across Gemini, ChatGPT and Claude models, not once, but 20 times each, and the results were pretty interesting to say the least. I am writing that as a separate article, since it contains a plethora of information, and it drifts apart from the true purpose of this post.

Anyway, I mainly encountered 4 issues which I needed to take a decision about, and most of them were, interestingly, around the same area: time.

Issue #1: How to actually measure time?

The biggest discovery while building the above example wasn’t a bug I saw, but one waiting to happen: the way I was measuring time could silently go wrong, by a few milliseconds.

A few milliseconds may not seem big. But when you actually incorporate it into large-scale heavy use applications, every 10th of a second counts. Ask a SQL developer what happens if you don't use indexing in a table, and then run a search on a table of 1 million records. The response you get would be a better explanation as to why shaving every microsecond matters. Pro Tip: Wait until they are in a good mood to ask this question 😆.

So I actually went through a few Stack Overflow threads and finally discovered that the problem was in the timing system which I was using. Turns out, using Date.now() for the clock was where everything went wrong. My chain-of-thought was that I've used the Date function for countless products I had built before, so that would work here as well. But the problem with Date.now() is that it measures the system time. So if you knowingly or unknowingly set the system clock back by even 2 seconds, the results would be inaccurate.

The right alternative is to use performance.now(). It is a monotonic timer, which means it can only count forward, and can also count sub-milliseconds. Which means we can now get decimal based millisecond values, which was not possible while using Date.now().

Issue #2: Stream vs Promise of the Stream

Just as I thought everything was cool, I see another inconsistency. The first TPS was around 5000 for an example I ran, while the following tps were in hundreds. I ran them multiple times, and realized that it was another bug which I needed to solve.

A bit of digging and experimenting with different types of streaming objects and the ways in which they are handled, I was able to put a finger on the cause. It was because the awaited object was being passed to the monitor instead of the promise of the object.

Now what happens when you do this? If you pass an await streaming object, the Metricia clock starts to count/measure only when we actually hear back from the LLM in the first place. Which means the TTFT would show single-digit milliseconds.

So the problem wasn't in the TPS, but rather in the TTFT, which actually showed values such as 4 or 5, so starting to measure from that point would result in the tokens being divided by a very small number, leading to large values for the first TPS.

So the first thing I did was make the initial TPS null as a stopgap. Then I changed the library to accept the actual object instead of the awaited one.

Issue #3: Users will pass the awaited stream anyway

This was another problem. While the fix above got the library working as intended, I cannot expect the average user to actually know not to pass the awaited object.

That too with the fact that most LLM model providers use await in their docs and examples, which meant people can pass it here as well. And if there is something I've learned through these years, it's that "Never assume the user knows everything". So I added a console warning, and also set it to return null whenever the wrong type of object is passed, and also set the first few TPS to null in those cases.

Is it a bit of a stretch? Yes. Does that mean I have to use 2 formulae? Yes. But it does give the users a bit of flexibility to still use it the old way if they want to, so it's a win for me.

Issue #4: How to actually measure the average TPS

So, in the cases where people end up passing the await object, from which point should the second average TPS start to count, from the beginning of the actual request, or actually measure it from the point the first token arrived?

The problem with option 1 is that the waiting time before the first token gets mixed into the writing speed, so the average comes out lower than how fast the model actually streams. So I went with option 2: average TPS only counts from the point where the first token arrived.

What I'd do if I had more time

The one thing which I did not yet implement due to time constraints, is the multi-vendor token adapter. Right now, Metricia estimates the response's tokens with a simple rule, which is usually close to the actual count. It works by splitting the text into streams of letters or numbers, and then if the word is shorter than 4 characters, it counts as 1 token. If not, it takes the ceiling of wordLength/3.5. But still, each vendor has their own way of splitting text into tokens, so I have added the option to pass custom tokenizers to the SDK. The bare minimum is just a name for the tokenizer and a function for the tokenizer, and you can pass your own tokenizer. My next job would be to start building adapters for different models myself, so that users won't have to do it themselves.

Conclusion

So this was all about Metricia: the NPM package I built to solve my own problem, and the problems I faced while solving this problem, and how I solved those as well. Metricia is available as @metricia/core on NPM, and can be installed using the command npm install @metricia/core . The homepage is here: Metricia: see inside your AI streams, and the docs are here: Introduction · Metricia docs