Here's a bug that never throws, never shows up in a log, and never fails a test you'd think to write: the user presses Stop, the text freezes on screen, and the model keeps writing anyway.

In my own test setup (a simulated model behind the real OpenAI .NET SDK), a "works fine in the demo" SignalR hub kept generating 263 more tokens after Stop, held the provider stream open for 5.3 seconds, and saved 1,672 characters to the conversation while the user had seen 200. Nothing crashed. The screen looked right. The bill and the chat history didn't.

The fix isn't a library. It's making sure one CancellationToken gets from the browser's Stop button all the way to the HTTP response from the model provider, and that nothing in between quietly swaps it out. This post walks through the five places I've seen it get dropped.

Tech Skill Builder is $5/month for a very limited time only (normally $10). Grab it now before it goes back to $10. The complete, tested .NET 10 project behind this post (SignalR streaming with Clean Architecture, 187 passing tests, running offline against a simulator) plus a step-by-step PDF guide is inside. Join Tech Skill Builder

About the numbers: they're local measurements with a simulated model (400 ms to the first token, then 20 ms per token). They show how the code behaves, not how fast any real model is.

The path the token has to survive

Before the leaks, the happy path. Stop has to travel like this:

browser Stop → SignalR stream cancel → hub method token → your use case → IChatClient.GetStreamingResponseAsync → the provider's HTTP response

Every arrow is a hand-off where someone can pass CancellationToken.None, start a new task, or catch the wrong exception. If any single arrow breaks, the screen still stops (the client just stops reading), so you never notice.

Leak 1: The detached producer

This is the version most of us write first, because it's what a lot of older examples show: start generation in a background task, push tokens into a channel, return the reader.

// ANTI-PATTERN: shown so you can recognise it, not copy it. public ChannelReader<ReplyEventDto> StreamReply(StreamReplyRequest request) { var channel = Channel.CreateUnbounded<ReplyEventDto>(); _ = Task.Run(async () => { // CancellationToken.None: nothing the client does can reach the model now await foreach (var e in streamer.StreamAsync(command, CancellationToken.None)) await channel.Writer.WriteAsync(e.ToDto()); channel.Writer.TryComplete(); }); return channel.Reader; } 

When the client cancels, SignalR stops reading the channel. The background task doesn't care. It runs until the model finishes the whole answer. That's where my 263 tokens came from.

The fix: make the hub method itself an async iterator and let SignalR hand you the token.

public async IAsyncEnumerable<ReplyEventDto> StreamReply( StreamReplyRequest request, [EnumeratorCancellation] CancellationToken cancellationToken) { // pass cancellationToken (and Context.ConnectionAborted) into the use case, // map domain rejections to HubException, yield each event as a DTO // ... full implementation in the complete project yield break; } 

A nice detail I confirmed by reading the SignalR source: that stream token is linked to ConnectionAborted, so closing the tab cancels it too. You get two exits for the price of one parameter.

Leak 2: Listening to only one reason to stop

"The user pressed Stop" isn't the only reason a reply should end. In a real chat you usually have at least three:

  • the client cancelled the stream,
  • the user pressed Stop in another tab (or on their phone),
  • the server's own budget ran out (time or length), for clients that never press Stop at all.

If the model call only listens to the first one, the other two are decoration. Link them into one token and pass that down:

using var budgetCts = new CancellationTokenSource(budget.MaxDuration, timeProvider); using var linked = CancellationTokenSource.CreateLinkedTokenSource( cancellationToken, // this stream was cancelled lease.StopRequested, // Stop came from somewhere else budgetCts.Token); // server-side limit // every call to the model from here on gets linked.Token // ... loop, batching and bookkeeping omitted; full version in the complete project 

Keep the individual sources around, though. When the stream ends, check which one fired and store a status like Stopped, Disconnected, StoppedElsewhere or BudgetExceeded. Six weeks later, when someone asks "why is this answer cut off?", that column answers it. (Passing a TimeProvider into the budget also means the time-limit test can run on FakeTimeProvider in milliseconds instead of waiting for a real timeout.)

Leak 3: The catch that eats the cancellation

The model stream is wrapped in error handling, which is good. Then someone writes this:

try { // ... read the next update from the model } catch (Exception ex) // swallows OperationCanceledException too { logger.LogWarning(ex, "Model hiccup, carrying on"); } 

Now a cancellation looks like a transient error. Depending on what comes next, you retry, keep reading, or fall through to "save the answer". Either way, Stop has been turned into a log line.

Two rules fix it:

  1. Handle cancellation first and explicitly: catch (OperationCanceledException) when (token.IsCancellationRequested) means "we were asked to stop", and it ends the loop.
  2. Dispose the model stream in a finally. Disposing the enumerator you got from GetStreamingResponseAsync disposes the HTTP response under it, and that's the moment the provider stops generating for you.
try { // ... pull deltas, batch them, yield them to SignalR } finally { // first line in finally, before saving or logging anything await modelStream.DisposeAsync(); // ... classify why we stopped, then persist (see Leak 4) } 

With the token linked through and the stream disposed in finally, the provider stream in my runs closed 8 ms after Stop at the median (p95 10 ms), and 0 tokens were generated after it. Closing the tab instead closed it in under 1 ms at the median.

When you review code for this leak, three things deserve a second look: CancellationToken.None, Task.Run without a token, and any catch (Exception) near the model call. Each looks harmless in a diff.

Leak 4: Saving with the token that just got cancelled

You've stopped the model. Now you want to save the partial reply. The obvious code passes the same request token to the database call. But that token is cancelled, which is exactly why you're in this code path. So the save gets cancelled too, and the partial answer silently disappears.

Give the save its own short budget instead (I use 5 seconds):

private async Task PersistAsync(ChatTurn turn) { // The request token is already cancelled here, so saving gets its own timeout. using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5), timeProvider); // ... await store.AppendAsync(turn, cts.Token), log (don't throw) on failure // full implementation in the complete project } 

And save the right text. The naive hub saved everything the model generated: 1,672 characters for a user who saw 200. The next prompt then gets built on paragraphs that never reached the screen, and the model starts saying "as I mentioned above" about things the user never read.

The pattern that fixed it for me is a tiny ledger that keeps two counters apart:

public sealed class ReplyLedger { public int GeneratedChars { get; private set; } // what the model produced public string SentText { get; } // what actually went to the client public void Generated(string delta) { /* ... */ } public void Sent(string chunk) { /* ... */ } // full implementation in the complete project } 

A chunk only counts as sent when SignalR asks for the next item, which it does after writing the previous one. Store SentText, add a short "[reply stopped by the user]" note when you build the next prompt, and the stored history matches the screen again: in my fixed runs, saved text equalled seen text (200 = 200).

Bonus: GeneratedChars - SentChars is a waste number you can log on every reply. Put it on a dashboard and Leak 1 can never come back quietly.

Leak 5: Treating a network blip like a Stop

This one is the reverse problem. The user didn't press anything. Their phone switched from Wi-Fi to cellular, the WebSocket died for a moment, and your server saw a disconnect. Correctly, by everything above, it cancelled the model. The answer is gone, the user asks again, and you pay for it twice.

SignalR in .NET has a built-in answer for short drops: stateful reconnect. Both sides buffer messages and replay them after a brief drop, on the same connection id, so the hub method never even notices.

// Server app.MapHub<ChatHub>("/hubs/chat", o => o.AllowStatefulReconnects = true); // .NET client (in JavaScript it's .withStatefulReconnect()) var connection = new HubConnectionBuilder() .WithUrl(url) // auth options omitted .WithStatefulReconnect() .Build(); 

I tested it with a small TCP proxy that resets every socket 1.5 seconds into an answer. Without stateful reconnect, 0 of 10 answers completed. With it, 10 of 10 did, on the same connection id, with the longest pause around 86 ms at the median.

Two limits worth knowing: it only works over WebSockets, and it only covers short drops. For a long outage you still want automatic reconnect on the client, and a server that saves the reply as Disconnected (see Leak 2) rather than pretending it completed.

One more trap: retrying after the first token

Not a token leak exactly, but it lives in the same loop. Retrying a 503 before any text arrives is free. Retrying after the model has already sent 40 words makes it start the answer again. The user sees the opening sentence twice and you pay twice. Retry before the first token only, in one layer. If the SDK and your HTTP pipeline both retry, three attempts quietly become nine.

A checklist before you ship a Stop button

  • The hub method returns IAsyncEnumerable<T> with an [EnumeratorCancellation] token. No detached Task.Run.
  • Stop, stop-from-another-tab and the server budget are linked into one token that reaches GetStreamingResponseAsync.
  • OperationCanceledException is handled explicitly, never swallowed by catch (Exception).
  • The model stream is disposed in finally, before anything else.
  • The partial reply is saved with its own timeout, and it's the sent text, not the generated text.
  • Every reply logs generated vs sent characters and a stop reason.
  • Stateful reconnect is on, on both server and client.
  • Retries happen before the first token only.
  • A test counts tokens generated after Stop and expects zero.

That last one is the test nobody writes, and it's the one that would have caught all of this. With a simulator that streams real OpenAI-format bytes, you can run it offline in CI.

What's in the complete project

The snippets above are skeletons: shapes and signatures, with the bodies left out on purpose. The complete project in Tech Skill Builder is a working .NET 10 service (Domain / Application / Infrastructure / Api, plus unit and integration tests, 187 passing) with:

  • the streaming hub and the use case with the linked token, finally disposal and stop-reason statuses,
  • the sent-vs-generated ledger, history building and the separate save budget,
  • first-token-only retries, a server budget and one-live-reply-per-conversation admission,
  • token batching that sends the first token immediately (200 → 51 messages per reply in my 100-client run),
  • IChatClient from Microsoft.Extensions.AI so OpenAI, Azure OpenAI or Ollama can plug in, and a simulator so everything runs offline,
  • the naive hub kept on purpose for the measurements, plus the benchmark with the TCP chaos proxy,
  • a PDF guide that walks through the architecture and every decision.

Get the complete, working and tested project in Tech Skill Builder: $5/month for a very limited time only (normally $10). Grab it now before it goes back to $10.

What does your chat save when someone presses Stop halfway through: the partial text, the full answer, or nothing? I'm curious which one you picked and what it broke.