Stayin’ Alive: Harnessing Custom Cancellations for Reliable Streaming

Pic credits: DALL.E

Note - This post focuses on solving a particular problem with custom cancellation tokens and not on basics understanding of it. I'd highly recommend to go through it first (my recommendation - this video). Also, this post uses C# .Net but the concepts are agnostic.

You are building a consumer application which get the responses from server in a streaming fashion. Think of AI agents running on the server and each providing their own reasoning until you get the final answer. You are reading the event types and accordingly showing it to the users to maintain the engagement and perceived performance.

The challenge? When building conversational or real-time streaming applications, you often deal with responses that trickle in at irregular intervals. To keep your end-users engaged, you might show typing indicators or partial responses, creating a more interactive feel.

While this might not be as much challenging when you have complete control on the UI built with modern day frameworks like ReactJS, it does become a challenge while integrating with third-party apps like Microsoft custom engine Copilot for example, where you have limited control and adheres to its own strict time-outs etc.

Solution? Create your own custom CancellationTokenSource .

Here's how you can do it step-by-step -

1. Setting the stage: The keep-alive timeout

var keepAliveTimeout = TimeSpan.FromMinutes(30);

This means if no event arrives from the server for 30 minutes (configurable), we assume something is wrong (like a network issue or an unresponsive server). We then cancel the streaming operation.

2. Creating a Custom Cancellation Token Source

using var customCancellationTokenSource = new CancellationTokenSource();

This line creates our custom token source. We’ll be able to call customCancellationTokenSource.Cancel() on it at will. This is different from using a pre-existing token from an external caller. It ensures we have full control over when or why the streaming operation is canceled.

3. Starting a Keep-Alive Timer

	using var keepAliveTimer = new Timer(_ => 
	{
		logger.LogInformation($"No events received within keep-alive 	timeout, cancelling..."); 													  customCancellationTokenSource.Cancel();
	}, null, Timeout.Infinite, Timeout.Infinite);

• Timer callback: If the timer elapses (i.e., the wait time runs out), we log a message and cancel our custom token source.

• Timeout.Infinite: Initially, the timer isn’t running. We’ll explicitly start or reset it each time we receive new data from the server.

This timer is a safeguard. If 30 minutes pass without new data, we assume the connection is stale and forcefully stop the streaming process.

4. Sending the HTTP Request and Reading the Stream

using (HttpResponseMessage response = await httpClient.SendAsync(
	request,
	HttpCompletionOption.ResponseHeadersRead,
	customCancellationTokenSource.Token))
{
	using (Stream stream = await 	response.Content.ReadAsStreamAsync(customCancellationTokenSource.Token))
	{
		using (StreamReader reader = new(stream))
		{
			// Start keep-alive timer
			if (keepAliveTimeout != Timeout.InfiniteTimeSpan)
			{
				ResetKeepAliveTimer(keepAliveTimer, keepAliveTimeout);
			}
			
			try
			{
				while (!reader.EndOfStream && !customCancellationTokenSource.Token.IsCancellationRequested)
				{
					....
					// Process as per event types, use typing indicators and keep pass customCancellationTokenSource.Token in each function.
				}
			}
		}
	}
}

• HttpCompletionOption.ResponseHeadersRead: We only want to read the response headers before continuing; we expect the content to stream over time.

• Passing customCancellationTokenSource.Token: This ensures that if our token is canceled (e.g., by the keep-alive timer), reading from the stream will stop gracefully.

• Reading line by line: The server may send event and data lines, typical in streaming scenarios like Server-Sent Events.

5. Resetting the Timer with Each Event

   In the code, each time we successfully read new data from the server, we reset the timer:

       private static void ResetKeepAliveTimer(Timer keepAliveTimer, 	TimeSpan keepAliveTimeout)
	{
		if (keepAliveTimeout != Timeout.InfiniteTimeSpan)
		{
			keepAliveTimer.Change(keepAliveTimeout, Timeout.InfiniteTimeSpan);
		}
	}

This means:

• We received data → The connection is still alive.

• Reset the timer → Push out the cancellation deadline another 30 minutes.

6. Handling Cancellation and Cleanup

catch (OperationCanceledException ex)
{
	logger.LogCritical(ex, Operation canceled, stopping reading.");
}
catch (ObjectDisposedException ex)
{
	logger.LogCritical(ex, Stream or HttpClient disposed, stopping 	reading.");
}
finally
{
	// Stop the timer to clean up
	keepAliveTimer.Change(Timeout.Infinite, Timeout.Infinite);
}

Why Use a Custom CancellationToken?

• Fine-grained control: You can decide exactly when to cancel, independent of other system-level tokens.

• Keep-alive timeouts: You can attach specialized logic (like a Timer) to this single token source that handles your streaming requirements.

• Graceful shutdown: By passing this token to every asynchronous call within your stream-reading loop (HTTP, reading, writing activities), you ensure that any signal to cancel will propagate through all these calls, preventing partial or orphaned operations.

Long-running or unpredictable streams don’t have to be daunting. By pairing a keep-alive timer with a custom CancellationToken, you’ve got all the power to detect stalled connections, gracefully exit when needed, and keep users actively engaged along the way.

As with any great performance, knowing how to take a bow when the show is over is just as important as dazzling your audience during the act. So, stay alive—and keep streaming on!