Why is my async/await with CancellationTokenSource leaking memory?

Question

I have a .NET (C#) application that makes extensive use of async/await. I feel like I've got my head around async/await, but I'm trying to use a library (RestSharp) that has an older (or perhaps I should just say different) programming model that uses callbacks for asynchronous operations.

RestSharp's RestClient class has an ExecuteAsync method that takes a callback parameter, and I wanted to be able to put a wrapper around that which would allow me to await the whole operation. The ExecuteAsync method looks something like this:

public RestRequestAsyncHandle ExecuteAsync(IRestRequest request, Action<IRestResponse> callback); 

I thought I had it all working nicely. I used TaskCompletionSource to wrap the ExecuteAsync call in something that I could await, as follows:

public static async Task<T> ExecuteRequestAsync<T>(RestRequest request, CancellationToken cancellationToken) where T : new() {     var response = await ExecuteTaskAsync(request, cancellationToken);      cancellationToken.ThrowIfCancellationRequested();      return Newtonsoft.Json.JsonConvert.DeserializeObject<T>(response.Content); }  private static async Task<IRestResponse> ExecuteTaskAsync(RestRequest request, CancellationToken cancellationToken) {     var taskCompletionSource = new TaskCompletionSource<IRestResponse>();      var asyncHandle = _restClient.ExecuteAsync(request, r =>      {         taskCompletionSource.SetResult(r);      });      cancellationToken.Register(() => asyncHandle.Abort());      return await taskCompletionSource.Task; } 

This has been working fine for most of my application.

However, I have one part of the application that does hundreds of calls to my ExecuteRequestAsync as part of a single operation, and that operation shows a progress dialog with a cancel button. You'll see that in the code above that I'm passing a CancellationToken to ExecuteRequestAsync; for this long-running operation, the token is associated with a CancellationTokenSource "belonging" to the dialog, whose Cancel method is called if the use clicks the cancel button. So far so good (the cancel button does work).

My problem is that my application's memory usage shoots up during the long-running application, to the extent that it runs out of memory before the operation completes.

I've run a memory profiler on it, and discovered that I have lots of RestResponse objects still in memory, even after garbage collection. (They in turn have huge amounts of data, because I'm sending multi-megabyte files across the wire).

According to the profiler, those RestResponse objects are being kept alive because they're referred to by the TaskCompletionSource (via the Task), which in turn is being kept alive because it's referenced from the CancellationTokenSource, via its list of registered callbacks.

From all this, I gather that registering the cancellation callback for each request means that the whole graph of objects that is associated with all those requests will live on until the entire operation is completed. No wonder it runs out of memory :-)

So I guess my question is not so much "why does it leak", but "how do I stop it". I can't un-register the callback, so what can I do?

Answer 1

I can't un-register the callback

Actually, you can. The return value of Register() is:

The CancellationTokenRegistration instance that can be used to deregister the callback.

To actually deregister the callback, call Dispose() on the returned value.

In your case, you could do it like this:

private static async Task<IRestResponse> ExecuteTaskAsync(     RestRequest request, CancellationToken cancellationToken) {     var taskCompletionSource = new TaskCompletionSource<IRestResponse>();      var asyncHandle = _restClient.ExecuteAsync(         request, r => taskCompletionSource.SetResult(r));      using (cancellationToken.Register(() => asyncHandle.Abort()))     {         return await taskCompletionSource.Task;     } }