GC interruptions and TPL

Question

I have a WCF service. During the service's work, it needs to call two web services. So there's code similar to this:

var task1 = Task.Factory.StartNew(() => _service1.Run(query));
var task2 = Task.Factory.StartNew(() => _service2.Run(query));
Task.WaitAll(new[] { task1 , task2 });

Most of the time this works OK, but occasionally I was seeing spikes in the execution time, where the first task took a few seconds to even begin. Looking at perfmon, I realized this was exactly when GC was happening. Apparently, GC was a higher priority then running my tasks. This is not acceptable, as latency is very important to me, and I'd prefer GC to happen between requests and not in the middle of a request.

I attempted to go about this a different way, and instead of spinning my own tasks, I used WebClient.DownloadStringTask.

return webClient.DownloadStringTask(urlWithParmeters).ContinueWith(t => ProcessResponse(clientQuery, t.Result),
                                                                           TaskContinuationOptions.ExecuteSynchronously);

This didn't help; The GC now runs after the task began, but before the continuation. Again, I guess it figured the system is now idle, so it is a good time to begin GC. Only, I can't afford the latency.

Using TaskCreationOptions.LongRunning, which causes the scheduler to use non thread-pool threads, seems to solve this, but I don't want to create so many new threads - this code is going to run a lot (several times per request).

What is the best way to overcome this issue?

Answer 1

Let me first clean up some misunderstandings seen on this page:

• GC does not take place when idle. It takes place when triggered due to a failing allocation (new), GC.Collect or OS memory pressure
• The GC can stop application threads. It does not run concurrently (at least for a certain amount of time)
• "% time in GC" is a counter that does not change between GCs meaning that you might be seeing a stale value
• Async code does not help with GC problems. In fact, it generates more garbage (Tasks, IAsyncResult's and probably something else)
• Running your code on dedicated threads does not prevent them being stopped

How to fix this?

1. Generate less garbage. Attach a memory profiler (JetBrains is easy to use) and see what is generating garbage and what is on your heap
2. Reduce heap size to reduce pause time (A 3GB heap is probably due to some caching? Maybe shrink the cache?)
3. Start multiple ASP.NET sites with the same app, hook up GC notifications to sense a GC coming and take some of the IIS sites out of load balancing rotation while they are having a GC (http://blogs.msdn.com/b/jclauzel/archive/2009/12/10/gc-notifications-asp-net-server-workloads.aspx?Redirected=true)

You'll notice that there is no easy fix. I don't know one, but if the problem is caused by GC one of the above will fix the problem.

Answer 2

I know your question is about GC, but I'd like to start with talking about the async implementation first and then see if you're still going to be suffering the same issues.

Going off of your initial implementation's sample code, you're going to be wasting three CPU threads waiting for I/O right now:

• The first thread being wasted is the original WCF I/O thread that is executing the call. It's going to be blocked by the Task.WaitAll while the child Tasks are still outstanding.
• The other two threads that are being wasted are the thread pool threads you're using to execute the calls to Service1 and Service2

All that time while the I/O to Service1 and Service2 is outstanding the three CPU threads you're wasting are not able to be used to execute other work and the GC has to tip toe around them.

Therefore my initial recommendation would be to change your WCF method itself to use the Asynchronous Programming Model (APM) pattern that is supported by the WCF runtime. This solves the problem of the first wasted thread by allowing the original WCF I/O thread that calls into your service implementation to be returned to its pool immediately to be able to service other incoming requests. Once you've done that, you next want to make the calls to Service1 and Service2 asynchronous as well from the client perspectice. That would involve one of two things:

1. Generating async versions of their contract interfaces that, again, use the APM BeginXXX/EndXXX that WCF supports in the client model as well.
2. If these are simple REST services you're talking to, you have the following other async choices:
   • WebClient::DownloadStringAsync implementation (WebClient is not my fav API personally)
   • HttpWebRequest::BeginGetResponse + HttpWebResponse::BeginGetResponseStream + HttpWebRequest::BeginRead
   • Go bleeding edge with the new Web API's HttpClient

Putting all that together, there would be no wasted threads while you're waiting for a response from Service1 and Service2 in your service. The code would look something like this assuming you took a WCF client route:

// Represents a common contract that you talk to your remote instances through
[ServiceContract]
public interface IRemoteService
{
   [OperationContract(AsyncPattern=true)]
   public IAsyncResult BeginRunQuery(string query, AsyncCallback asyncCallback, object asyncState);
   public string EndRunQuery(IAsyncResult asyncResult);

}

// Represents your service's contract to others
[ServiceContract]
public interface IMyService
{
   [OperationContract(AsyncPattern=true)]
   public IAsyncResult BeginMyMethod(string someParam, AsyncCallback asyncCallback, object asyncState);
   public string EndMyMethod(IAsyncResult asyncResult);
}

// This would be your service implementation
public MyService : IMyService
{
    public IAsyncResult BeginMyMethod(string someParam, AsyncCallback asyncCallback, object asyncState)
    {
        // ... get your service instances from somewhere ...
        IRemoteService service1 = ...;
        IRemoteService service2 = ...;

        // ... build up your query ...
        string query = ...;

        Task<string> service1RunQueryTask = Task<string>.Factory.FromAsync(
            service1.BeginRunQuery,
            service1.EndRunQuery,
            query,
            null);

        // NOTE: obviously if you are really doing exactly this kind of thing I would refactor this code to not be redundant
        Task<string> service2RunQueryTask = Task<string>.Factory.FromAsync(
            service2.BeginRunQuery,
            service2.EndRunQuery,
            query,
            null);

        // Need to use a TCS here to retain the async state when working with the APM pattern
        // and using a continuation based workflow in TPL as ContinueWith 
        // doesn't allow propagation of async state
        TaskCompletionSource<object> taskCompletionSource = new TaskCompletionSource<object>(asyncState);

        // Now we need to wait for both calls to complete before we process the results
        Task aggregateResultsTask = Task.ContinueWhenAll(
             new [] { service1RunQueryTask, service2RunQueryTask })
             runQueryAntecedents =>
             {
                 // ... handle exceptions, combine results, yadda yadda ...
                 try
                 {
                     string finalResult = ...;

                     // Propagate the result to the TCS
                     taskCompletionSoruce.SetResult(finalResult);
                 }
                 catch(Exception exception)
                 {
                     // Propagate the exception to the TCS 
                     // NOTE: there are many ways to handle exceptions in antecedent tasks that may be better than this, just keeping it simple for sample purposes
                     taskCompletionSource.SetException(exception);
                 }
             });

         // Need to play nice with the APM pattern of WCF and tell it when we're done
         if(asyncCallback != null)
         {
             taskCompletionSource.Task.ContinueWith(t => asyncCallback(t));
         }

         // Return the task continuation source task to WCF runtime as the IAsyncResult it will work with and ultimately pass back to use in our EndMyMethod
         return taskCompletionSource.Task;
    }

    public string EndMyMethod(IAsyncResult asyncResult)
    {
        // Cast back to our Task<string> and propagate the result or any exceptions that might have occurred
        return ((Task<string>)asyncResult).Result;
    }
}

Once you've got that all in place, you will technically have NO CPU threads executing while the I/O with Service1 and Service2 is outstanding. In doing this, there are no threads for the GC to even have to worry about interrupting most of the time. The only time there will be actual CPU work happening now is the original scheduling of the work and then continuation on the ContinueWhenAll where you handle any exceptions and massage the results.