HttpClient with infinite time out throws time out exception

Question

My HttpClient uses digest authentication to connect to the server and expects search queries in response. These search queries can come in any time so the client is expected to leave the connection open at all times.

The connection is made using the following code:

public static async void ListenForSearchQueries(int resourceId)
{
    var url = $"xxx/yyy/{resourceId}/waitForSearchRequest?token=abc";

    var httpHandler = new HttpClientHandler { PreAuthenticate = true };

    using (var digestAuthMessageHandler = new DigestAuthMessageHandler(httpHandler, "user", "password"))
    using (var client = new HttpClient(digestAuthMessageHandler))
    {
        client.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite);

        var request = new HttpRequestMessage(HttpMethod.Get, url);

        var tokenSource = new CancellationTokenSource();
            tokenSource.CancelAfter(TimeSpan.FromMilliseconds(Timeout.Infinite));

        using (var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, tokenSource.Token))
        {
            Console.WriteLine("\nResponse code: " + response.StatusCode);

            using (var body = await response.Content.ReadAsStreamAsync())
            using (var reader = new StreamReader(body))
                while (!reader.EndOfStream)
                    Console.WriteLine(reader.ReadLine());
         }
    }
}

This is how the method is being used in the Main method of a console application.

private static void Main(string[] args)
{
   const int serviceId = 128;
   .
   .
   .
   ListenForSearchQueries(resourceId);
   Console.ReadKey();
}

This is what the output on the console window looks like:

Response code: OK
--searchRequestBoundary

Even though the timeout for the client is set to infinity, the connection times out after roughly five minutes (which is not the default timeout of the HttpClient) after the first output, throwing the following exception.

System.IO.IOException occurred
  HResult=0x80131620
  Message=The read operation failed, see inner exception.
  Source=System.Net.Http
  StackTrace:
   at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.Http.DelegatingStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.StreamReader.ReadBuffer()
   at System.IO.StreamReader.get_EndOfStream()
   at ConsoleTester.Program.<ListenSearchQueriesDigestAuthMessageHandler>d__10.MoveNext() in C:\Users\xyz\ProjName\ConsoleTester\Program.cs:line 270

Inner Exception 1:
WebException: The operation has timed out.

The DelegateHandler used for the authentication is a a rough adaption of this code (see the source section).

Why is the client timing out and how can I prevent this?

My ultimate goal is to make a call and wait indefinitely for a response. When a response does come, I don't want the connection to close because more responses might come in the future. Unfortunately, I can't change anything at the server end.

Answer 1

Although the default value for Stream.CanTimeout is false, returning a stream via the response.Content.ReadAsStreamAsync() gives a stream where the CanTimeout property returns true.

The default read and write time out for this stream is 5 minutes. That is after five minutes of inactivity, the stream will throw an exception. Much similar to the exception shown in the question.

To change this behavior, ReadTimeout and/or the WriteTimeout property of the stream can be adjusted.

Below is the modified version of the ListenForSearchQueries method that changes the ReadTimeout to Infinite.

public static async void ListenForSearchQueries(int resourceId)
{
    var url = $"xxx/yyy/{resourceId}/waitForSearchRequest?token=abc";

    var httpHandler = new HttpClientHandler { PreAuthenticate = true };

    using (var digestAuthMessageHandler = new DigestAuthMessageHandler(httpHandler, "user", "password"))
    using (var client = new HttpClient(digestAuthMessageHandler))
    {
        client.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite);

        var request = new HttpRequestMessage(HttpMethod.Get, url);

        var tokenSource = new CancellationTokenSource();
            tokenSource.CancelAfter(TimeSpan.FromMilliseconds(Timeout.Infinite));

        using (var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, tokenSource.Token))
        {
            Console.WriteLine("\nResponse code: " + response.StatusCode);

            using (var body = await response.Content.ReadAsStreamAsync())
            {
                body.ReadTimeout = Timeout.Infinite;

                using (var reader = new StreamReader(body))
                    while (!reader.EndOfStream)
                        Console.WriteLine(reader.ReadLine());
            }
         }
    }
}

This fixed the exception which was actually being thrown by the stream but seemed like was being thrown by the HttpClient.