.NET HTTP Request Timeout not "Aborting" on continuous streaming site

Question

Scope

I have developed a Wrapper for the Http Web Request class to execute Gets and Posts in an easier way. This library is being used without any problem for over two years already, but today, we faced a weird problem.

There's one website (it will take a while to load), that apparently, keeps streaming characters to the HTML frontend, so our library "Get" request gets stuck into a loop.

Different Timeout Properties

Looking at the Http Web Request reference we find that there are two different timeout properties. We are using both of them, but none of them seem to abort properly.

• Read Write Timeout - Gets or sets a time-out in milliseconds when writing to or reading from a stream.
• Timeout - Gets or sets the time-out value in milliseconds for the GetResponse and GetRequestStream methods.

One Timeout to Rule them all

Is there any way to set an "Operation Timeout" that will force the stream/connection disposing after some pre-configured time?

Some Code Sample

public class Program
{
public static void Main()
{
    string url = "http://anus.io";

    Console.WriteLine("Initializing Request State Object");

    RequestState myRequestState = new RequestState();

    // Creating Http Request
    myRequestState.request  =  (HttpWebRequest)WebRequest.Create(url);
    myRequestState.request.Method           = "GET";
    myRequestState.request.ReadWriteTimeout = 4000;
    myRequestState.request.Timeout          = 4000;

    Console.WriteLine("Begining Async Request");


    IAsyncResult ar = myRequestState.request.BeginGetResponse(new AsyncCallback(ResponseCallback), myRequestState);

    Console.WriteLine("Waiting for Results");
    ar.AsyncWaitHandle.WaitOne();

    myRequestState.response = (HttpWebResponse)myRequestState.request.EndGetResponse(ar);
    Console.WriteLine("Response status code = {0}", myRequestState.response.StatusCode);

}

public static void ResponseCallback (IAsyncResult asyncResult)
{
    Console.WriteLine("Completed");
}
}

This Code seem to work fine, but how do i read the response as string, since it is being constantly transmited (as the browser shows)?

Question

What's the proper way to handle websites like this ? (By handle I mean to be able to identify and skip situations where the server simply won't stop transmitting?)

Answer 1

Try this:

public static void Main()
{

    string url = "http://anus.io";

    ***int DefaultTimeOut = 15000;*** //timeout after 15 sec

    Console.WriteLine("Initializing Request State Object");

    RequestState myRequestState = new RequestState();

    // Creating Http Request
    myRequestState.request  =  (HttpWebRequest)WebRequest.Create(url);
    myRequestState.request.Method           = "GET";
    myRequestState.request.ReadWriteTimeout = 4000;
    myRequestState.request.Timeout          = 4000;

    Console.WriteLine("Begining Async Request");


    IAsyncResult ar = myRequestState.request.BeginGetResponse(new AsyncCallback(ResponseCallback), myRequestState);

    ***ThreadPool.RegisterWaitForSingleObject(ar.AsyncWaitHandle, new WaitOrTimerCallback(TimeoutCallback), myRequestState.request, DefaultTimeOut, true);***

    Console.WriteLine("Waiting for Results");
    ar.AsyncWaitHandle.WaitOne();

    try{

    myRequestState.response = (HttpWebResponse)myRequestState.request.EndGetResponse(ar);
    Console.WriteLine("Response status code = {0}", myRequestState.response.StatusCode);
    }catch(Exception){
       Console.WriteLine("Request Aborted");
    }
}

public static void ResponseCallback (IAsyncResult asyncResult)
{
    Console.WriteLine("Completed");
}

//call the timeout if more than 15 seconds

**public static void TimeoutCallback(object state, bool timedOut)
{
            if (timedOut)
            {
                HttpWebRequest request = state as HttpWebRequest;
                if (request != null)
                {
                    request.Abort();
                }
            }
}**