Menu
I have a hard time figuring out if there is a way to handle potential connectivity problems when using .NET's HttpWebRequest class to call a remote server (specifically a REST web service). From my investigations the behaviour of the WebClient class is the same, which is somewhat expected since it appears to only offer a more simple interface to the HttpWebRequest.
For simulation purposes, I've written a very simple HTTP server that does not behave according to the HTTP 1.1 RFC. What it does is it accepts a client connection, then sends appropriate HTTP 1.1 headers and a "Hello World!" payload back to the client and closes the socket, the thread accepting client connections on the server side looks as follows:
private const string m_defaultResponse = "<html><body><h1>Hello World!</h1></body></html>";
private void Listen()
{
while (true)
{
using (TcpClient clientConnection = m_listener.AcceptTcpClient())
{
NetworkStream stream = clientConnection.GetStream();
StringBuilder httpData = new StringBuilder("HTTP/1.1 200 OK\r\nServer: ivy\r\nContent-Type: text/html\r\n");
httpData.AppendFormat("Content-Length: {0}\r\n\r\n", m_defaultResponse.Length);
httpData.AppendFormat(m_defaultResponse);
Thread.Sleep(3000); // Sleep to simulate latency
stream.Write(Encoding.ASCII.GetBytes(httpData.ToString()), 0, httpData.Length);
stream.Close();
clientConnection.Close();
}
}
}
Since the HTTP 1.1 RFC states that HTTP 1.1 by default keeps connections alive and that a server must send a "Connection: Close" response header if it wants to close a connection this is unexpected behaviour for the client-side. The client uses HttpWebRequest in the following way:
private static void SendRequest(object _state)
{
WebResponse resp = null;
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://192.168.0.32:7070/asdasd");
request.Timeout = 50 * 1000;
DateTime requestStart = DateTime.Now;
resp = request.GetResponse();
TimeSpan requestDuration = DateTime.Now - requestStart;
Console.WriteLine("OK. Request took: " + (int)requestDuration.TotalMilliseconds + " ms.");
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.Timeout)
{
Console.WriteLine("Timeout occurred");
}
else
{
Console.WriteLine(ex);
}
}
finally
{
if (resp != null)
{
resp.Close();
}
((ManualResetEvent)_state).Set();
}
}
The above method is queued via ThreadPool.QueueUserWorkItem(waitCallback, stateObject). The ManualResetEvent is used to control queuing behavior so that not the entire thread pool gets filled up with waiting tasks (since the HttpWebRequest implicitly uses worker threads because it functions asynchronously internally to implement the timeout functionality).
The problem with all this is that once all connections of the HttpWebRequest's underlying ServicePoint are "used up" (i.e. closed by the remote server), there will be no new ones opened up. It also does not matter if the ConnectionLeaseTimeout of the ServicePoint is set to a low value (10 seconds). Once the system gets into this state, it will no longer function properly because it does not reconnect automatically and all subsequent HttpWebRequests will time out. Now the question really is if there is a way to solve this by somehow destroying a ServicePoint under certain conditions or closing underlying connections (I did not have any luck with ServicePoint.CloseConnectionGroup() yet, the method is also pretty undocumented in terms of how to properly use it).
Does anybody have any idea how I could approach this problem?
432
The solution I came up with based on some of the ideas here is to manage the connections myself. If a unique ConnectionGroupName is assigned to a WebRequest (e.g. Guid.NewGuid().ToString()), a new connection group with one connection will be created in the ServicePoint for the request. Note that there's no more connection limiting at this point, since .NET limits per connection group rather than per ServicePoint, so you'll have to handle it yourself. You'll want to reuse connection groups so that existing connections with KeepAlive are reused, but if a WebException exception occurs, the request's connection group should be destroyed since it might be stale. Something like this (create a new instance for each host name):
public class ConnectionManager {
private const int _maxConnections = 4;
private Semaphore _semaphore = new Semaphore(_maxConnections, _maxConnections);
private Stack<string> _groupNames = new Stack<string>();
public string ObtainConnectionGroupName() {
_semaphore.WaitOne();
return GetConnectionGroupName();
}
public void ReleaseConnectionGroupName(string name) {
lock (_groupNames) {
_groupNames.Push(name);
}
_semaphore.Release();
}
public string SwapForFreshConnection(string name, Uri uri) {
ServicePoint servicePoint = ServicePointManager.FindServicePoint(uri);
servicePoint.CloseConnectionGroup(name);
return GetConnectionGroupName();
}
private string GetConnectionGroupName() {
lock (_groupNames) {
return _groupNames.Count != 0 ? _groupNames.Pop() : Guid.NewGuid().ToString();
}
}
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
