Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

HttpWebRequest keep alive - how is the connection reused by .net?

Hy, I am using HttpWebRequest in 10 concurent threads to download a list of images. I sorted the images after the hostName so each of this threads are getting an image from the same Hostname.

HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
myReq.ServicePoint.ConnectionLimit = 10;
myReq.Timeout = 2000;
myReq.KeepAlive = true;

HttpWebResponse myResp = (HttpWebResponse )myReq.GetResponse();

After the program is running for a while I keep getting timeout exception. My thoughts are that I get exception because maybe the Host server has some limitation regarding the concurent connections from the same user.

So how is a connection reused in .net ? In my program each thread is creating a new connection to a hostname, or is reusing the existing one because of the KeepAlive property ??

like image 408
Dorin Avatar asked Oct 06 '11 08:10

Dorin


People also ask

How does connection keep alive work?

Keep-Alive, also known as a persistent connection, is a communication pattern between a server and a client to reduce the HTTP request amount and speed up a web page. When Keep-Alive is turned on, the client and the server agree to keep the connection for subsequent requests or responses open.

What is connection keep alive header?

The Keep-Alive general header allows the sender to hint about how the connection may be used to set a timeout and a maximum amount of requests. Note: Set the Connection header to "keep-alive" for this header to have any effect.

How do I close keep alive connection?

Use “KeepAlive On” to enable it. To disable, just use “KeepAlive Off”. It sets the maximum number of requests for every Keep-Alive connection. A value of 100 is normally good enough for almost any scenario.

What is http keep alive timeout?

The keep alive timeout on the Message Processor allows a single TCP connection to send and receive multiple HTTP requests/responses from/to the backend server, instead of opening a new connection for every request/response pair.


1 Answers

It seems that the problem was some servers that were using http/1.0. HttpWebRequest is using 100 Continue behaviour but the server doesn't support it.

So i changed the property System.Net.ServicePointManager.Expect100Continue to false and then everything worked fine.

like image 131
Dorin Avatar answered Sep 22 '22 01:09

Dorin