Upload Speed Issue : HttpWebRequest [duplicate]

Question

I am using HttpWebRequest to upload file to certain server, now the problem is I am having speed issues.

I am not able to get the same upload speed as I get with browser (Mozilla Firefox), the speed I am getting is 1/5 of the speed I get from Browser.

Here are my HttpWebRequest object's setting

//headers is a NameValueCollection type object,
//Method is a struct { GET, POST, HEAD }

        HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
        request.UserAgent = headers["User-Agent"];
        request.KeepAlive = false;
        request.Accept = headers["Accept"];
        request.AllowAutoRedirect = AllowRedirect;
        request.Headers.Add(HttpRequestHeader.AcceptLanguage, "en-US,en;q=0.5");
        request.Method = Method.ToString();
        request.AllowWriteStreamBuffering = false;
        request.ReadWriteTimeout = 60000;

Some global options that i have kept enabled

        ServicePointManager.Expect100Continue = false;
        ServicePointManager.DefaultConnectionLimit = 200;
        ServicePointManager.MaxServicePointIdleTime = 2000;
        ServicePointManager.MaxServicePoints = 1000;
        ServicePointManager.SetTcpKeepAlive(false, 0, 0);

How i am sending file in chunk...

            if (PostMethod == PostType.MultiPart && uploadFiles.Count > 0)
            {
                for (int i = 0; i < uploadFiles.Count; i++)
                {
                    string fileParam = uploadFiles.GetKey(i);
                    string tmpFilename = uploadFiles.Get(i);
                    string tmpData =
                    string.Format(
                        "--{0}\r\nContent-Disposition: form-data; name=\"{1}\"; filename=\"{2}\"\r\nContent-Type: {3}\r\n\r\n", boundary, fileParam, Path.GetFileName(tmpFilename), MimeType.GetMimeByExtension(Path.GetExtension(tmpFilename)));
                    byte[] tmpBytes = Encoding.Default.GetBytes(tmpData);
                    writer.Write(tmpBytes, 0, tmpBytes.Length);
                    bSent += tmpBytes.Length;

                    arg.Progress = (int)(bSent * 100 / totalBytes);
                    arg.Speed = (bSent / sw.Elapsed.TotalSeconds);
                    OnProgress(arg);

                    //write the file
                    int fileBytesRead;

                    FileStream fileStream = File.Open(tmpFilename, FileMode.Open, FileAccess.Read, FileShare.Read);
// buffer size = 8192
                    while ((fileBytesRead = fileStream.Read(buffer, 0, BUFFER_SIZE)) > 0) 

                    {
                        writer.Write(buffer, 0, fileBytesRead);
                        bSent += fileBytesRead;

                        int timeNow = Environment.TickCount;
                        if (timeNow - lastTime >= 500)
                        {
                            lastTime = timeNow;
                            arg.Progress = (int)(bSent * 100 / totalBytes);
                            arg.Speed = (bSent / sw.Elapsed.TotalSeconds);
                            OnProgress(arg);
                        }
                    }

                    tmpBytes = Encoding.Default.GetBytes("\r\n");
                    writer.Write(tmpBytes, 0, tmpBytes.Length);
                    bSent += tmpBytes.Length;

                    arg.Progress = (int)(bSent * 100 / totalBytes);
                    arg.Speed = (bSent / sw.Elapsed.TotalSeconds);
                    OnProgress(arg);

                }
            }

I would be happy to attain even a 75% of browser upload speed.

Answer 1

I found the answer by searching this site. It appears to be a duplicate of ".NET HttpWebRequest Speed versus Browser".

From the answers posted there:

"The first time you request a page, .net tries to detect proxy settings. The solution is to pass in an empty WebProxy object. This way it just connects to remote server instead of autodetecting the proxy server." - Markos

Try adding

    request.Proxy = new WebProxy();

Answer 2

Two things.

1. You're defeating the purpose of TCP/IP stacks by chunking. TCP/IP is designed to handle near asynchronous transmission of data. It will do its own chunking and continue to send chunks until it is finished or reaches its maximum number of unacknowledged chunks. If the max number of chunks is reached, it will wait for an ACK(acknowledge) to continue - similar to a semaphore ideology expect the releases can be "stacked", AKA a single ACK contains the last point it received and can "release"/acknowledge multiple chunk packets) - or the request times out.

   By chunking it yourself, each chunk has to be fully acknowledged to send anymore data so you're increasing your overhead. You are still using the same connection, but essentially making multiple separate requests. In addition, if the TCP/IP chunk size is not a divisor of you chunk size (YOUR_CHUNK_SIZE % TCP_IP_CHUNK_SIZE != 0) you are leaving a lingering TCP/IP chunk that has to timeout and resolve to a pseudo ACK; it gets acknowledged the next time it can in the next chunk.

2.

Another option that can have an impact on performance is the use of the UseNagleAlgorithm property. When this property is set to true, TCP/IP will try to use the TCP Nagle algorithm for HTTP connections. The Nagle algorithm aggregates data when sending TCP packets. It accumulates sequences of small messages into larger TCP packets before the data is sent over the network. Using the Nagle algorithm can optimize the use of network resources, although in some situations performance can also be degraded. Generally for constant high-volume throughput, a performance improvement is realized using the Nagle algorithm. But for smaller throughput applications, degradation in performance may be seen.

An application doesn't normally need to change the default value for the UseNagleAlgorithm property which is set to true. However, if an application is using low-latency connections, it may help to set this property to false.

From MSDN: HttpWebRequest Class (System.Net)