Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the best memory buffer size to allocate to download a file from Internet?

Tags:

c#

.net

download

What is the best memory buffer size to allocate to download a file from Internet? Some of the samples said that it should be 1K. Well, I need to know in general why is it? And also what's the difference if we download a small .PNG or a large .AVI?

Stream remoteStream;
Stream localStream;
WebResponse response;

try
{
    response = request.EndGetResponse(result);

    if (response == null)
        return;

    remoteStream = response.GetResponseStream();

    var localFile = Path.Combine(FileManager.GetFolderContent(), TaskResult.ContentItem.FileName);
    localStream = File.Create(localFile);

    var buffer = new byte[1024];
    int bytesRead;

    do
    {
        bytesRead = remoteStream.Read(buffer, 0, buffer.Length);
        localStream.Write(buffer, 0, bytesRead);
        BytesProcessed += bytesRead;
    } while (bytesRead > 0);
}
like image 736
Friend Avatar asked Oct 21 '11 13:10

Friend


People also ask

What is the ideal buffer size?

A good buffer size for recording is 128 samples, but you can also get away with raising the buffer size up to 256 samples without being able to detect much latency in the signal. You can also decrease the buffer size below 128, but then some plugins and effects may not run in real time.

What is the typical size of the file buffer?

The buffer size is, by default, 4096 bytes for sequential access files and 512 bytes for all other files. If the G run-time switch is turned on, then all files use 512-byte buffers.

Why is buffer size 1024?

1024 is the exact amount of bytes in a kilobyte. All that line means is that they are creating a buffer of 16 KB. That's really all there is to it. If you want to go down the route of why there are 1024 bytes in a kilobyte and why it's a good idea to use that in programming, this would be a good place to start.

Is bigger buffer better?

It is recommended that you increase buffer sizes to avoid flow control under normal operating conditions. A larger buffer size reduces the potential for flow control to occur, and results in improved CPU utilization. However, a large buffer size can have a negative effect on performance in some cases.


1 Answers

For what it's worth, I tested reading a 1484 KB text file using progressive powers of two (sizes of 2,4,8,16...). I printed out to the console window the number of milliseconds required to read each one. Much past 8192 it didn't seem like much of a difference. Here are the results on my Windows 7 64 bit machine.

2^1 = 2 :264.0151
2^2 = 4 :193.011
2^3 = 8 :175.01
2^4 = 16 :153.0088
2^5 = 32 :139.0079
2^6 = 64 :134.0077
2^7 = 128 :132.0075
2^8 = 256 :130.0075
2^9 = 512 :133.0076
2^10 = 1024 :133.0076
2^11 = 2048 :90.0051
2^12 = 4096 :69.0039
2^13 = 8192 :60.0035
2^14 = 16384 :56.0032
2^15 = 32768 :53.003
2^16 = 65536 :53.003
2^17 = 131072 :52.003
2^18 = 262144 :53.003
2^19 = 524288 :54.0031
2^20 = 1048576 :55.0031
2^21 = 2097152 :54.0031
2^22 = 4194304 :54.0031
2^23 = 8388608 :54.003
2^24 = 16777216 :55.0032
like image 91
DevonTaig Avatar answered Sep 22 '22 21:09

DevonTaig