How much memory is affected from StreamReader

Question

When I use StreamReader as below, how much memory is affected. I presume since each line is read into a variable 'line', only that line remains in memory and that is overall a good thing. My purpose is not to burden memory too much when I read a very very large containing thousands of lines. A clarification will be appreciated.

string line = string.Empty;
using(StreamReader Reader = new StreamReader(@"C:\Users\UK\Panfile.txt"))
{
    while((line = Reader.ReadLine())!= null)
    {
        //other code to process the line now being read.
    }
Reader.Close(); Reader.Dispose();
}

Answer 1

A StreamReader will use byteBuffer.Length of memory between calls. If you have not specified a default it uses 1024 bytes. It also allocates a char[] charBuffer of size encoding.GetMaxCharCount(bufferSize); which allocates two bytes per element in the array.

If you do not pass in a Stream object and let it generate it's own FileStream it will use a default filestream buffer of 4096.

The ReadLine call itself will allocate a StringBuilder internally and will read data into the byteBuffer then decode the bytes and store it in the charBuffer, it will then copy the chars out of the charBuffer and into the StringBuilder which then is returned to you via a .ToString() call.

So in summary, new StreamReader(@"C:\Users\UK\Panfile.txt") at rest it will will allocate 1024 + (1025 * 2) + 4096¹ bytes of memory (5120 total), and during the ReadLine call it will allocate at most an additional line.Length * 2 + StringBuilderOverhead + line.Length * 2². The *2's you see are for the char[] because each char takes up two bytes.

---

^{1:byteBuffer + charBuffer + the FileStream buffer
2:The char[] internal to the StringBuilder + Any slack space in the string builder buffer + the string returned by the .ToString() call.}

Answer 2

It will only hold 1 line in memory at a time so this method is perfect for processing large files without eating up lots of memory.