What defines the capacity of a memory stream

Question

I was calculating the size of object(a List that is being Populated), using the following code:

 long myObjectSize = 0;
 System.IO.MemoryStream memoryStreamObject = new System.IO.MemoryStream();
 System.Runtime.Serialization.Formatters.Binary.BinaryFormatter binaryBuffer = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
 binaryBuffer.Serialize(memoryStreamObject, myListObject);
 myObjectSize = memoryStreamObject.Position;

A the initial point the capacity of the memoryStreamObject was 1024

Later(after adding more elements to the list) It was shown as 2048.

And It seems to be increasing as the stream content increasing. Then what is the purpose of capacity in this scenario.?

Answer 1

This is caused by the internal implementation of the MemoryStream. The Capacity property is the size of the internal buffer. This make sense if the MemoryStream is created with a fixed size buffer. But in your case the MemoryStream can grow and the actual implementation doubles the size of the buffer if the buffer is too small.

Code of MemoryStream

private bool EnsureCapacity(int value)
{
if (value < 0)
{
    throw new IOException(Environment.GetResourceString("IO.IO_StreamTooLong"));
}
if (value > this._capacity)
{
    int num = value;
    if (num < 256)
    {
        num = 256;
    }
    if (num < this._capacity * 2)
    {
        num = this._capacity * 2;
    }
    if (this._capacity * 2 > 2147483591)
    {
        num = ((value > 2147483591) ? value : 2147483591);
    }
    this.Capacity = num;
    return true;
    }
  return false;
}

And somewhere in Write

int num = this._position + count;
// snip
if (num > this._capacity && this.EnsureCapacity(num))

Answer 2

The purpose of the capacity for a memory stream and for lists is that the underlying data structure is really an array, and arrays cannot be dynamically resized.

So to start with you use an array with a small(ish) size, but once you add enough data so that the array is no longer big enough you need to create a new array, copy over all the data from the old array to the new array, and then switch to using the new array from now on.

This create+copy takes time, the bigger the array, the longer the time it takes to do this. As such, if you resized the array just big enough every time, you would effectively do this every time you write to the memory stream or add a new element to the list.

Instead you have a capacity, saying "you can use up to this value before having to resize" to reduce the number of create+copy cycles you have to perform.

For instance, if you were to write one byte at a time to this array, and not have this capacity concept, every extra byte would mean one full cycle of create+copy of the entire array. Instead, with the last screenshot in your question, you can write one byte at a time, 520 more times before this create+copy cycle has to be performed.

So this is a performance optimization.

An additional bonus is that repeatedly allocating slightly larger memory blocks will eventually fragment memory so that you would risk getting "out of memory" exceptions, reducing the number of such allocations also helps to stave off this scenario.

A typical method to calculate this capacity is by just doubling it every time.