Does StringBuilder become immutable after a call to ToString?

Question

I distinctly remember from the early days of .NET that calling ToString on a StringBuilder used to provide the new string object (to be returned) with the internal char buffer used by StringBuilder. This way if you constructed a huge string using StringBuilder, calling ToString didn't have to copy it.

In doing that, the StringBuilder had to prevent any additional changes to the buffer, because it was now used by an immutable string. As a result the StringBuilder would switch to a "copy-on-change" made where any attempted change would first create a new buffer, copy the content of the old buffer to it and only then change it.

I think the assumption was that StringBuilder would be used to construct a string, then converted to a regular string and discarded. Seems like a reasonable assumption to me.

Now here is the thing. I can't find any mention of this in the documentation. But I'm not sure it was ever documented.

So I looked at the implementation of ToString using Reflector (.NET 4.0), and it seems to me that it actually copies the string, rather than just share the buffer:

[SecuritySafeCritical]
public override unsafe string ToString()
{
    string str = string.FastAllocateString(this.Length);
    StringBuilder chunkPrevious = this;
    fixed (char* str2 = ((char*) str))
    {
        char* chPtr = str2;
        do
        {
            if (chunkPrevious.m_ChunkLength > 0)
            {
                char[] chunkChars = chunkPrevious.m_ChunkChars;
                int chunkOffset = chunkPrevious.m_ChunkOffset;
                int chunkLength = chunkPrevious.m_ChunkLength;
                if ((((ulong) (chunkLength + chunkOffset)) > str.Length) ||     (chunkLength > chunkChars.Length))
                {
                    throw new ArgumentOutOfRangeException("chunkLength",     Environment.GetResourceString("ArgumentOutOfRange_Index"));
                }
                fixed (char* chRef = chunkChars)
                {
                    string.wstrcpy(chPtr + chunkOffset, chRef, chunkLength);
                }
            }
            chunkPrevious = chunkPrevious.m_ChunkPrevious;
        }
        while (chunkPrevious != null);
    }
    return str;
}

Now, as I mentioned before I distinctly remember reading that this was the case in the early days if .NET. I even found a mention of in this book.

My question is, was this behavior dropped? If so, anyone knows why? It made perfect sense to me...

Answer 1

Yup, this has been completely redesigned for .NET 4.0. It now uses a rope, a linked list of string builders to store the growing internal buffer. This is a workaround for a problem when you can't guess the initial Capacity well and the amount of text is large. That creates a lot of copies of the dis-used internal buffer, clogging up the Large Object Heap. This comment from the source code as available from the Reference Source is relevant:

    // We want to keep chunk arrays out of large object heap (< 85K bytes ~ 40K chars) to be sure.
    // Making the maximum chunk size big means less allocation code called, but also more waste 
    // in unused characters and slower inserts / replaces (since you do need to slide characters over
    // within a buffer).
    internal const int MaxChunkSize = 8000;

Answer 2

Yes, you remember correctly. The StringBuilder.ToString method used to return the internal buffer as the string, and flag it as used so that additional changes to the StringBuilder had to allocate a new buffer.

As this is an implementation detail, it's not mentioned in the documentation. This is why they can change the underlying implementation without breaking anything in the defined behaviour of the class.

As you see from the code posted, there is not a single internal buffer any more, instead the characters are stored in chunks, and the ToString method puts the chunks together into a string.

The reason for this change in implementation is likely that they have gathered information about how the StringBuilder class is actually used, and come to the conclusion that this approach gives a better performance weighed between average and worst case situations.