Why does instantiating XmlSerializer before ServiceHost.Open is called produce a memory & handle leak

Question

When looking for a memory- and handleleak in a .NET/WCF/Windows Service I noticed strange behavior that I cannot explain. Here the setup and the resolution. What I am looking for would be an explanation for the observed behavior.

I installed a Windows Service.
I started the service.
I called a simple method with a transactional WCF call (new channel per call - no caching).
For each call about 2 handles remain in memory.

This can be observed if the following items are applicable:

1. It is a Windows Service; don't run it as a Console App.
2. Use a Transaction (separate process or machine tested only) to call the WCF method.
3. Before calling ServiceBase.Run(servicesToRun); instantiate XmlSerializer with some type.
4. The type is a custom type. It does not occur with new XmlSerializer(typeof(string)) or new XmlSerializer(typeof(XmlDocument)). No call to serialize is necessary. It is enough if the custom type has only a string as property (no handles anywhere!)
5. Creating a static XmlSerialization.dll using i.e. SGen.exe will not produce this problem.

My Code already includes the fix:
Use XmlSerializer earliest in OnStart():

Program.cs

WindowsService winSvc = new WindowsService();
ServiceBase[] servicesToRun = new ServiceBase[]{winSvc};                    
ServiceBase.Run(servicesToRun);

WindowsService.cs

internal sealed class WindowsService : ServiceBase
{
    private ServiceHost wcfServiceHost = null;

    internal WindowsService()
    {
        AutoLog = true;
        CanStop = true;
        CanShutdown = true;
        CanPauseAndContinue = false;
    }

    internal void StartWcfService()
    {
        wcfServiceHost = new ServiceHost(typeof(DemoService));
        wcfServiceHost.Open();
    }

    protected override void Dispose(bool disposing)
    {
        if (wcfServiceHost != null)
        {
            wcfServiceHost.Close();
        }

        base.Dispose(disposing);
    }

    protected override void OnStart(string[] args)
    {
        new XmlSerializer(typeof(MyType));

        StartWcfService();
    }
}

DemoService.cs

[ServiceBehavior
    (
        InstanceContextMode = InstanceContextMode.PerSession,
        TransactionAutoCompleteOnSessionClose = false,
        IncludeExceptionDetailInFaults = true
    )
]
public sealed class DemoService : IDemoService
{           
    [TransactionFlow(TransactionFlowOption.Allowed)]
    [OperationBehavior(TransactionScopeRequired = true, TransactionAutoComplete = true)]
    public int Add(int a, int b)
    {
        return a + b;
    }
}

Client.cs:

IChannelFactory<IDemoService> channelFactory = new ChannelFactory<IDemoService>("defaultClientConfiguration");
IDisposable channel = null;
for (int index = 0; index < 5000; index++)
{
    using
    (
        channel = (IDisposable)channelFactory.CreateChannel(new EndpointAddress("net.tcp://localhost:23456/DemoService")))
        {                       
        IDemoService demoService = (IDemoService)channel;
        using (TransactionScope tx = new TransactionScope(TransactionScopeOption.RequiresNew))
        {
            demoService.Add(3, 9);
            tx.Complete();  
        }
    )
}

Can someone explain this behavior?

Please note, I'm not interested in finding a way to avoid the leak (I already know how to do this) but in an explanation (i.e. WHY is it happening).

Answer 1

I think some of the inner workings do this question justice. I do this from the back of my head, since I ran into this problem as well some time ago, and it took me a day to track back including extensive use of Reflector and ANTS Memory profiler (at my previous company)... here goes:

What the XML Serializer internally does is it creates a class (let's call it 'A') using System.Reflection.Emit that accepts the type you pass to it. Constructing such a class costs a lot of time relatively speaking, and are reusable since types don't change. Because of this, the constructed types are stored in a dictionary, e.g. it ends up with some Dictionary .

For known (basic) types, the serializer code is fixed, e.g. the serialization of a string is not going to change no matter how many times you restart your application. Note the difference with 'A' where any type that's unknown to the serialization factory till it's first passed to the XMLSerializer.

The first time the type is used by the XMLSerializer, this process takes place for both the type you pass and all types it needs (e.g. all fields and properties that require serialization).

About the leak... When you call the ChannelFactory, it constructs serializer if it doesn't exist yet. For that it checks if the serializer already exists in the Dictionary and if not, creates one by making an instance of ISomeSerializerType.

For some stupid reason, there is a bug in the factory that constructs a new serializer without storing it in the dictionary. Once constructed, you end up with a new type - which shows up as a leak (remember: types can never be unloaded) - even though the objects are correctly disposed. When you use XMLSerializer first or create a static class, it correctly uses the Dictionary cache, which means it won't leak. So there you have it, it is a bug. I used to have access to ANTS Memory Profiler, which showed this quite nicely.

Hope this explains.

Answer 2

The XmlSerializer documentation says this:

To increase performance, the XML serialization infrastructure dynamically generates assemblies to serialize and deserialize specified types. The infrastructure finds and reuses those assemblies. This behavior occurs only when using the following constructors:

XmlSerializer.XmlSerializer(Type)

XmlSerializer.XmlSerializer(Type, String)

If you use any of the other constructors, multiple versions of the same assembly are generated and never unloaded, which results in a memory leak and poor performance. The easiest solution is to use one of the previously mentioned two constructors. Otherwise, you must cache the assemblies in a Hashtable, as shown in the following example.

http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx