How to read MEF metadata in a DLL plugin without copying the entire DLL into memory?

Question

Background:

I'm interested in using MEF to provide a plug-in architecture in a WinForm application using C# with .NET 4.0, but I am not clear on a couple of things.

First: I have not worked at all with building DLLs yet in C#, and I'm a little fuzzy on the concept of DLL Assemblies and how a DLL is normally loaded into memory ( meaning, all at once or in pieces as needed )

Intent:

The program will be a machine hardware control framework, and will be made up of a primary WinForm GUI that is a generic environment with basic toolbars, menus, etc - but no bulk GUI content. ( Think: MDI Parent but not actually ).

The plug-ins provide all of the controls for a specific machine. Each of many possible plug-ins will contain potentially 30 to 50 large UserControls, each populated with many dozens of WinForm Controls and tons of support code that makes up the various machine control panels.

This means that the main program is a lightweight general framework and the plug-ins contain the bulk of the GUI controls and functionality that will be shown in the main program user interface, including lots of icons, images, and other resources. This will make the plug-in DLLs potentially quite large.

The goal is to allow the user to select a plug-in from a menu, and then upon selection, load and run the plug-in which will then fill the mostly empty main GUI with panels, menus, and toolboxes galore.

To do this, I need to first extract metadata from each plug-in to populate the initial menu for the program, which will include the plug-in title, description, icon, version numbers, and other bits of info.

Here are the questions:

With MEF, if I try to read the metadata from each of many large DLLs that are stored in a plug-in folder, will the entire DLL be copied into memory before the metadata values are accessed ?

If so, is there any way to open each DLL file and only read the metadata into memory to build the initial menu - then later load the full selected DLL through MEF ?

I am assuming that the typical DirectoryCatalog and AggregateCatalog template for reading plugins through MEF will copy all of the discovered DLLs into memory and store them in the catalog collection.

Do DLLs contain one contiguous code block ( assembly ), or can they contain multiple separate blocks that are indexed and copied to memory individually as needed ( multiple assemblies ) ?

I'm probably not understanding the fundamentals, and maybe confusing terms. I would appreciate any insight into the load behavior of MEF, DLLs, and Assemblies in general. Thanks !

Answer 1

With MEF, if I try to read the metadata from each of many large DLLs that are stored in a plug-in folder, will the entire DLL be copied into memory before the metadata values are accessed ?

As far as I know the entire DLL will be loaded into memory. This does not have anything to do with MEF though. DirectoryCatalog will load the assembly (through AssemblyCatalog) using a call to Assembly.Load. This method is not part of MEF but it is a core method of the .NET Framework. Most of the loaded assemblies are loaded this way. If you monitor the process running your application using Process Explorer you can see that the Virtual Size will be increased by the size of the loaded assembly. So if you load huge assemblies the Virtual Size of your process will be high.

If so, is there any way to open each DLL file and only read the metadata into memory to build the initial menu - then later load the full selected DLL through MEF ?

There are ways to do this.

One way is to create a new application domain, create the CompositionContainer in the new AppDomain and check the discovered parts. Then serialize information about these parts to the main AppDomain. Finally unload the new AppDomain. Then you can check which parts you really need and only load the assemblies that contain them. An example on how to do this can be found in this answer.

Another approach is to use Mono.Cecil. This is great library that helps you inspect assemblies without loading them. You can combine it with MEF's Export Metadata in a method like this:

public static bool IncludesTypeWithSpecificExportMetadata<T>(string assemblyPath, string name, T value)
    {
        AssemblyDefinition assemblyDefinition = AssemblyDefinition.ReadAssembly(assemblyPath);

        bool typeWasFound = false;          

        foreach (TypeDefinition typeDefinition in assemblyDefinition.MainModule.GetTypes())
        {
            foreach (CustomAttribute customAttribute in typeDefinition.CustomAttributes)
            {
                if (customAttribute.AttributeType.FullName == typeof(ExportMetadataAttribute).FullName)
                {
                    string actualName = (string)customAttribute.ConstructorArguments[0].Value;
                    T actualValue = (T)((CustomAttributeArgument)customAttribute.ConstructorArguments[1].Value).Value;
                    if (actualName.Equals(name) && actualValue.Equals(value))                        
                    {
                        typeWasFound = true;                       
                    }
                }
            }
        }

        return typeWasFound;
    }

Given an assembly file path and a name/value pair this method will inspect the assembly using Mono.Cecil and look for types decorated with the ExportMetadataAttribute and with the same name/value pair.

I am assuming that the typical DirectoryCatalog and AggregateCatalog template for reading plugins through MEF will copy all of the discovered DLLs into memory and store them in the catalog collection.

True.

Do DLLs contain one contiguous code block ( assembly ), or can they contain multiple separate blocks that are indexed and copied to memory individually as needed ( multiple assemblies ) ?

I don't know about this. You might find the answer in "Essential .NET Volume 1" by Don Box or "C# via CLR" by Jeffrey Richter.

I'm probably not understanding the fundamentals, and maybe confusing terms. I would appreciate any insight into the load behavior of MEF, DLLs, and Assemblies in general. Thanks !

The books I mentioned above include in detail how assemblies are resolved/loaded blah blah. Also have a look at Suzanne Cook's blog.

Now I would like to ask you something. Do you really need to embed large files to your assemblies? If you can find another way then you will not need any of this. Your plug-in engine will be a little bit simple.

Finally I would suggest to have a look at Microsoft's Smart Client Software Factory. It can do pretty much everything you mention and more. It will take some effort to understand it and feel comfortable with it but in the long run it will probably save you loads of time.