Loading more dll's into code causes crash

Question

One of the requirements of the system that I am helping to develop is the ability to import files and we have a set of adapters to process the different types of file (csv, xml etc.) we would expect to encounter. During the early part of development we were hard-coding the data adapters via reference and using commands. Obviously when this goes live we would want the situation where we could just write a new adapter and throw the dll into a folder and run the procedure without recompiling the code.

To implement this I adapted the code from this question. The code in question is located in the constructor as follows

string dllLocation = @"C:MyLocation\dllLocation";
DirectoryInfo dir = new DirectoryInfo(dllLocation);
var tempfiles = dir.GetFiles("*Adapter*.dll", SearchOption.AllDirectories);     // This will need to be changed when we go live

    foreach (var file in tempfiles)
    {
        Assembly tempAssembly = null;

        //Before loading the assembly, check all current loaded assemblies in case already loaded
        //has already been loaded as a reference to another assembly
        //Loading the assembly twice can cause major issues
        foreach (Assembly loadedAssembly in AppDomain.CurrentDomain.GetAssemblies())
        {
            //Check the assembly is not dynamically generated as we are not interested in these
            if (loadedAssembly.ManifestModule.GetType().Namespace != "System.Reflection.Emit")
            {
                //Get the loaded assembly filename
                string loadedFilename = loadedAssembly.CodeBase.Substring(loadedAssembly.CodeBase.LastIndexOf('/') + 1);

                //If the filenames match, set the assembly to the one that is already loaded
                if (loadedFilename.ToUpper() == file.Name.ToUpper())
                {
                    tempAssembly = loadedAssembly;
                    break;
                }
            }
        }

        //If the assembly is not aleady loaded, load it manually
        if (tempAssembly == null)
        {
            tempAssembly = Assembly.LoadFrom(file.FullName);
        }

        Assembly a = tempAssembly;

Later when the method runs we have this

private IEnumerable<IUniversalDataAdapter> DataAdapters
{
    get
    {
        foreach (var asm in AppDomain.CurrentDomain.GetAssemblies())
        {
            foreach (var type in asm.GetTypes().Where(x => x.GetInterfaces().Contains(typeof(IUniversalDataAdapter))))
            {
                if (type.IsAbstract) continue; // can't create abstract classes

                if (!dataAdapters.Any(y => y.GetType().Equals(type)))
                {
                    IUniversalDataAdapter adapter = (IUniversalDataAdapter)Activator.CreateInstance(type);
                    dataAdapters.Add(adapter);
                }
            }
        }
        return dataAdapters;
    }
}

which successfully loads the data adapters and allows them to be used as I would expect.

Now for the question, in the first bit of code we have the line

var tempfiles = dir.GetFiles("*Adapter*.dll", SearchOption.AllDirectories);

If I change it to

var tempfiles = dir.GetFiles("*.dll", SearchOption.AllDirectories);

the routine crashes before the second bit of code runs at this routine

private IEnumerable<IDataValidator> DataValidators
{
    get
    {
        if (validators.Count == 0)
        {
            foreach (var asm in AppDomain.CurrentDomain.GetAssemblies())
            {
                foreach (var type in asm.GetTypes().Where(x => x.GetInterfaces().Contains(typeof(IDataValidator))))
                {
                    if (!type.IsAbstract)
                    {
                        var validator = (IDataValidator)Activator.CreateInstance(type, context);
                        validators.Add(validator);
                    }
                }
            }
        }
        return validators;
    }
}

Edit: added the exception

System.Reflection.ReflectionTypeLoadException was unhandled
  HResult=-2146232830
  Message=Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
  Source=mscorlib
  StackTrace:
     at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
     at System.Reflection.RuntimeModule.GetTypes()
     at System.Reflection.Assembly.GetTypes()
     at TTi.Data.Pipeline.Server.Common.DataPipeline.get_DataValidators() in C:\Users\anorcross\Source\Workspaces\Universal System\Data\Main\TTi.Data\TTi.Data.Pipeline.Server.Common\DataPipeline.cs:line 124
     at TTi.Data.Pipeline.Server.Common.DataPipeline.CheckConfiguration(DataConfiguration config) in C:\Users\anorcross\Source\Workspaces\Universal System\Data\Main\TTi.Data\TTi.Data.Pipeline.Server.Common\DataPipeline.cs:line 528
     at TTi.Data.Pipeline.Server.Common.DataPipeline.ProcessDataSource(IDataSource dataSource, DataConfiguration config) in C:\Users\anorcross\Source\Workspaces\Universal System\Data\Main\TTi.Data\TTi.Data.Pipeline.Server.Common\DataPipeline.cs:line 213
     at TTi.Data.Test.Program.ImportTest(String testFolders) in C:\Users\anorcross\Source\Workspaces\Universal System\Data\Main\TTi.Data\TTi.Data.Test\Program.cs:line 362
     at TTi.Data.Test.Program.Main(String[] args) in C:\Users\anorcross\Source\Workspaces\Universal System\Data\Main\TTi.Data\TTi.Data.Test\Program.cs:line 48
     at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
     at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
     at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
     at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
     at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
     at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
     at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
     at System.Threading.ThreadHelper.ThreadStart()
InnerException: 

EndEdit:

The DataValidators definately runs before the DataAdapters routine. DataValidators are just bits of code contained in the main code base and check to ensure the imported data is of the expected format. At this point we are just loading them so we can make sure the required ones exist.

Looking at the loaded assemblies, both versions of the code load the adapters as required, but the second version loads more than the first as I would expect.

So, why does the second version of tempfiles crash at what looks like a completely unrelated part of the code? And if we add enough data adapters will it cause the code to crash?

Answer 1

This exception states that required dll was not found so it cannot be loaded. Now which dll he was looking for and why it did not fail on compile? To answer the first question you would need to investigate deeper and eventually you will find that your code is loading some strange dll that you do not use or need. This can happen because of the code that iterates on ALL types from ALL loaded assemblies. It is possible to have as a reference a dll that exposes type that use some types from other dll that you do not reference. And it is fine as long as you do not use that type.

In order to fix this make your loop more specific. It's obvious that your adapter will not be declared in any system dll so why would you iterate those?

foreach (var asm in AppDomain.CurrentDomain.GetAssemblies().Where(IsMyAssembly))
            {
                foreach (var type in asm.GetTypes().Where(x => x.GetInterfaces().Contains(typeof(IDataValidator))))
                {
//...

Your first file scan with Adapter is better than the one without it because it is more specific. I would advise to make your code even more specific by for example explicit registration of types in some config file this would eliminate dll scan entirely.