Loading assemblies as modules/plugins while avoiding duplication and fragility

Question

We have a fairly large C# code base for a product that has been separated into many assemblies to avoid a monolithic product and to enforce some code quality standards (customer-specific features go in customer-specific assemblies to keep the "core" generic and unencumbered by dependencies on customer-specific business logic). We call these plugins internally, but they're more the modules that make up the total product.

The way this works is that the DLLs of these modules get copied to a directory, the application runtime (either a ServiceStack IIS web application or a Quartz-based console application) then does an Assembly.LoadFile for every module that is not in the list of current assemblies already loaded (AppDomain.CurrentDomain.GetAssemblies()).

This PluginLoader only loads assemblies that are present in plugins.config file, but I think that's mostly irrelevant for the problem at hand.

The full code for the PluginLoader class:

https://gist.github.com/JulianRooze/9f6d1b5e61c855579203

This.... works. Sort of. It's fragile though and suffers from a problem that assemblies get loaded twice this way, from different locations (usually from the /bin/ folder of the application and the plugin directory). This seems to happen because at the moment the PluginLoader class is invoked, the AppDomain.CurrentDomain.GetAssemblies() (at startup) does not necessarily return the final list of the assemblies that the program will load by itself. So if there's an assembly in the /bin/ called dapper.dll (a common dependency of both the core and many plugins/modules) that has not been used by the program yet, then it will not have been loaded yet (in other words: it loads them lazily). Then, if that dapper.dll is also by a plugin, the PluginLoader will see that it has not been loaded yet and will load it. Then, when the program uses its Dapper dependency, it will load the dapper.dll from /bin/ and we now have two dapper.dll's loaded.

In most cases, that seems to be fine. However, we make use of the RazorEngine library which complains about duplicate assemblies with the same name when you try to compile your templates.

In investigating this, I came across this question:

Is there a way to force all referenced assemblies to be loaded into the app domain?

I tried both the accepted answer and Jon Skeet's solution. The accepted answer works (though I haven't verified if there's any odd behavior yet) but it feels nasty. For one, this also makes the program try to load native DLLs that happen to be in /bin/ which obviously fails because they're not .NET assemblies. So you now have to try-catch-swallow this. I'm also worried about weird side effects if the /bin/ contains some old DLL that isn't actually used any more, but now gets loaded anyway. That isn't a problem in production, but it is in development (in fact, this whole thing is more of a problem in dev than production, but the added robustness of solving this would be appreciated in production as well).

As said, I also tried Jon Skeet's answer, and my implementation is visible in that Gist of the PluginLoader class in the method LoadReferencedAssemblies. This has two problems:

1. It fails on some assembly names, like System.Runtime.Serialization with a file not found.
2. It causes a failure later where a plugin is suddenly unable to find a dependency. I can't find why yet.

I also briefly investigated using Managed Extensibility Framework, but I'm not sure if it applies. That seems to be more aimed at providing a framework for loading components and defining how they can interact, whereas I'm literally only interested in loading assemblies dynamically.

So, given the requirement "I want to dynamically load a specified list of DLLs from a directory without any chance of loading duplicate assemblies", what is the best solution? :)

I'm willing to overhaul how the plugin system works, if that's what it takes.

Answer 1

There are a few ways to tackle this problem (modules/plugins deployed in a hierarchy of directories) and frankly, you have chosen the most difficult.

The simplest one is to add all your folders in the private probing path of your app/web.config. Then replace all calls to Assembly.LoadFile with Assembly.Load. This will let the .NET assembly-resolving mechanism to automatically resolve all assemblies for you. You will not need to load referenced assemblies since they will be loaded when needed automatically. Only the modules/plugins will have to be loaded using Assembly.Load.

The drawbacks of this approach is when either one of the following is true:

• The deployed assemblies do not reside in directories under the application base. This can be overcome (at a cost, see the remarks in the Assembly.Load(AssemblyName) documentation) if you just set the AssemblyName.Codebase property. This will make Load work like LoadFrom. MEF uses this approach in AssemblyCatalog.
• You have different assemblies (not just files) with the same identity. If this is the case then you probably need to use a different assembly naming approach. Take for example the DevExpress approach that has strong-named assemblies with the assembly version in the file name. This will allow you to have a flat directory structure. If you cannot go with this approach, then strong-name your assemblies and deploy them in the GAC or in different folders. If you cannot do this will all your assemblies then load as few assemblies as possible with your current approach but try male a plan to slowly replace them with new strong-named versions.

Note that I am not familiar with ServiceStack. In ASP.NET you can use the Assembly.Codebase property to load assemblies deployed outside of bin.

Finally have a look at Suzanne Cook's blog entry on LoadFile vs. LoadFrom.