Force unloading of DLL from assembly

Question

I am attempting to unload a misbehaving third-party DLL from my .NET process, as it seems to be causing a problem which is always resolved by restarting my application. Rather than restarting the application, I'd like to remove and reload the DLL instead.

The DLL is being loaded using LoadLibrary and removed using FreeLibrary (using DllImports taken from the P/Invoke website). When I call LoadLibrary() I see the DLL appear in the list of DLLs in Process Explorer, and when I call FreeLibrary() I see the DLL disappear from the list of DLLs - as expected.

However, once I have called the Initialize() function of the third-party library, FreeLibrary() no longer removes the DLL from the list, even if I call a corresponding Deinit() method beforehand. Calling another function in the library does not have this problem. However, I must Initialise() the library before use!

I have tried isolating the DLL by creating it in its own AppDomain, then unloading this domain after the DLL is freed.

I get no error return codes or exceptions from Initialize() or Deinit(), from the LoadLibrary() or FreeLibrary() or from creating or unloading the AppDomain.

I used the following code to create the AppDomain and initialise:

string pathToDll = Assembly.GetExecutingAssembly().CodeBase;
m_Domain = AppDomain.CreateDomain("MyAppDomain", null, new AppDomainSetup { PrivateBinPath = pathToDll });
m_Module = (ThirdPartyModule)m_Domain.CreateInstanceFromAndUnwrap(pathToDll, typeof(ThirdPartyModule).FullName);
m_Module.Init();

To deinitialise and unload the AppDomain:

m_Module.Free();
m_Module = null;
if (m_Domain != null)
{
    AppDomain.Unload(m_Domain);
    m_Domain = null;
}

Finally, my ThirdPartyModule assembly class:

internal class ThirdPartyModule : MarshalByRefObject
{
    [DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
    internal static extern IntPtr LoadLibrary(string lpFileName);

    [DllImport("kernel32.dll", SetLastError = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    internal static extern bool FreeLibrary(IntPtr hModule);

    public IntPtr Module { get; set; }

    public ThirdPartyModule()
    {
        Module = LoadLibrary("Misbehaving.dll");
    }

    public void Free()
    {
        FreeLibrary(Module);
        Module = IntPtr.Zero;
    }

    // ...
}

Does this look like it should behave as I expected? If not, is there any other way I can ensure that this DLL is totally unloaded by my process?

Edit: More info

• The DLL contains native code, likely compiled from C/C++
• Unfortunately my process is constrained to using .NET 2 only (so no WCF solution).
• I am using WinXP Pro x64 SP2 but the solution must be XP, Win7 x32/x64 etc compatible.
• The DLL is used to communicate with a USB token

Answer 1

I would recommend to implement a separate process (EXE) which your application launches and which in turn loads the DLL.

This allows you to kill the process whenever need be...

I see several options on how to communicate - for example:

• you could use COM (if you implement it as an out-of-process COM server)
• you could use shared memory (very high performance, see this for a walkthrough and this for a .NET 2 wrapper)

Since you write that the method must be compatible with several Windows versions and some come with a desktop firewall I would refrain from using anything "networky" for the IPC.