Referencing an unstable DLL

Question

We are referencing a 3rd party proprietary CLI DLL in our .net project. This DLL is only an interface to their proprietary C++ library. Our project is an asp.net (MVC4/Web API) web application.

The C++ unmanaged library is rather unstable. Sometimes it crashes with e.g. dangling pointers. We have no way of solving it, and using this library is a first-class customer requirement.

When the application crashes, the application pool in IIS doesn't respond anymore. We have to restart it, and doing so takes a couple minutes (yes, that long!).

We would like to keep this unstable DLL from crashing our application. What's the best way of doing it? Can we keep the CLI DLL in a separate AppDomain? How?

Thanks in advance.

Answer 1

I think every answer to this question will be some kind of work around.

My workaround would be to not interact directly with the DLL from your web application.

Instead write your requests from the web application to either a Message Queue or a SQL table. You can then have another application such as a Windows Service which reads the requests, interacts with the DLL and then writes the results back for your web application to read.

I'm not saying that SQL / Message Queues are the right way, I'm more thinking of the general process flow.

Answer 2

I had this exact problem with a third party library that accessed protected memory for purposes of interacting with a hardware copy protection dongle. It worked fine in a console or winforms app, but crashed like crazy when called from an IIS application.

We tried several different things, some of which are mentioned in other answers on this page. But ultimately, the best solution for us was to us a very old technology - .Net Remoting. I know - it's somewhat frowned on these days. But it fit this particular need quite well.

The unstable code was placed in a Windows Service application. The web application made remoting calls to this service, which relayed the commands to the third-party library.

Now I'm sure you could do the same thing with WCF, sockets, etc. But remoting was quick and easy to setup, and since we only talk to the same server it works without opening any ports. It just talks on a named pipe.

It does mean a second service to install besides the web application, but that was acceptable in my particular use case.

If you did something similar, and the third-party code actually crashed the service, you could probably write some code in your main application to bring it back up.

So perhaps a process boundary is more useful than an App Domain when you have unstable code to wrangle.

Answer 3

I would first increase the IIS process recyling rate, maybe the the DLL code fails after a certain number of calls, or after the process reaches a certain amount of memory usage.

You can find information on the configuration of IIS 7.0 recycling options here: http://technet.microsoft.com/en-us/library/cc753179(v=ws.10).aspx

In your case I would recycle the process at a specific time, when you know there is less load on the application. And after a certain number of requests (lower than the default) to try and have "fresh" process most of the time.

The recycling process is graceful in the sense that the the old process is not terminated until the one that will replace it is ready, so there should be no noticeable downtime. More information about the recycling mechanism here: http://technet.microsoft.com/en-us/library/cc745955.aspx

If the above does not solve the problem I would wrap the calls in my own code that manages the unstable DLL execution.

This code should recover from the failures for example by repeating the failing calls until a result is obtained and failing with a graceful error if it is not possible after a number of attempts.

Internally the calls to the unstable DLL could be made in a spawned thread or even the code could be in an new external executable that you could launch with Process.Start.

This last option has more overhead but it might be your only option. See this SO question for more information on this: How do you handle a thread that has a hung call?