Should Application_End fire on an automatic App Pool Recycle?

Question

I have read this, this, this and this plus a dozen other posts/blogs.

I have an ASP.Net app in shared hosting that is frequently recycling. We use NLog and have the following code in global.asax

void Application_Start(object sender, EventArgs e) 
{
    NLog.Logger logger = NLog.LogManager.GetCurrentClassLogger();
    logger.Debug("\r\n\r\nAPPLICATION STARTING\r\n\r\n");
}
protected void Application_OnEnd(Object sender, EventArgs e)
{
    NLog.Logger logger = NLog.LogManager.GetCurrentClassLogger();
    logger.Debug("\r\n\r\nAPPLICATION_OnEnd\r\n\r\n");
}

void Application_End(object sender, EventArgs e) 
{
        HttpRuntime runtime = (HttpRuntime)typeof(System.Web.HttpRuntime).InvokeMember("_theRuntime", BindingFlags.NonPublic  | BindingFlags.Static  | BindingFlags.GetField,  null,  null,  null);

if (runtime == null)
    return;

string shutDownMessage = (string)runtime.GetType().InvokeMember("_shutDownMessage",  BindingFlags.NonPublic  | BindingFlags.Instance  | BindingFlags.GetField,  null,  runtime,  null);

string shutDownStack = (string)runtime.GetType().InvokeMember("_shutDownStack",   BindingFlags.NonPublic  | BindingFlags.Instance  | BindingFlags.GetField,  null,  runtime,  null);

ApplicationShutdownReason shutdownReason = System.Web.Hosting.HostingEnvironment.ShutdownReason;

NLog.Logger logger = NLog.LogManager.GetCurrentClassLogger();
logger.Debug(String.Format("\r\n\r\nAPPLICATION END\r\n\r\n_shutDownReason = {2}\r\n\r\n _shutDownMessage = {0}\r\n\r\n_shutDownStack = {1}\r\n\r\n",
                shutDownMessage, shutDownStack, shutdownReason));
}

void Application_Error(object sender, EventArgs e) 
{
    NLog.Logger logger = NLog.LogManager.GetCurrentClassLogger();
    logger.Debug("\r\n\r\nApplication_Error\r\n\r\n");

}

Our log file is littered with "APPLICATION STARTING" entries, but neither Application_OnEnd, Application_End, nor Application_Error are ever fired during these spontaneous restarts. I know they are working because there are entries for touching the web.config or /bin files. We also ran a memory overload test and can trigger an OutOfMemoryException which is caught in Application_Error.

We are trying to determine whether the virtual memory limit is causing the recycling. We have added GC.GetTotalMemory(false) throughout the code, but this is for all of .Net, not just our App´s pool, correct? We've also tried

var oPerfCounter = new PerformanceCounter();
oPerfCounter.CategoryName = "Process";
oPerfCounter.CounterName = "Virtual Bytes";
oPerfCounter.InstanceName = "iisExpress";
logger.Debug("Virtual Bytes: " + oPerfCounter.RawValue + " bytes");

but don't have permission in shared hosting.

I've monitored the app on a dev server with the same requests that caused the recycles in production with ANTS Memory Profiler attached and can't seem to find a culprit. We have also run it with a debugger attached in dev to check for uncaught exceptions in spawned threads that might cause the app to abort.

My questions are these:

• How can I effectively monitor memory usage in shared hosting to tell how much my application is consuming prior to an application recycle?
• Why are the Application_[End/OnEnd/Error] handlers in global.asax not being called?
• How else can I determine what is causing these recycles?

Thanks.

EDIT: Based on answer by @Jani Hyytiäinen

Scenario: Thread #1 begins and is followed by thread #2. Thread #1 hits memory limit but continues processing. Thread #3 begins. Thread #1 finishes, but #2 processes more than 60 seconds after #1 has hit the memory limit.

The pool then ungracefully aborts? What http responses will #2 & #3 receive (These are AJAX calls, but I am getting 504 errors in Fiddler)?
Is the request for #3 accepted or does it just queue until a new pool has started up?
Is there any way to know that the memory limit has been hit or is about to be?

Any strategies welcome.

Answer 1

There is an application pool shutdown time limit in iis. Let's say it's 60 seconds and in shared hosting environment there is an application pool memory limit. Your application pool reaches this limit and iis tells the application pool to finish all the work with current requests. If all the requests finish processing before 60 seconds is passed, application_end will trigger and application pool will gracefully shut itself down. However if 60 seconds pass and requests are still being processed, IIS gets upset and kills the app pool. This time no application_end would trigger. Similarly, no error event handler would be triggered.