IIS7 stops working after 5 requests

Question

Here is my problem:

I have just been brought onto a massive asp.net C# project and I've been charged with fixing some performance issues (not my area of expertise). More specifically after 5 - 7 redirects/ajax calls the web server stops responding and the whole page (and eventually the browser) freezes.

I don't think this is a coding issue as I've set up break points in a few pages (Page_Load method) and after the 5 requests it does not even reach the break points.

I don't believe this is related to this issue as I've increased the browser's maximum connections per server parameter and I got the same behavior. Furthermore after these 5 request in one browser IE, the application stops working in FF as well.

This is not a resource issue as the w3wp.exe process never exceeds 500MB memory.

One thing I've noticed when using Fiddler and other tools to monitor the requests is that the server takes a very long time when loading image files (png, jpg). I don't know if this is relevant.

I've enabled failed request tracing on the server and the only thing I've noticed is that some request fail with a 401 error even dough I've set Anonymous Authentication to enabled. Here is the exact message

MODULE_SET_RESPONSE_ERROR_STATUS 

 ModuleName     ManagedPipelineHandler 

 Notification   128 
 HttpStatus     401 
 HttpReason     Unauthorized 
 HttpSubStatus  0 
 ErrorCode      0 

 ConfigExceptionInfo

 Notification   EXECUTE_REQUEST_HANDLER

 ErrorCode       The operation completed successfully. (0x0)

This message is sometimes thrown with ModuleName: ScriptModule

I have already wasted 2 days on this thing and I'm running out of ideas so any suggestions would be appreciated.

Answer 1

Like any large generic problem, your best bet in diagnosing the issue is to figure out how to break down the issue into smaller parts, how to hypothesize the issues, and how to validate or invalidate your hypotheses. My first inclination would be to hypothesize that the server-side processes in this particular are taking a long time, causing your client requests to block, making the whole thing seem frozen.

From there, I would attempt to replicate the long running server side processes by creating isolated client side tests - perhaps if the URLs are HTTP gets, I would test the same URLs individually. If they were HTTP posts, I'd create an isolated test form if feasible to see what happens with each request. If a long running server side process is found then you have a starting point.

If there are no long running server side processes then it may be JavaScript / client side coding issues that need to be looked into. But definitely when you're working a large, unfamiliar project, your best bet is to figure out how to break down the issue into smaller components that can then be tested

Answer 2

I solved the issue finally. Here is what I did:

1. Experimented with IIS settings and App_Pool recycling and noticed that there is nothing wrong with the way it handles requests that actually reach it.

2. I focused on the Http.sys module and noticed that in the log files there were a lot of Timer_ConnectionIdle and Client_Reset errors.

3. After some more experimentation and a lot of Google searches, I accidentally found this answer and it solved my issue. As the answer suggests the problem was caused by the AVG antivirus installed and incorrectly configured on the server.

Thanks for all the help and suggestions.