Menu
I already know the answer to this, but wanted to share with the community since it is NOT documented from Microsoft.
The scenario: A surge of traffic hits your IIS 7.5 ASP.NET website, and you notice that requests start queuing up. The performance of the site slows to a crawl, yet you have plenty of CPU and RAM available.
This is the problem we saw recently with a site that made a bunch of internal web-service calls. An internal health check would start timing-out, which would cause this server to drop out of our cluster. (However, this server is the most powerful hardware of the bunch ...)
515
After searching around the internet, I found the following articles from Microsoft that relate to the problem:
KB 821268: Contention, poor performance, and deadlocks when you make Web service requests from ASP.NET applications
This article gives some great performance-tweaking tips, however it fails to mention some some VERY important ceilings that we ran in to.
The solution for us was to modify our machine.config, and populate the following XML nodes:
<system.web>
<processModel autoConfig="false" maxWorkerThreads="xxx" maxIoThreads="xxx" minWorkerThreads="xxx" minIoThreads="xxx" requestQueueLimit="5000" responseDeadlockInterval="00:03:00"/>
<httpRuntime minFreeThreads="xxx" minLocalRequestFreeThreads="xxx"/>
</system.web>
I purposefully set some of these numbers to "xxx" since they are dependent upon your hardware.
From the KB article above, Microsoft suggests some equations for figuring out these values. However, they fail to mention that the MAXIMUM value for these numbers is the size of an INT, or 32767.
So, the CORRECT equations for figuring these out are as follows:
This solution is not for everyone, and should only be used if you meet the criteria in the KB article.
Another tool we used in helping us diagnose this was an .ASPX page with no code-behind that we could throw out on any server (without resetting the Application Pool). This page uses reflection to tell you what is actually going on in the thread pool, and what the values of these settings render to on your server.
<%@ Page Language="C#" %>
<!DOCTYPE html>
<html lang="en">
<head>
<style>
body { margin: 20pt; padding: 0pt; font-family: Verdana, "san-serif";}
fieldset { border-radius: 5px; border: none; background-color: #fff; margin: 10pt;}
fieldset.parent { background-color: #f0f0f0; }
legend { font-size: 10pt; color: #888; margin: 5pt; }
.ports div { padding: 10pt 0pt 0pt 0pt; clear: both; }
.ports div:first-child { padding: 0pt; }
.ports div div { padding: 0pt; clear: none; margin: 1pt; background-color: #eef; display: block; float: left; border: 5pt solid #eef; }
.ports div div:first-child { border-top-left-radius: 5pt; border-bottom-left-radius: 5pt; background-color: #ccf; border-color: #ccf;}
.ports div div:last-child { border-top-right-radius: 5pt; border-bottom-right-radius: 5pt; background-color: #ccf; border-color: #ccf; padding: 0pt 10pt 0pt 10pt; }
</style>
</head>
<body>
<%
Response.Cache.SetCacheability(HttpCacheability.NoCache);
int worker, workerMIN, workerMAX;
int port, portMIN, portMAX;
System.Threading.ThreadPool.GetAvailableThreads(out worker, out port);
System.Threading.ThreadPool.GetMinThreads(out workerMIN, out portMIN);
System.Threading.ThreadPool.GetMaxThreads(out workerMAX, out portMAX);
%>
<fieldset class="parent">
<legend>Thread Information</legend>
<fieldset>
<legend>Worker Threads</legend>
<div class="ports">
<div>
<div>Min: <%=workerMIN %></div>
<div>Current: <%=workerMAX - worker %></div>
<div>Max: <%=workerMAX %></div>
</div>
</div>
</fieldset>
<fieldset>
<legend>Completion Port Threads</legend>
<div class="ports">
<div>
<div>Min: <%=portMIN %></div>
<div>Current: <%=portMAX - port %></div>
<div>Max: <%=portMAX %></div>
</div>
</div>
</fieldset>
<fieldset>
<legend>Request Queue Information</legend>
<div class="ports">
<%
var fi = typeof(HttpRuntime).GetField("_theRuntime", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.Static).GetValue(null);
var rq = typeof(HttpRuntime).GetField("_requestQueue", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance).GetValue(fi);
var fields = rq.GetType().GetFields(System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance);
foreach (var field in fields)
{
string name = field.Name;
string value = "";
switch (name)
{
case "_localQueue":
case "_externQueue":
System.Collections.Queue queue = field.GetValue(rq) as System.Collections.Queue;
value = queue.Count.ToString();
break;
default:
value = field.GetValue(rq).ToString();
break;
}
%>
<div>
<div><%=name %></div>
<div><%=value %></div>
</div>
<%
//Response.Write(string.Format("{0}={1}<br/>", name, value));
}
%>
</div>
</fieldset>
</fieldset>
</body></html>
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With