I'm managing a web site which is running fine for last couple of months over IIS 7.5 built with MVC 3.0 ASP.net. Every now and then we are facing a problem when our AJAX POST request (fired through jQuery) fails as the JSON being posted is getting truncated.
What we've found so far is that for all such request, the "Content-Length" header of the request has more data than we are actually getting in the request.
We've already set maxRequestLength to 51200 in our web.config and I believe the value of maxAllowedContentLength has a pretty large default value (we've not set that in our configuration). Also I have a failed request with "Content-Length" as low as 7301 (bytes) but we managed to get only 2179 bytes of it. So I'm not suspecting this to be hitting any limit.
Request Headers for a problematic request are as follows
Any ideas ??
Update : I've been able to further isolate the problem from our code. Have written an independent controller that accepts a JSON string and just deserializes it. In case of an error, it logs the error.
When I hit this controller in parallel with 150 concurrent threads in a loop of 50 requests then I get a few failures where the JSON that this controller receives is truncated. Now we are strongly focussed on optimizing IIS and reading more about various parameters that may be related (currently we are running with default parameters on IIS).
I strongly feel that 150 concurrent connections should not be a big deal and am sincerely hoping that tuning some parameters we should be able to get past this problem. Once we get past this problem, will share my findings.
*Update 2 (October 8) * : I have further narrowed down the problem. I turned on error log in IIS and discovered that my failed request gets the following error while reading data
BytesReceived = 0
ErrorCode = 2147943395
Error Description = "The I/O operation has been aborted because of either a thread exit or an application request.(0x800703e3)"
I am finding information about this error on iis forums but am yet to experiment with (mutliple) suggestions being given. The following link can be a good starting point for searching more on this
http://forums.iis.net/p/1149787/1917288.aspx
I finally figured out the cause and the fix for the solution. Unfortunately it is not applicable for all my environments but helps the production environment. Find details below
This turns out to be due to a bug in Windows 2008 R2, where it makes asp.net to believe that a client has disconnected even when it hasn't. A hotfix for the same is available and can be found at http://support.microsoft.com/kb/977453. The fix is already part of Windows 2008 R2 SP1 (found from here).
While the hotfix worked for me on one environment running Windows 2008 R2, it can not be applied to Windows 2008 R2 with SP1. Unfortunately the problem can still be reproduced on the environments running on SP1 and remains un-resolved. I have opened a new case on IIS forums for this issue at http://forums.iis.net/t/1192310.aspx and will track it there instead.
To read more about this issue - you can follow the thread at http://forums.iis.net/p/1149787/1917288.aspx
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With