Apache with JBOSS using AJP (mod_jk) giving spikes in thread count

Question

We used Apache with JBOSS for hosting our Application, but we found some issues related to thread handling of mod_jk.

Our website comes under low traffic websites and has maximum 200-300 concurrent users during our website's peak activity time. As the traffic grows (not in terms of concurrent users, but in terms of cumulative requests which came to our server), the server stopped serving requests for long, although it didn't crash but could not serve the request till 20 mins. The JBOSS server console showed that 350 thread were busy on both servers although there was enough free memory say, more than 1-1.5 GB (2 servers for JBOSS were used which were 64 bits, 4 GB RAM allocated for JBOSS)

In order to check the problem we were using JBOSS and Apache Web Consoles, and we were seeing that the thread were showing in S state for as long as minutes although our pages take around 4-5 seconds to be served.

We took the thread dump and found that the threads were mostly in WAITING state which means that they were waiting indefinitely. These threads were not of our Application Classes but of AJP 8009 port.

Could somebody help me in this, as somebody else might also got this issue and solved it somehow. In case any more information is required then let me know.

Also is mod_proxy better than using mod_jk, or there are some other problems with mod_proxy which can be fatal for me if I switch to mod__proxy?

The versions I used are as follows:

Apache 2.0.52
JBOSS: 4.2.2
MOD_JK: 1.2.20
JDK: 1.6
Operating System: RHEL 4

Thanks for the help.

Guys!!!! We finally found the workaround with the configuration mentioned above. It is use of APR and is mentioned here: http://community.jboss.org/thread/153737. Its issue as correctly mentioned by many people in answers below i.e. connector issue. Earlier we made temporary workaround by configuring hibernate and increasing response time. The full fix is APR.

Answer 1

We are experiencing similar issues. We are still working on solutions, but it looks like alot of answers can be found here:

http://www.jboss.org/community/wiki/OptimalModjk12Configuration

Good luck!

Answer 2

Deploy the Apache native APR under jboss/bin/native.

Edit your jboss run.sh to make sure it is looking for the native libs in the right folder.

This will force jboss to use native AJP connector trheads rather than the default pure-java ones.

Answer 3

You should also take a look at the JBoss Jira issue, titled "AJP Connector Threads Hung in CLOSE_WAIT Status":

https://jira.jboss.org/jira/browse/JBPAPP-366