Apache + Tomcat - Problems with sticky sessions and load balancing

Question

I'm facing some issues with Apache mod_proxy_balancer regarding sticky sessions.

We've developped a restful webservice in Java, running on Tomcat. The actual backend is using Acegi security, with Auth Basic authentication.

The architecture is (sorry I'm a new user, I can't post images) :

     --------------------
     |Java Reverse Proxy|
     --------------------
            |
     --------------------
     |Apache load balancer|
     --------------------               
            |
    --------|--------
    |               |
--------        --------    
|tomcat1|       |tomcat2|
--------        --------    

We have this "Java Reverse Proxy" to perform various business things. It also does the Basic Auth authentication on the Tomcat (Tomcat1, Tomcat2).

The end-user calls urls like : http:///a/b?username=foo&password=bar&session=xxx

The reverse proxy then proxies the request to Apache, sending along the credentials as Basic Auth tokens.

The end-user has three different urls:

http://<java reverse proxy domain>/service1
http://<java reverse proxy domain>/service2
http://<java reverse proxy domain>/service3

Only service1 and service2 are protected through Acegi. service3 is anonymously accessible (this is a requirement).

We've have the following configuration in Apache to perform the load balancing:

<Proxy balancer://cluster>
    Header set Cache-Control no-cache
    Header set Pragma no-cache
    BalancerMember http://xxx:9671 route=server1
    BalancerMember http://xxx:9672 route=server2
</Proxy>

ProxyPreserveHost On
ProxyPass / balancer://cluster/ stickysession=JSESSIONID
ProxyPassReverse / balancer://cluster/ stickysession=JSESSIONID

On the first call to service1, then JSESSIONID is returned to the user, and then he sends this session information as part of the request (in the query string, session parameter)

To maintain sessions states in the backend tomcats (tomcat1, tomcat2), the java reverse proxy gets the session from the query string and sends it to the proxied tomcats as a JSESSIONID cookie.

Everything works perfectly fine for the URLS which are auth basic protected. But then when the user calls the third url (which is publicly available), Apache doesn't perform the load balancing correctly.

For instance, when I call service 1 or 2, I get the following Apache logs:

[Wed Feb 22 10:48:52 2012] [debug] mod_proxy_balancer.c(280): proxy: BALANCER: Found value "3FB8F8135173BBBE78E5E4BBD6F5C8FB" for stickysession JSESSIONID
[Wed Feb 22 10:48:52 2012] [debug] mod_proxy_balancer.c(1003): proxy: Entering byrequests for BALANCER (balancer://cluster)
[Wed Feb 22 10:48:52 2012] [debug] mod_proxy_balancer.c(1046): proxy: byrequests selected worker "http://xxx:9672" : busy 0 : lbstatus 1

Which is perfectly fine, as the request is meant to target tomcat2.

But then when I call service3, I get :

[Wed Feb 22 10:49:27 2012] [debug] mod_proxy_balancer.c(280): proxy: BALANCER: Found value "3FB8F8135173BBBE78E5E4BBD6F5C8FB" for stickysession JSESSIONID
[Wed Feb 22 10:49:27 2012] [debug] mod_proxy_balancer.c(1003): proxy: Entering byrequests for BALANCER (balancer://cluster)
[Wed Feb 22 10:49:27 2012] [debug] mod_proxy_balancer.c(1046): proxy: byrequests selected worker "http://xxx:9671" : busy 0 : lbstatus 0

As you can see, even though the JSESSIONID cookie is the same, Apache sends the request to the wrong tomcat (here tomcat1).

Could it be the fact that the url for service3 doesn't require Auth Basic authentication, wherease service1 and service2 do ?

I'm pretty sure I've done someting wrong, but I've been looking around for a long time, and I can't get it to work.

Your help is very much appreciated.

Thanks

Answer 1

Maybe tis will help. This is my config on a web server:

<Proxy balancer://hybriscluster>
BalancerMember ajp://tomcatServer1:8009 route=tomcat1 keepalive=On ping=5 max=200 ttl=120
BalancerMember ajp://tomcatServer2:8009 route=tomcat2 keepalive=On ping=5 max=200 ttl=120
ProxySet stickysession=JSESSIONID|jsessionid lbmethod=byrequests timeout=60
</Proxy>

Config in server.xml of tomcat Server 1:

<Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1"}">

Answer 2

I don't see a jvmRoute suffix on your JSESSIONID. mod_proxy uses jvmRoute to correctly route sticky sessions to your Tomcat instances. jvmRoute is declared in your tomcat server configuration (where each server instance has its own unique jvmRoute identifier.