How to setup a Spring Boot application with embedded tomcat session clustering?

Question

I want to setup a Spring Boot application with embedded tomcat session clustering.

Since embedded tomcat does not have a server.xml file, I've created a TomcatEmbeddedServletContainerFactory and programmatically setup the cluster configuration. The code is as follows:

@Configuration
public class TomcatConfig
{
    @Bean
    public EmbeddedServletContainerFactory servletContainerFactory()
    {
        return new TomcatEmbeddedServletContainerFactory()
        {
            @Override
            protected TomcatEmbeddedServletContainer getTomcatEmbeddedServletContainer(
                Tomcat tomcat)
            {
                configureCluster(tomcat);
                return super.getTomcatEmbeddedServletContainer(tomcat);
            }

            private void configureCluster(Tomcat tomcat)
            {
                // static membership cluster 

                SimpleTcpCluster cluster = new SimpleTcpCluster();
                cluster.setChannelStartOptions(3);
                {
                    DeltaManager manager = new DeltaManager();
                    manager.setNotifyListenersOnReplication(true);
                    cluster.setManagerTemplate(manager);
                }
                {
                    GroupChannel channel = new GroupChannel();
                    {
                        NioReceiver receiver = new NioReceiver();
                        receiver.setPort(localClusterMemberPort);
                        channel.setChannelReceiver(receiver);
                    }
                    {
                        ReplicationTransmitter sender = new ReplicationTransmitter();
                        sender.setTransport(new PooledParallelSender());
                        channel.setChannelSender(sender);
                    }
                    channel.addInterceptor(new TcpPingInterceptor());
                    channel.addInterceptor(new TcpFailureDetector());
                    channel.addInterceptor(new MessageDispatch15Interceptor());
                    {
                        StaticMembershipInterceptor membership =
                            new StaticMembershipInterceptor();
                        String[] memberSpecs = clusterMembers.split(",", -1);
                        for (String spec : memberSpecs)
                        {
                            ClusterMemberDesc memberDesc = new ClusterMemberDesc(spec);
                            StaticMember member = new StaticMember();
                            member.setHost(memberDesc.address);
                            member.setPort(memberDesc.port);
                            member.setDomain("MyWebAppDomain");
                            member.setUniqueId(memberDesc.uniqueId);
                            membership.addStaticMember(member);
                        }
                        channel.addInterceptor(membership);
                    }
                    cluster.setChannel(channel);
                }
                cluster.addValve(new ReplicationValve());
                cluster.addValve(new JvmRouteBinderValve());
                cluster.addClusterListener(new ClusterSessionListener());

                tomcat.getEngine().setCluster(cluster);
            }
        };
    }

    private static class ClusterMemberDesc
    {
        public String address;
        public int port;
        public String uniqueId;

        public ClusterMemberDesc(String spec) throws IllegalArgumentException
        {
            String[] values = spec.split(":", -1);
            if (values.length != 3)
                throw new IllegalArgumentException("clusterMembers element " +
                    "format must be address:port:uniqueIndex");
            address = values[0];
            port = Integer.parseInt(values[1]);
            int index = Integer.parseInt(values[2]);
            if ((index < 0) || (index > 255))
                throw new IllegalArgumentException("invalid unique index: must be >= 0 and < 256");
            uniqueId = "{";
            for (int i = 0; i < 16; i++, index++)
            {
                if (i != 0)
                    uniqueId += ',';
                uniqueId += index % 256;
            }
            uniqueId += '}';
        }
    };

    // This is for example. In fact these are read from application.properties
    private int localClusterMemberPort = 9991;
    private String clusterMembers = "111.222.333.444:9992:1";
}

And I tested the code with the following environment:

• Single Windows PC
• 2 Spring boot application instances with different localClusterMemberPort and clusterMembers

Since a cookie does not take the port in account, the cookie that contains JSESSIONID is shared between the two instances.

When the instances are started, tomcat clustering seems to work, since the JSESSIONID value of the requests for the 2 instances are the same. But when I make a request to the second instance after I logged in using the first instance, the second instance does not find the HttpSession. It logs he following message:

w.c.HttpSessionSecurityContextRepository : No HttpSession currently exists
w.c.HttpSessionSecurityContextRepository : No SecurityContext was available from the HttpSession: null. A new one will be created.

Apparently the HttpSession is not being shared. But as the second instance creates a new session, the first instance's login information is cleared and login is invalidated.

What is happening here? Session is shared but HttpSession is not shared?

BTW, I've read that <distributed /> tag must be specified on web.xml for the applications to use tomcat session clustering. But I don't know how to specify it with Spring Boot's no-xml environment. Is this the cause of the problem? Then how can it be specified?

I've searched and found a few documents that show the clustering using Redis. But currently I don't want to add another moving part to my configuration. In my configuration 3~4 nodes are the maximum.

Answer 1

The key was to make the context distributable, and setting manager.

When I modified the code of the question as follows, the session clustering worked.

@Configuration
public class TomcatConfig
{
    @Bean
    public EmbeddedServletContainerFactory servletContainerFactory()
    {
        TomcatEmbeddedServletContainerFactory factory = new TomcatEmbeddedServletContainerFactory()
        {
            ...
        };

        factory.addContextCustomizers(new TomcatContextCustomizer()
        {
            @Override
            public void customize(Context context)
            {
                context.setManager(new DeltaManager());
                context.setDistributable(true);
            }
        });

        return factory;
    }

    ...
} 

For Spring Boot 1.2.4, context.setManager() is not needed. But for Spring Boot to 1.3.0, if context.setManager() is not called, clustering fails and the following log is shown.

2015-11-18 19:59:42.882  WARN 9764 --- [ost-startStop-1] o.a.catalina.ha.tcp.SimpleTcpCluster     : Manager [org.apache.catalina.session.StandardManager[]] does not implement ClusterManager, addition to cluster has been aborted.

I am somewhat worried about this version dependency. So I opened an issue for this.