minimize Tomcat "per connection" memory footprint

Question

I have built a custom message server in Java that takes a stream of messages and delivers each message to its client (1:1, drop msg if not connected - very simple). I am running Tomcat 7 on Win7x64 & Java 7 and am using the NIO connector (implemented a Comet servlet). It works great but I am now looking into scaling that beast and am currently seeing about 85kb of RAM allocated for each connected client. 10.000 clients @ under 900MB and scaling linearly. (I am not doing anything else but holding the connection yet) That's quite a lot to my opinion, so I am wondering whether there are some tweaks to make Tomcat or Java save more memory with their NIO impl. All the Tomcat settings I tried so far did not affect this at all.

Does anybody have experience how to put Java or Tomcat on a memory diet regarding socket connections?

UPDATE: I am now down under 70kb/connection by trimming the socket buffers and some other tomcat internals. Not sure how this now influences throughput. I've also tried it on 32bit / 64bit linux with the same result.

Answer 1

After quite some research and playing around I got to the conclusion that it is simply not possible with Tomcat to handle a huge amount of concurrent connections with reasonable amount of memory. (I'd still be happy to be proven wrong here btw)

However, there is a savior:

Netty: http://www.jboss.org/netty/downloads

It's a Java IO framework that builds on Java's new NIO architecture and seems very well designed and written. You can stack some lightweight modules together and create a mini webserver or simply handle the TCP connections yourself in an asynchronous way.

I ran a loadtest on EC2 and made it to a mindblowing 7 MILLION connections @ only 1.5GB of RAM! (like with the tomcat test I did nothing but store the connections, so a real app will of course consume a bit more mem, but 200 bytes / connection "overhead" is nothing!) And it only stopped there because I limited the Java VM to 1.5GB, I am sure a C10M test would be easily doable.

Big kudos to Netty and the Java VM guys! I'm impressed.