When does a single JMS connection with multiple producing sessions start becoming a bottleneck?

Question

I've recently read a lot about best practices with JMS, Spring (and TIBCO EMS) around connections, sessions, consumers & producers

When working within the Spring world, the prevailing wisdom seems to be

• for consuming/incoming flows - to use an AbstractMessageListenerContainer with a number of consumers/threads.
• for producing/publishing flows - to use a CachingConnectionFactory underneath a JmsTemplate to maintain a single connection to the broker and then cache sessions and producers.

For producing/publishing, this is what my (largeish) server application is now doing, where previously it was creating a new connection/session/producer for every single message it was publishing (bad!) due to use of the raw connection factory under JmsTemplate. The old behaviour would sometimes lead to 1,000s of connections being created and closed on the broker in a short period of time in high peak periods and even hitting socket/file handle limits as a result.

However, when switching to this model I am having trouble understanding what the performance limitations/considerations are with the use of a single TCP connection to the broker. I understand that the JMS provider is expected to ensure it can be used in the multi-threaded way etc - but from a practical perspective

• it's just a single TCP connection
• the JMS provider to some degree needs to co-ordinate writes down the pipe so they don't end up an interleaved jumble, even if it has some chunking in its internal protocol
• surely this involves some contention between threads/sessions using the single connection
• with certain network semantics (high latency to broker? unstable throughput?) surely a single connection will not be ideal?

On the assumption that I'm somewhat on the right track

• Am I off base here and misunderstanding how the underlying connections work and are shared by a JMS provider?
• is any contention a problem mitigated by having more connections or does it just move the contention to the broker?
• Does anyone have any practical experience of hitting such a limit they could share? Either with particular message or network throughput, or even caused by # of threads/sessions sharing a connection in parallel
• Should one be concerned in a single-connection scenario about sessions that write very large messages blocking other sessions that write small messages?

Would appreciate any thoughts or pointers to more reading on the subject or experience even with other brokers.

Answer 1

When thinking about the bottleneck, keep in mind two facts:

1. TCP is a streaming protocol, almost all JMS providers use a TCP based protocol

2. lots of the actions from TIBCO EMS client to EMS server are in the form of request/reply. For example, when you publish a message / acknowledge a receive message / commit a transactional session, what's happening under the hood is that some TCP packets are sent out from client and the server will respond with some packets as well. Because of the nature of TCP streaming, those actions have to be serialised if they are initiated from the same connection -- otherwise say if from one thread you publish a message and in the exact same time from another thread you commit a session, the packets will be mixed on the wire and there is no way server can interpret the right message from the packets. [ Note: the synchronisation is done from the EMS client library level, hence user can feel free to share one connection with multiple threads/sessions/consumers/producers ]

My own experience is multiple connections always output perform single connection. In a lossy network situation, it is definitely a must to use multiple connections. Under best network condition, with multiple connections, a single client can nearly saturate the network bandwidth between client and server.

That said, it really depends on what is your clients' performance requirement, a single connection under good network can already provides good enough performance.