On my JMS applications we use temporary queues on Producers to be able to receive replies back from Consumer applications.
I am facing exactly same issue on my end as mentioned in this thread: http://activemq.2283324.n4.nabble.com/jira-Created-AMQ-3336-Temporary-Destination-errors-on-H-A-failover-in-broker-network-with-Failover-tt-td3551034.html#a3612738
Whenever I restarted an arbitrary broker in my network, I was getting many errors like this in my Consumer application log while trying to send reply to a temporary queue:
javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-queue://ID:...
Then I saw Gary's response there suggesting to use
jms.watchTopicAdvisories=false
as a url param on the client brokerURL
. I promptly changed my client broker URLs with this additional parameter. However now I am seeing errors like this when I restart my brokers in network for this failover testing:
javax.jms.JMSException: The destination temp-queue: //ID:client.host-65070-1308610734958-2:1:1 does not exist.
I am using ActiveMQ 5.5 version. And my client broker URL looks like this:
failover:(tcp://amq-host1:61616,tcp://amq-host2.tred.aol.com:61616,tcp://amq-host3:61616,tcp://amq-host4:61616)?jms.useAsyncSend=true&timeout=5000&jms.watchTopicAdvisories=false
Additionally here is my activemq config XML for one of the 4 brokers: amq1.xml
Can someone here please look into this problem and suggest me what mistake I am making in this setup.
To clarify further on how I am doing request-response in my code:
The Failover transport randomly chooses one of the composite URIs and attempts to establish a connection to it. If it does not succeed, or if it subsequently fails, a new connection is established choosing one of the other URIs randomly from the list. Configuration Syntax.
There is no arbitrary limit on the number of queues. The only limitation is the resources available to the JVM as each new queue will consume heap memory not just for the messages in the queue but for the queue's own data-structures.
Generally speaking, ActiveMQ Artemis is notably faster than ActiveMQ "Classic" due to the significant architectural differences between them. In short, ActiveMQ Artemis was designed to be completely non-blocking and performs very well at scale compared to ActiveMQ "Classic".
There is a broker attribute, org.apache.activemq.broker.BrokerService#cacheTempDestinations that should help in the failover: case. Set that to true in xml configuration, and a temp destination will not be removed immediately when a client disconnects. A fast failover: reconnect will be able to producer and/or consume from the temp queue again.
There is a timer task based on timeBeforePurgeTempDestinations (default 5 seconds) that handles cache removal.
One caveat though, I don't see any tests in activemq-core that make use of that attribute so I can't give you any guarantee on this one.
Temporary queues are created on the broker to which the requestor (producer) in your request-reply scenario connects. They are created from a javax.jms.Session
, so on that session disconnecting, either because of client disconnect or broker failure/failover, those queues are permanently gone. None of the other brokers will understand what is meant when one of your consumers attempts to reply to those queues; hence your exception.
This requires an architectural shift in mindset assuming that you want to deal with failover and persist all your messages. Here is a general way that you could attack the problem:
queue:response.<client id>
. The client id might be a standard name if you have a limited number of clients, or a UUID if you have a large number of these.JMSCorrelationID
header, and ought to be copied from the request to the response message.This is a similar approach to that taken by Apache Camel for request-response over messaging.
One thing to be mindful of is that the queue will not go away when the client does, so you should set a time to live on the response message such that it gets deleted from the broker if it has not been consumed, otherwise you will get a backlog of unconsumed messages. You will also need to set up a dead letter queue strategy to automatically discard expired messages.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With