RabbitMQ Java client auto reconnect

Question

When my application looses connection to RabbitMQ I have its connection factory set to automatically try and reconnect

ConnectionFactory factory = new ConnectionFactory();
    factory.setUsername(username);
    factory.setPassword(password);
    factory.setRequestedHeartbeat(1);
    factory.setConnectionTimeout(5000);
    factory.setAutomaticRecoveryEnabled(true);
    factory.setTopologyRecoveryEnabled(true);

When it is trying to reconnect it blocks but it never stops blocking once it gets connected again and I am not to sure why.

I am using the latest version of the java client 3.3.4

This also seems to happen when I force disconnect the client connection via the rabbitmq management interface.

Some further research it seems like its hanging while it is trying to get a channel but the web interface says there is a channel connected.

Answer 1

To wrap it all together, here my answer.

First of all rabbitmq, or at least the java client, has some weird default properties which are sometimes in seconds and sometimes in milliseconds. Also their default values do not seem well thought from my point of view. For example connectionTimeouts default value is 0, which means wait for ever.

You should also set the networkRecoveryInterval property to some usefull value in your environment.

To get a more information what caused the channel to close, implement a ShutdownListener - Even if it just prints the exception, it will help you to find out what caused the error.

To test auto recovery of a connection I used an ip tables command on the client side. Then you can also see what happens on the server side in the rabbitmq.log

And do not forget to adjust the requestHeartbeet property, in my setup its always above 5 seconds, the default one is in some client implementations 580 seconds which seems a bit strange also, keep that in mind.

In any way, even if you run rabbitmq in a cluster, it does not work well on unreliable networks. Maybe you are interested in more regarding this than I should advise you to look in the documentation.