I am currently using EventingBasicConsumer from RabbitMQClient.dll C# client, we spawn a different thread to handle each message that is delivered to the consumer.
We encountered a strange behavior, the RabbitMQ server closes connections at
times with the error missed heartbeats from client, timeout: 60s
. Few moments later the client reports an error saying Already closed: The AMQP operation was interrupted: AMQP close-reason, initiated by Library, code=541
. I also see this error client unexpectedly closed TCP connection
happening more frequently.
In some situations the clients may take more than 60 seconds to process one job request and this error happens under such conditions.
Is it required that a job be processed within 60 seconds ?, because for our process this can vary between 30 seconds to 5 minutes.
RabbitMQ server: 3.6.6 RabbitMQ.Client.dll (C# client): RabbitMQ.Client.4.1.1
Any insight into this issue is greatly appreciated.
The default connection timeout for the RabbitMQ connection factory is 600 seconds (at least in the Java client API), hence your 10 minutes. You can change this by specifying to the connection factory your timeout of choice.
RabbitMQ can also process a million messages per second but requires more resources (around 30 nodes).
Acknowledgement is basically a notification for RabbitMQ broker that a particular message has been processed and the broker doesn't need to worry about it anymore. In particular it shouldn't redeliver to message to this or some other consumer.
Below is the default TCP socket option configuration used by RabbitMQ: TCP connection backlog is limited to 128 connections.
I used to run much longer jobs (minutes) with EasyNetQ. It's more high-level client that wraps RabbitMQ.Client
.
For me, the reason of these errors is something like Evk wrote in this comment. I would try EasyNetQ as it likely has fetching of messages decoupled from the handling process.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With