Paho MQTT client behaviour when broker times out / client disconnects

Question

I have a number of QoS2 level messages that are causing problems in the scenario when the MQTT broker or client is having issues. These issues can include

• client starts to see server timeouts
• client has lost connectivity with the broker (internet connection down , issue with the broker, ....) for a while and reconnects.

Typically, when the MQTT client starts receiving timeouts or other errors from the broker, the message are stored in the persistence storage (in-flight messages) and will eventually get republished.

However, in the case where the Paho client loses the connection to the broker, messages will not be considered in-flight anymore and will not be stored by Paho. At that point it seems that the app becomes responsible for persisting these msgs (outside of paho) and re-publishing them.

Am I correct in saying that when the MQTT broker becomes unavailable, the Paho MQTT client cannot help me out in guaranteeing that these QoS2 level messages will get re-delivered ?

So how can I make the distinction between the following case, where client.publish resulted in an MqttException where Paho did not persist the message inflight.

Client is currently disconnecting (32102)
    at org.eclipse.paho.client.mqttv3.internal.ClientComms.shutdownConnection(ClientComms.java:297)
    at org.eclipse.paho.client.mqttv3.internal.CommsSender.handleRunException(CommsSender.java:154)
    at org.eclipse.paho.client.mqttv3.internal.CommsSender.run(CommsSender.java:131)
    at java.lang.Thread.run(Thread.java:745)

And the following where it did persist it inflight

Timed out waiting for a response from the server (32000)
    at org.eclipse.paho.client.mqttv3.internal.Token.waitForCompletion(Token.java:94)
    at org.eclipse.paho.client.mqttv3.MqttToken.waitForCompletion(MqttToken.java:50)
    at org.eclipse.paho.client.mqttv3.MqttClient.publish(MqttClient.java:315)
    at org.eclipse.paho.client.mqttv3.MqttClient.publish(MqttClient.java:307)

Obviously I can also start doing bookkeeping and persist all failed messages seperately, but then I might end up with QoS level 2 duplicaties (messages getting both republished by Paho and myself).

How should the client be programmed ?

• Do need a to do my own message persistence alongside Paho based on exception codes ?
• Do I need to take into account the connectionLost callback and assume that from that point on Paho will not persist anything for me until the MQTT Client reconnects ?
• Before publishing do I need to check if the client is properly connected, and if so assume that Paho will persist the message ?

Here are some exceptions and persistence behaviour by Paho

• Connection lost (32109) : message is persisted by paho
• Client is currently disconnecting (32102) : message is lost by paho
• Timed out waiting for a response from the server (32000) : message is persisted paho
• Client is not connected (32104) : message is lost by paho

What are some of the best practices here with Paho ?

Answer 1

I didn't design the Java client, but I can see how this behaviour came about, and that it can be confusing. I assume that we are talking about the synchronous client here? And that all these exceptions are encountered when calling publish?

The main principles are:

1. if the client API is not connected (including disconnecting) at the point at which the message is to be sent, then no attempt is made to send the message, and it is not persisted.
2. if the client API is currently connected, then the message is attempted to be sent, and it is persisted before doing so. The connection may fail before the QoS 2 exchange completes, in which the message will be retried when the client reconnects.

but both these situations are reported as exceptions, which is unhelpful.

In the C client I follow the rules:

1. if the message is persisted, then success is returned, regardless of whether the QoS 2 exchange completes
2. the connectionLost callback is invoked if the client becomes disconnected

so you know that if you get an error, you must retry the publish call.

For the Java client we could improve the situation by:

1. only throw an exception if the message is not persisted

or

2. create a another class of exceptions which clearly showed when the message was and was not persisted, something like

   • MqttException superclass - message not persisted
   • MqttIncompleteException - message was persisted.

Note that we are planning "offline buffering" for release 1.2 in June, which would mean that the message can be persisted when the client is not connected.