MQTT_ERROR_SEND_BUFFER_IS_FULL error periodically about mqtt-c

[LiamBindle]

Hi @daanpape, thanks for writing.

That is strange, especially since your messages are QOS 0. Since they're QoS 0, all your publish messages should go to the MQTT_QUEUED_COMPLETE state as soon as __mqtt_send (called in mqtt_sync) is called (that happens here).

Do you think it's possible there's a message at the head of your message queue that remains in some unackknowledged state (e.g. your connection request)? When mqtt_mq_clean is called, it can only free-up space from the start of the buffer to the first message that is not MQTT_QUEUED_COMPLETE. If, for example, you never receive the CONNACK packet from the broker, the CONNECT message at the head of your queue would prevent the entire message queue from ever being cleaned (here is where the received CONNACK puts your initial CONNECT message into the completed state). To me it seems like there must be something at the start of your queue that goes unacknowledged. Do you think this is possible?

To troubleshoot your issue, you could try inspecting your message queue (here is the documentation for working the with the internal message queue...it's pretty straightforward to inspect it). You could essentially loop through each message in the queue (you can get the messages with mqtt_mq_get) and inspect its state (e.g. MQTT_QUEUED_AWAITING) and control type (e.g. MQTT_CONTROL_PUBLISH).

Other questions:

1. Are you using the default Unix PAL?
2. Are you working on any unusual architectures or platforms?
3. What broker are you using?

[daanpape]

Hello @LiamBindle,

Thank you for the fast reply. I'm first going to answer your other questions:

1. Yes I'm using an unmodified PAL file.
2. I'm running the software on OpenWRT on a MIPS processor
3. I'm using a broker i wrote myself but which is tested with multiple MQTT clients

To answer your first question: I learned that everything is possible :), ok now to be more serious. I highly doubt that there would be a pending CONACK. I have monitored traffic between my broker and the mqttfx client application with the following result:

It seems to me that the broker responds correctly with a CONACK. I'm now going to look to the docs and try to inspect the transmit queue. I will update with more info soon.

Kind regards,
Daan

[daanpape]

Hello @LiamBindle,

I managed to see the problem by inspecting the message queue:

MQTT_PAL_MUTEX_LOCK(&client.mutex);

        for(size_t i = 0; i < mqtt_mq_length(&(client.mq)); ++i) {
            struct mqtt_queued_message *msg = mqtt_mq_get(&(client.mq), i);

            log_debug("MQTT", "Message %d has state %s and control type %s\n", i, _mqtt_state_to_str(msg->state), _mqtt_control_type_to_str(msg->control_type)); 
        }

        MQTT_PAL_MUTEX_UNLOCK(&client.mutex);

And I now get the output:

[2019-12-03 19:24:14][DEBUG][MQTT] Message 0 has state MQTT_QUEUED_AWAITING_ACK and control type MQTT_CONTROL_SUBSCRIBE
[2019-12-03 19:24:14][DEBUG][MQTT] Message 1 has state MQTT_QUEUED_AWAITING_ACK and control type MQTT_CONTROL_SUBSCRIBE
[2019-12-03 19:24:14][DEBUG][MQTT] Message 2 has state MQTT_QUEUED_AWAITING_ACK and control type MQTT_CONTROL_SUBSCRIBE
[2019-12-03 19:24:14][DEBUG][MQTT] Message 3 has state MQTT_QUEUED_COMPLETE and control type MQTT_CONTROL_PUBLISH

As you expected something keeps hanging at the start of the queue. I'm going to fix this in my broker and report back here. But I think you can set this issue as handled.

I'm sorry for the disturbance and thank you very much for helping me out and pointing me in the right direction.

[daanpape]

I have now updated my broker implementation and this completely fixed the problem. Thanks again @LiamBindle

[LiamBindle]

@daanpape Oh great, I'm glad you got your issue solved 😄

Let me know if you run into any more problems!