MQTT connection times out every 60 seconds due to TCP about espresense

[alexmoras]

Exactly the same issue here with the ESP32-WROOM-32. Exactly 60 seconds of connection before TCP is dropped and it reconnects to Mosquito.

Issue clearly lies with the ESP32 device since I have quite a large MQTT network and no other devices experience this issue. I have another ESP32 device running ESPHome and using MQTT - this is functioning fine.

[leccelecce]

I have turned on debug logging on Mosquitto, which is showing some interesting data. I've filtered on 'PING' and 'disconnecting'.

2024-03-24T15:11:38: Received PINGREQ from espresense-office
2024-03-24T15:11:38: Sending PINGRESP to espresense-office
2024-03-24T15:11:49: Received PINGREQ from espresense-office
2024-03-24T15:11:49: Sending PINGRESP to espresense-office
2024-03-24T15:11:59: Received PINGREQ from espresense-office
2024-03-24T15:11:59: Sending PINGRESP to espresense-office
2024-03-24T15:12:24: Client espresense-office has exceeded timeout, disconnecting.
2024-03-24T15:12:38: Received PINGREQ from espresense-office
2024-03-24T15:12:38: Sending PINGRESP to espresense-office
2024-03-24T15:12:49: Received PINGREQ from espresense-office
2024-03-24T15:12:49: Sending PINGRESP to espresense-office
2024-03-24T15:13:24: Client espresense-office has exceeded timeout, disconnecting.
2024-03-24T15:13:38: Received PINGREQ from espresense-office
2024-03-24T15:13:38: Sending PINGRESP to espresense-office
2024-03-24T15:13:48: Received PINGREQ from espresense-office
2024-03-24T15:13:48: Sending PINGRESP to espresense-office
2024-03-24T15:13:59: Received PINGREQ from espresense-office
2024-03-24T15:13:59: Sending PINGRESP to espresense-office
2024-03-24T15:14:24: Client espresense-office has exceeded timeout, disconnecting.
2024-03-24T15:14:38: Received PINGREQ from espresense-office
2024-03-24T15:14:38: Sending PINGRESP to espresense-office
2024-03-24T15:14:49: Received PINGREQ from espresense-office
2024-03-24T15:14:49: Sending PINGRESP to espresense-office
2024-03-24T15:15:00: Received PINGREQ from espresense-office
2024-03-24T15:15:00: Sending PINGRESP to espresense-office
2024-03-24T15:15:24: Client espresense-office has exceeded timeout, disconnecting.

It looks like ESPresence is usually sending a ping around every 11 seconds. Given the default timeout is 15 seconds, this should normally be sufficient.

However, every 3rd or 4th table it doesn't seem to send a ping, with the result that the broker then terminates the connection.

From what I can see, ESPresence doesn't explicitly send pings but leaves it to the underlying library (AsyncMqttClient) here. You can see it sends a ping every timeout * 0.7 (giving 10.5) which matches up to the ~11 seconds.

https://github.com/marvinroger/async-mqtt-client/blob/89bf46485d5b60ce1e8e5e4d265a9c1570de3dc5/src/AsyncMqttClient.cpp#L331

What isn't clear is what is calling the the onPoll() method that checks whether to send a ping. This comment suggests it's the underlying AsyncTCP library and it should be called every 125ms - question now is to find out whether that's actually happening or not.

The other possibility is that the AsyncMqttClient isn't sending a ping because it thinks there are other messages being sent anyway - next step in debugging is to look for that.

[leccelecce]

Looking at the AsyncMqttClient library, worryingly it's had no commits since 2022.

The current published release (used by ESPresense) was released in 2021. Since then it looks like there was an attempt to fix the ping code in marvinroger/async-mqtt-client#284 but there is no released version with this code.

Looking at that library, and ASyncTCP, it seems like there have been various issues over time and various half-abandoned forks. Unfortunately I'm not sure where that leaves this because it would be non-trivial to swap out the library from ESPresence. My C++ / TCP / hardware build/debugging is also not good enough to figure out where in the stack this is going wrong without investing a load more time.

[leccelecce]

I don't suppose any maintainers with familiarity with these areas of code could comment?