connection problems about opendtu

[tbnobody]

woa.... thats interessting! It shows exactly the issue I think!
Look at the last received telegram!

I just formatted the output a little bit:

    1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
RX 95 81 10  2 41 81 10  2 41  1  0  1  1 45  0 52  0 4F  1  B  1  0  0  0 26 D5 34 
RX 95 81 10  2 41 81 10  2 41  2  0  0 3B 8F  4 EE  5  7  1 46  0 53  0 50  1  D 83 
RX 95 81 10  2 41 81 10  2 41  3  1  6  0  0 14 B4  0  0 42 78  4 F6  5  E  9 3A C1 
RX 95 81 10  2 41 81 10  2 41  0 D9  0 2A  3 D1  1 4C  0  C  B 98 66 

The first line shows the index of the bytes.
Byte 1 - 4: DTU Serial
Byte 5 - 9: Inverter Serial
Byte 10: Fragment Number

But the Fragment Number should never ever be zero! But the last line shows a Fragment number of zero! With this information I can add the required changes to the parser!

[tbnobody]

Hello,

could you please paste any serial output of the specific time? If the serial number is shown as unknown means that the serial of the inverter is invalid. this should not occour during runtime. if it's once detected it should be always the same.

[braini75]

Hello,
thanks for your quick response. I just restartet the ESP32 and do logging of serial output. Let you know once I have those values ;)

[tbnobody]

It would also be interesting which version you are using. Firmware Version + Git Hash. You can find both values in Info --> System dialog.

[pangamut]

I, too have connection problems. Funny thing is, the inverter serial gets preceeded by "100" or "200". See image. After saving inverter settings, everything goes back to normal. Firmware is 0.1.17, Git hash is gebd0643.

[tbnobody]

@pangamut, is this the first version of this sofware that you are using? Or did you also run a previous version?
It would be interesting if e.g. this problem also occours with git hash 96e66dd

(git checkout 96e66dde4743a5d69de0878269f0d00ebdd0ced9)

In the version after this commit I changed some stuff regarding the parser.

[pangamut]

I did use previous versions, and experienced similar problems with them. What puzzles me, is that it works fine for hours or even days, then i get this issue. Saving settings again, everythings fine. Like some bits flip over time.

[tbnobody]

Based on your screenshot I see that your inverter does not have a name. Have you entered a name and the value is gone after the corruption or did you just leave the field empty?

[pangamut]

Oh, i do have a name ("Balkon"), but you are right, its not displayed in the screenshot/error situation. I didn't notice that yet, actually.

[tbnobody]

@braini75 what type of inverter do you have? Does your serial number also begin with 1141?

It seems as there gets some memory overwritten in some place. (In my setup, 2 times 1161 inverter, one is just configured for debug reasons and does not really exist, I don't face any issues, but my uptime is only ~1 day at max because I am testing a lot of stuff)

[pangamut]

I have one HM-600, serial begins with 1141. I switch power off on sunset, and on on sunrise, so uptime this afternoon was about 10hrs.

[tbnobody]

Could you please also paste the information regarding hardware and memory usage? (I just want to see if there are differences regarding heap size etc.)

I will also apply some patches to the source code to double check several package length and also outpput error messages if this conditions occour. If it occurs you should find something like "FATAL: (" in the serial log.

If you run the serial monitor via the Visual Studio PlatformIO extension the serial log should also be logged to a file.

[pangamut]

i can run a serial monitor tomorrow.

[tbnobody]

(Your system output looks exactly like my one...)
I also added a 1141xxxxx serial number to my config but of course I will not get any answers. Currently I suspect a heap corruption in the receiving routine otherwise this problem should occour more often.

[braini75]

My system output looks the same, except my Git Hash is g608456b.
My Inverter is HM-1500 serial 1161xxxxxxxx.
Since this morning everything is running flawless. But I never got more than 24h running time so far.

[tbnobody]

Are you using the MQTT support? What NTP Server settings are you using?
(Just in case that I cannot reproduce that problem within the next 24 or 48hr I want to know what's maybe the different in the setups)

[pangamut]

Yes, i use mqtt with a local server, NTP server is de.pool.ntp.org. I try to have it on a serial console today. unfortunatly, the error happens quite sporadic.

[braini75]

I'm using mqtt and default ntp server pool.ntp.org.
My monitoring last night didn't worked :( Somehow the Computer I used for monitoring had rebooted at night. Now I started monitoring again with a Raspberry Pi connected. ;)

[braini75]

So, the communcation with the inverter just failed again... The log doesn't show much:

[ ttyUSB0: 2022-07-15 18:19:09 ]
[ ttyUSB0: 2022-07-15 18:19:14 ]
Fetch inverter: 116181100241
TX 15 81 10 2 41 78 56 34 12 80 B 0 62 D1 A1 92 0 0 0 5 0 0 0 0 93 5A 8 
Interrupt received
RX 95 81 10 2 41 81 10 2 41 1 0 1 1 45 0 53 0 4F 1 D 1 2 0 0 26 D5 31 
Interrupt received
RX 95 81 10 2 41 81 10 2 41 2 0 0 3B 8F 4 EE 5 7 1 46 0 53 0 51 1 F 80 
Interrupt received
RX 95 81 10 2 41 81 10 2 41 3 1 7 0 0 14 B4 0 0 42 78 4 F6 5 E 9 37 CD 
Interrupt received
RX 95 81 10 2 41 81 10 2 41 84 13 86 3 EF 0 D9 0 2B 3 D2 1 4C 0 C 43 42 B 
RX Period End
Success
[ ttyUSB0: 2022-07-15 18:19:14 ]
[ ttyUSB0: 2022-07-15 18:19:19 ]
Fetch inverter: 116181100241
TX 15 81 10 2 41 78 56 34 12 80 B 0 62 D1 A1 97 0 0 0 5 0 0 0 0 C3 65 62 
Interrupt received
RX 95 81 10 2 41 81 10 2 41 1 0 1 1 45 0 52 0 4F 1 B 1 0 0 0 26 D5 34 
Interrupt received
RX 95 81 10 2 41 81 10 2 41 2 0 0 3B 8F 4 EE 5 7 1 46 0 53 0 50 1 D 83 
Interrupt received
RX 95 81 10 2 41 81 10 2 41 3 1 6 0 0 14 B4 0 0 42 78 4 F6 5 E 9 3A C1 
Interrupt received
RX 95 81 10 2 41 81 10 2 41 0 D9 0 2A 3 D1 1 4C 0 C B 98 66 
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
RX Period End
...

The inverter list still shows the correct inverter number and name, but the "Type" is unkown...

After openning and just saving inverter settings, everthing starts working like a charm and the inverter Type will be set to HM-1200, HM-1500.

[tbnobody]

I added a check in c6499e0
You should try the latest source. Based on the current experience this is maybe a problem in the inverters firmware as a zero packet should not happen. But on the other hand 2 CRC checks of this packet succeeded.

[braini75]

Thanks! That was fast WOW!
I just updated my ESP and will test your implementation.

I also got an idea about Point 2 of my issue:
> Web access not working every morning. No new values ​​arrive at the MQTT either. Only when I press the reset button on the esp32 everything works again.

I think the ESP will not reconnect WIFI connection, when it got lost some time in between! Because the device was logging to serial output this morning as normal, but WIFI connection was lost midnight:
Sa 16. Jul 00:00:05 BST 2022 WiFi lost connectionSa 16. Jul 00:00:05 BST 2022 WiFi lost connection

Every night I reset my Wifi for parental control reasons. So it is offline for few seconds. ;) But ESP will not reconnect...

[tbnobody]

That's very strange. The Arduino Framework offers a setAutoReconnect method which is true by default. But it seems to do nothing in case of ESP32 (https://github.com/espressif/arduino-esp32/search?q=_autoReconnect)

I will doublecheck several things and maybe implement a reconnect by my own.

[ahinrichs]

It does something line 971ff:

I did had similar problems with another project that are gone now. I will check that too.

[tbnobody]

You are right. I was only looking for the internal variable.
But the reconnect feature depends on the following disconnect reasons:

if((reason == WIFI_REASON_AUTH_EXPIRE) || (reason >= WIFI_REASON_BEACON_TIMEOUT && reason != WIFI_REASON_AUTH_FAIL))

Question is, what is a Fritzbox or Speedport doing when disconnecting it's clients. Because if I look at the reason codes >= BEACON_TIMEOUT:

    WIFI_REASON_BEACON_TIMEOUT           = 200,
    WIFI_REASON_NO_AP_FOUND              = 201,
    WIFI_REASON_AUTH_FAIL                = 202,
    WIFI_REASON_ASSOC_FAIL               = 203,
    WIFI_REASON_HANDSHAKE_TIMEOUT        = 204,
    WIFI_REASON_CONNECTION_FAIL          = 205,
    WIFI_REASON_AP_TSF_RESET             = 206,
    WIFI_REASON_ROAMING                  = 207,

it should reconnect e.g. when the AP was not found.

[ahinrichs]

Were having kind of the same ideas. I just researched and the reason gets logged with log_w. @braini75 Could you add a esp_log_level_set("wifi", ESP_LOG_WARN); in src/main.cpp, maybe line 55 where WiFi gets initialized and check, whether the log shows something more helpful tomorrow.

[sivar2311]

Afaik you have to call WiFi.setAutoReconnect(true); after the connection has been established successfully. Calling it before doesn't work.

[braini75]

I just did a restart of my Wifi. I implemented the log_level setup recommandation from @ahinrichs. Following log was generated:

13:49:08.259 > WiFi lost connection
13:49:08.262 > Disconnected from MQTT.
13:49:08.264 > Websocket: [/livedata][2] disconnect
13:49:08.991 > Fetch inverter: 114181025555
13:49:08.994 > TX 15 81 2 55 55 78 56 34 12 80 B 0 62 D3 F7 34 0 0 0 5 0 0 0 0 D2 61 D1 
13:49:09.025 > Interrupt received
13:49:09.027 > RX 95 81 2 55 55 81 2 55 55 1 0 1 1 39 3 70 A C0 1 18 3 C7 A 8F 0 0 4C 
13:49:09.065 > Interrupt received
13:49:09.067 > RX 95 81 2 55 55 81 2 55 55 2 3 EA 0 0 4 78 3 9A 4 2B 9 42 13 89 14 5A 2B 
13:49:09.126 > Interrupt received
13:49:09.129 > RX 95 81 2 55 55 81 2 55 55 83 0 0 0 DC 3 E8 2 D 0 C 87 88 2D 
13:49:09.196 > RX Period End
13:49:09.197 > Success
13:49:13.991 > Fetch inverter: 116181100241
13:49:13.994 > TX 15 81 10 2 41 78 56 34 12 80 B 0 62 D3 F7 39 0 0 0 5 0 0 0 0 42 39 45 
13:49:14.060 > Interrupt received
13:49:14.062 > RX 95 81 10 2 41 81 10 2 41 1 0 1 1 43 2 73 2 75 7 E7 7 ED 0 0 2F D6 22 
13:49:14.080 > Interrupt received
13:49:14.081 > RX 95 81 10 2 41 81 10 2 41 2 0 0 45 13 6 52 6 C8 1 44 2 78 2 75 7 FC E8 
13:49:14.141 > Interrupt received
13:49:14.143 > RX 95 81 10 2 41 81 10 2 41 3 7 F1 0 0 1D CB 0 0 4C 14 6 60 6 D7 9 3E 6E 
13:49:14.181 > Interrupt received
13:49:14.183 > RX 95 81 10 2 41 81 10 2 41 84 13 89 1E 2B 0 E1 1 46 3 E8 2 96 0 6 C 21 4C 
13:49:14.212 > RX Period End
13:49:14.214 > Success
13:49:18.992 > Fetch inverter: 114181025555
13:49:18.995 > TX 15 81 2 55 55 78 56 34 12 80 B 0 62 D3 F7 3E 0 0 0 5 0 0 0 0 72 1F 5 
13:49:19.120 > Interrupt received
13:49:19.122 > RX 95 81 2 55 55 81 2 55 55 83 0 4 0 CE 3 E8 2 C 0 C 33 6D 6B 
13:49:19.207 > RX Period End
13:49:19.208 > Middle missing
13:49:19.210 > Request retransmit: 1
13:49:19.212 > TX 15 81 2 55 55 78 56 34 12 81 1F 
13:49:19.241 > Interrupt received
13:49:19.243 > RX 95 81 2 55 55 81 2 55 55 1 0 1 1 39 3 3A A 18 1 19 3 80 9 D4 0 0 C0 
13:49:19.273 > RX Period End
13:49:19.275 > Middle missing
13:49:19.277 > Request retransmit: 2
13:49:19.279 > TX 15 81 2 55 55 78 56 34 12 82 1C 
13:49:19.325 > Interrupt received
13:49:19.327 > RX 95 81 2 55 55 81 2 55 55 2 3 EA 0 0 4 78 3 9A 4 2B 9 3E 13 89 13 6 C 
13:49:19.342 > RX Period End
13:49:19.344 > Success
13:49:23.508 > Disable search for AP... done
13:49:23.994 > Fetch inverter: 116181100241
13:49:23.997 > TX 15 81 10 2 41 78 56 34 12 80 B 0 62 D3 F7 43 0 0 0 5 0 0 0 0 20 22 46 
13:49:24.035 > Interrupt received
13:49:24.038 > RX 95 81 10 2 41 81 10 2 41 1 0 1 1 42 1 C4 1 C4 5 AF 5 AE 0 0 2F D6 2E 
13:49:24.097 > Interrupt received
13:49:24.099 > RX 95 81 10 2 41 81 10 2 41 2 0 0 45 13 6 52 6 C8 1 42 1 C6 1 C1 5 B5 AF 
13:49:24.155 > Interrupt received
13:49:24.157 > RX 95 81 10 2 41 81 10 2 41 3 5 A7 0 0 1D CC 0 0 4C 15 6 61 6 D8 9 3A 36 
13:49:24.196 > Interrupt received
13:49:24.198 > RX 95 81 10 2 41 81 10 2 41 84 13 89 15 97 0 DE 0 EA 3 E7 2 96 0 6 F2 10 A9 
13:49:24.218 > RX Period End
13:49:24.219 > Success
13:49:28.995 > Fetch inverter: 114181025555
13:49:28.997 > TX 15 81 2 55 55 78 56 34 12 80 B 0 62 D3 F7 48 0 0 0 5 0 0 0 0 10 51 5F 
13:49:29.047 > Interrupt received
13:49:29.048 > RX 95 81 2 55 55 81 2 55 55 1 0 1 1 39 1 C5 5 86 1 1A 1 EF 5 71 0 0 6B 
13:49:29.105 > Interrupt received
13:49:29.107 > RX 95 81 2 55 55 81 2 55 55 2 3 EB 0 0 4 79 3 9B 4 2C 9 39 13 89 A 79 6B 
13:49:29.146 > Interrupt received
13:49:29.147 > RX 95 81 2 55 55 81 2 55 55 83 0 2 0 72 3 E8 2 C 0 C 3C 0 B3 
13:49:29.221 > RX Period End
13:49:29.222 > Success
13:49:33.996 > Fetch inverter: 116181100241
13:49:33.999 > TX 15 81 10 2 41 78 56 34 12 80 B 0 62 D3 F7 4D 0 0 0 5 0 0 0 0 40 6E 64 
13:49:34.041 > Interrupt received
13:49:34.043 > RX 95 81 10 2 41 81 10 2 41 1 0 1 1 40 1 73 1 77 4 A3 4 B0 0 0 2F D7 3B 
13:49:34.079 > Interrupt received
13:49:34.081 > RX 95 81 10 2 41 81 10 2 41 2 0 0 45 14 6 53 6 C9 1 40 1 78 1 78 4 B4 AD 
13:49:34.140 > Interrupt received
13:49:34.143 > RX 95 81 10 2 41 81 10 2 41 3 4 B5 0 0 1D CC 0 0 4C 15 6 61 6 D8 9 3F 20 
13:49:34.183 > Interrupt received
13:49:34.186 > RX 95 81 10 2 41 81 10 2 41 84 13 89 11 CC 0 DA 0 C1 3 E7 2 96 0 6 C 59 6E 
13:49:34.203 > RX Period End
13:49:34.205 > Success

Wifi will connect only. if I push the ESP32 reset button. You should be able to reconstruct the issue. Just restart you wifi. ;)

[ahinrichs]

I'm sorry, but my recommendation was not effective. I did find an old esp here and could do some tests here. Here are my findings:

• autoReconnect is true by default (as @tbnobody said)
• when I restart wifi on my AP, my ESP shows disconnect with reason 2 WIFI_REASON_AUTH_EXPIRE, but reconnect is not triggerd
• when I - redundantly - set WiFi.setAutoReconnect(true) after connect and then restart wifi on my router it shows disconnect reason 3 WIFI_REASON_AUTH_LEAVE. (Tripple checked that, because that does not make any sense: i set autoReconnect to true that is already true changed the disconnect reason... no idea)

So there must be something different, that prevents the esp reaching the cited reconnect code.

@braini75 : Could you please revert the suggested line and add this

WiFiEventId_t eventID = WiFi.onEvent([](WiFiEvent_t event, WiFiEventInfo_t info){
  Serial.print("WiFi lost connection. Reason: ");
  Serial.println(info.wifi_sta_disconnected.reason);
}, WiFiEvent_t::ARDUINO_EVENT_WIFI_STA_DISCONNECTED);

at the end of WiFiSettingsClass::setupMode(), that is after line 36 in src/WiFiSettings.cpp. That worked here and gave the numeric reason on serial log.

[ahinrichs]

Just restart you wifi.

Can you please check with my kids, that I need to "just restart the wifi" several times on sunday :-)

[braini75]

Thanks! Don't touch the Wifi :)

I got Reason 3:

23:29:36.137 > Nothing received, resend count exeeded
23:29:37.041 > WiFi lost connection. Reason: 3
23:29:37.045 > WiFi lost connection
23:29:37.047 > Disconnected from MQTT.

[ahinrichs]

More infos here. Firmware with CORE_DEBUG=5 gives (ignore HoyMiles error):

23:01:03.127 > Starting OpenDTU
23:01:03.127 > Initialize FS... done
23:01:03.127 > Reading configuration... done
23:01:03.127 > Initialize WiFi... [    94][W][WiFiGeneric.cpp:852] _eventCallback(): Arduino Event: 0 - WIFI_READY
23:01:03.127 > done
23:01:03.127 > Setting Hostname... done
23:01:03.127 > Configuring WiFi STA using [   183][V][WiFiGeneric.cpp:283] _arduino_event_cb(): STA Started
23:01:03.127 > new credentials... [   184][W][WiFiGeneric.cpp:852] _eventCallback(): Arduino Event: 2 - STA_START
23:01:03.127 > [   197][V][WiFiGeneric.cpp:96] set_esp_interface_ip(): Configuring Station static IP: 0.0.0.0, MASK: 0.0.0.0, GW: 0.0.0.0
23:01:03.127 > done
23:01:03.127 > Initialize NTP... done
23:01:03.127 > Initialize MqTT... done
23:01:03.127 > Initialize WebApi... done
23:01:03.127 > Initialize Hoymiles interface... Connection error!!
23:01:03.127 > done
23:01:03.127 > [   251][V][WiFiGeneric.cpp:295] _arduino_event_cb(): STA Connected: SSID: IoT, BSSID: e2:94:f6:xx:xx:xx, Channel: 1, Auth: WPA2_PSK
23:01:03.127 > [   252][W][WiFiGeneric.cpp:852] _eventCallback(): Arduino Event: 4 - STA_CONNECTED
23:01:03.127 > [   286][V][WiFiGeneric.cpp:305] _arduino_event_cb(): STA Got New IP:192.168.1.33
23:01:03.127 > [   286][W][WiFiGeneric.cpp:852] _eventCallback(): Arduino Event: 7 - STA_GOT_IP
23:01:03.127 > [   289][D][WiFiGeneric.cpp:914] _eventCallback(): STA IP: 192.168.1.33, MASK: 255.255.255.0, GW: 192.168.1.254
23:01:03.127 > WiFi connected
23:01:41.108 > [ 38785][V][WiFiGeneric.cpp:300] _arduino_event_cb(): STA Disconnected: SSID: IoT, BSSID: e2:94:f6:xx:xx:xx, Reason: 2
23:01:41.119 > [ 38786][W][WiFiGeneric.cpp:852] _eventCallback(): Arduino Event: 5 - STA_DISCONNECTED
23:01:41.127 > [ 38793][W][WiFiGeneric.cpp:873] _eventCallback(): Reason: 2 - AUTH_EXPIRE
23:01:41.134 > [ 38800][D][WiFiGeneric.cpp:889] _eventCallback(): WiFi Reconnect Running
23:01:41.140 > [ 38808][V][WiFiGeneric.cpp:96] set_esp_interface_ip(): Configuring Station static IP: 0.0.0.0, MASK: 0.0.0.0, GW: 0.0.0.0
23:01:41.152 > WiFi lost connection. Reason: 2
23:01:41.155 > WiFi lost connection
23:03:41.107 > [158787][V][WiFiGeneric.cpp:309] _arduino_event_cb(): STA IP Lost
23:03:41.113 > [158788][W][WiFiGeneric.cpp:852] _eventCallback(): Arduino Event: 9 - STA_LOST_IP

And that was it. Waited 10 minutes.

So, when the ESP is kicked out and happens to get reason 2, then the reconnect is triggered in line 889 independently from autoReconnect setting, because first_connect is true at least one time.

if(first_connect && ((reason == WIFI_REASON_AUTH_EXPIRE) ||
  (reason >= WIFI_REASON_BEACON_TIMEOUT)))
{
  log_d("WiFi Reconnect Running");
  WiFi.disconnect();
  WiFi.begin();
  first_connect = false;
}
else if(WiFi.getAutoReconnect()){
...

Wifi.disconnect(); WiFi.begin(); is called at least once but does not connect. I'm not sure the Wifi gets fully started, because I would expect this to result in some WiFiEvents (timeout, ... something).

[ahinrichs]

I got Reason 3:

As I read the code then reason 3 will not trigger any attempt to reconnect. Unfortunately I haven't found any pattern when or why the reason 2 or 3 occur. Here I mostly get reason 2, sometimes 3.

[braini75]

Hey @ahinrichs ,
it seems like WiFiSettings.loop(); is only doing something in AP mode.
See line 63 in src/WiFiSettings.cpp.

So the main.cpp:WiFiSettings.loop() will do nothing when the ESP is connected to a WIFI (WIFI_STA mode). I guess we might need to do some reconnect in the else path, if WiFi.status() != WL_CONNECTED:

Serial.println("Reconnecting to WiFi..."); WiFi.disconnect(); WiFi.reconnect();

see also: https://randomnerdtutorials.com/solved-reconnect-esp32-to-wifi/

[tbnobody]

I am currently rewriting the WiFiSettingsClass to support multiple interfaces.
My first try will be to just call WiFi.reconnect() in the ARDUINO_EVENT_WIFI_STA_DISCONNECTED event.

[ahinrichs]

Yes, actually I would suggest the same. I've not found a predictive way how to improve the built in reconnect.

Beside this I still wonder, why the first WiFi.begin() does not succeed. In most of my tests the wifi was already restarted and reachable, when it was called.

[tbnobody]

I've just commited an updated version of the connection handling. I've tested it several times by kicking out the client of the WiFi AP. Reconnect worked perfectly. (I didn't power off my WiFi because of a redundant setup)

[tbnobody]

@braini75 ist eigentlich Problem 1 nochmal aufgetreten?

[pangamut]

@braini75 ist eigentlich Problem 1 nochmal aufgetreten?

Bei mir läuft die c6499e0 seit Veröffentlichung ohne Fehler durch. Danke!

[braini75]

Problem 1 scheint tatsächlich behoben zu sein! Vielen Dank @tbnobody !!!

Problem 2 habe ich bei mir Lokal einen fix in der Wifisettings.cpp gemacht, so dass es auch funktioniert. Aber ich habe gerade festgestellt, dass tbnobody scheinbar schon überarbeitet und eine neue Klasse bereitstellt. Bin gespannt!

[tbnobody]

Closed from my side... If one of the errors still exist, be free to open this issue again.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new discussion or issue for related concerns.