[Bug] Integration stops updating after connectivity issue about homeassistant-mikrotik_router

[tomaae]

I never noticed this myself. How long does it take to stop working for you? is it consistent?
Can you check HA log in case there are some messages?

[kuradi]

ohhooo.... yes, found something in logs:

2020-01-06 15:54:50 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 64, in force_fwupdate_check await self.async_fwupdate_check() File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 117, in async_fwupdate_check await self.hass.async_add_executor_job(self.get_firmware_update) File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 430, in get_firmware_update {'name': 'latest-version', 'default': 'unknown'} File "/config/custom_components/mikrotik_router/helper.py", line 48, in parse_api for entry in source: File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 103, in __iter__ yield from self('print') File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 108, in __call__ **kwargs, File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 28, in __call__ yield from self.readResponse() File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 60, in readResponse reply_word, words = self.readSentence() File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 47, in readSentence words = dict(parse_word(word) for word in words) File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 47, in <genexpr> words = dict(parse_word(word) for word in words) File "/config/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 25, in parse_word _, key, value = word.split('=', 2) ValueError: not enough values to unpack (expected 3, got 2) 2020-01-06 15:54:51 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 57, in force_update await self.async_update() File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 135, in async_update await self.hass.async_add_executor_job(self.get_interface_client) File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 239, in get_interface_client mac2ip, bridge_used = self.update_arp(mac2ip, bridge_used) File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 261, in update_arp for entry in data: File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 103, in __iter__ yield from self('print') File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 108, in __call__ **kwargs, File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 28, in __call__ yield from self.readResponse() File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 60, in readResponse reply_word, words = self.readSentence() File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 47, in readSentence words = dict(parse_word(word) for word in words) File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 47, in <genexpr> words = dict(parse_word(word) for word in words) File "/config/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 25, in parse_word _, key, value = word.split('=', 2) ValueError: not enough values to unpack (expected 3, got 2)

[tomaae]

that is not a root cause. can you find the first error which occurred after HA restart? it should be at the top of your HA log

[kuradi]

those are only ones where is something about mikrotik...

2020-01-06 12:55:24 DEBUG (SyncWorker_8) [homeassistant.components.mikrotik] [192.168.1.1] Connecting to Mikrotik device 2020-01-06 12:55:24 INFO (SyncWorker_8) [homeassistant.components.mikrotik] Mikrotik Connected to MikroTik (192.168.1.1) 2020-01-06 12:55:46 DEBUG (SyncWorker_5) [homeassistant.components.mikrotik.device_tracker] Mikrotik 192.168.1.1: Not a CAPsMAN controller. Trying local wireless interfaces

[tomaae]

Oh, I mean first error. It does not have to be at very top, depending on other messages. But it will be easier to find the first error from the top.
I dont have such issue, so we need to identify what is happening.

[radikovo]

I have the same problem.

2020-01-25 06:25:43 ERROR (SyncWorker_2) [custom_components.mikrotik_router.mikrotikapi] Mikrotik 192.168.88.1 error while path timed out
2020-01-25 06:25:43 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 64, in force_fwupdate_check
    await self.async_fwupdate_check()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 117, in async_fwupdate_check
    await self.hass.async_add_executor_job(self.get_firmware_update)
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 430, in get_firmware_update
    {'name': 'latest-version', 'default': 'unknown'}
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/helper.py", line 48, in parse_api
    for entry in source:
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 103, in __iter__
    yield from self('print')
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 108, in __call__
    **kwargs,
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 28, in __call__
    yield from self.readResponse()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 60, in readResponse
    reply_word, words = self.readSentence()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 46, in readSentence
    reply_word, words = self.protocol.readSentence()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 189, in readSentence
    sentence = tuple(word for word in iter(self.readWord, ''))
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 189, in <genexpr>
    sentence = tuple(word for word in iter(self.readWord, ''))
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 205, in readWord
    word = self.transport.read(length)
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/connections.py", line 25, in read
    data += self.sock.recv((length - len(data)))
socket.timeout: timed out

[tomaae]

2020-01-25 06:25:43 ERROR (SyncWorker_2) [custom_components.mikrotik_router.mikrotikapi] Mikrotik 192.168.88.1 error while path timed out

Okey, this gives more information I needed.
I Just dont see how it cant get past captured error to:

File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/helper.py", line 48, in parse_api
for entry in source:

But this is a good lead. Thanks to this, I know where to look.

[hwmland]

Same issue here, stops working after few hours ut to few days

[tomaae]

If you can post your error message too, it would help. Even if it is very similar, it may show some additional information.
I dont have any issues on my network, so I plan to setup a lab at home to simulate network issues you guys are having, so I can weed out all issues related to connectivity problems.

[hwmland]

Hello, sure, here is my log. It simply works up to today. BTW: I dno't have issues on my network as well. RPi3b running hassio is connected using patch cable directly to mikrotik in question.

2020-02-02 16:22:50 ERROR (SyncWorker_18) [custom_components.mikrotik_router.mikrotikapi] Mikrotik semar.local error while path not enough values to unpack (expected 3, got 2) 2020-02-02 16:22:50 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 64, in force_fwupdate_check await self.async_fwupdate_check() File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 117, in async_fwupdate_check await self.hass.async_add_executor_job(self.get_firmware_update) File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 430, in get_firmware_update {'name': 'latest-version', 'default': 'unknown'} File "/config/custom_components/mikrotik_router/helper.py", line 48, in parse_api for entry in source: File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 103, in __iter__ yield from self('print') File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 108, in __call__ **kwargs, File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 28, in __call__ yield from self.readResponse() File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 60, in readResponse reply_word, words = self.readSentence() File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 47, in readSentence words = dict(parse_word(word) for word in words) File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 47, in <genexpr> words = dict(parse_word(word) for word in words) File "/config/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 25, in parse_word _, key, value = word.split('=', 2) ValueError: not enough values to unpack (expected 3, got 1)

[radikovo]

Next log. First error is "new", second is the same as my previous comment.

2020-01-31 10:30:40 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 57, in force_update
    await self.async_update()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 133, in async_update
    await self.hass.async_add_executor_job(self.get_interface)
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 198, in get_interface
    {'name': 'tx-bits-per-second', 'default': 0}
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/helper.py", line 48, in parse_api
    for entry in source:
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 103, in __iter__
    yield from self('print')
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 108, in __call__
    **kwargs,
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 28, in __call__
    yield from self.readResponse()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 60, in readResponse
    reply_word, words = self.readSentence()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 46, in readSentence
    reply_word, words = self.protocol.readSentence()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 189, in readSentence
    sentence = tuple(word for word in iter(self.readWord, ''))
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 189, in <genexpr>
    sentence = tuple(word for word in iter(self.readWord, ''))
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 205, in readWord
    word = self.transport.read(length)
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/connections.py", line 25, in read
    data += self.sock.recv((length - len(data)))
socket.timeout: timed out
2020-01-31 10:30:40 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 64, in force_fwupdate_check
    await self.async_fwupdate_check()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 117, in async_fwupdate_check
    await self.hass.async_add_executor_job(self.get_firmware_update)
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 430, in get_firmware_update
    {'name': 'latest-version', 'default': 'unknown'}
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/helper.py", line 48, in parse_api
    for entry in source:
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 103, in __iter__
    yield from self('print')
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 108, in __call__
    **kwargs,
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 28, in __call__
    yield from self.readResponse()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 60, in readResponse
    reply_word, words = self.readSentence()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 46, in readSentence
    reply_word, words = self.protocol.readSentence()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 189, in readSentence
    sentence = tuple(word for word in iter(self.readWord, ''))
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 189, in <genexpr>
    sentence = tuple(word for word in iter(self.readWord, ''))
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 205, in readWord
    word = self.transport.read(length)
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/connections.py", line 25, in read
    data += self.sock.recv((length - len(data)))
socket.timeout: timed out

[tomaae]

I'm not able to reproduce this issue no matter what I do. Does anyone know how to reproduce it on purpose?

[tomaae]

I'm running 3 mikrotik devices on 10 sec refresh in my dev environment and my main mikrotik device on 30 sec refresh.
manual cable disconnects or blocking via firewall did not crash it. tho I found and fixed different issue.
I'm keeping them running in hope something will happen, but unless I can reproduce it on purpose, this issue may be difficult to solve.
I'm hoping someone can figure out how to trigger this issue

[hwmland]

Hello, no idea how to reproduce it (it usually works for several days before crash). I have no idea about python/web/HA dev, but from my other development experience: -There is this 'hint' about 'too few parameters to parse': are you perhaps log much more information in such event? (e.g. full buffer you are parsing). This sometime could help. -Even you are not able to reproduce this problem 'live' - what about putting into code bug to produce such error input buffer (something like 'after half an hour of working replace real buffer you received with this faulty one'). My blind gues is that program is not able to recover from such situation (for whatever reason). BR, Marek

…

On Thu, Mar 12, 2020 at 10:49 AM Tomaae ***@***.***> wrote: I'm running 3 mikrotik devices on 10 sec refresh in my dev environment and my main mikrotik device on 30 sec refresh. manual cable disconnects or blocking via firewall did not crash it. tho I found and fixed different issue. I'm keeping them running in hope something will happen, but unless I can reproduce it on purpose, this issue may be difficult to solve. I'm hoping someone can figure out how to trigger this issue — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC6VE7ZQWXR6KEST3YKC56DRHCVZLANCNFSM4KDBBESQ> .

[tomaae]

Connection error itself should be ok, I'm handling them correctly and when I cause connection error on purpose, it will close the connection and reconnect without a problem.
That error is coming from librouteros library. I'm catching exceptions from that library, but somehow this error evades it.
What I'm confused about is that this error is raised when I try to iterate the result. It is probably something python specific.
I think I will modify the library directly and release a version like that.

[hwmland]

Let's try. Such errors which cannot be reproduced in meaningful way (and happen only very sporadically) are painfull to fix:( BR, Marek

…

On Thu, Mar 12, 2020 at 1:16 PM Tomaae ***@***.***> wrote: Connection error itself should be ok, I'm handling them correctly and when I cause connection error on purpose, it will close the connection and reconnect without a problem. That error is coming from librouteros library. I'm catching exceptions from that library, but somehow this error evades it. What I'm confused about is that this error is raised when I try to iterate the result. It is probably something python specific. I think I will modify the library directly and release a version like that. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC6VE7YIVCEVC5N3JGGQM53RHDHBFANCNFSM4KDBBESQ> .

[tomaae]

Yeah, indeed. I have already released 1.2.1 with this change and some more fixes.
I just installed it on my main HA and made a nice graph to monitor it

[hwmland]

Great, I'll install it immediately after returning home. BTW: how did you manage to have it in kbps? Most probably simple, but I did not found it:( BR, Marek

…

On Thu, Mar 12, 2020 at 1:56 PM Tomaae ***@***.***> wrote: Yeah, indeed. I have already released 1.2.1 with this change and some more fixes. I just installed it on my main HA and made a nice graph to monitor it [image: image] <https://user-images.githubusercontent.com/23486452/76523692-1c751400-6469-11ea-999c-c2317b652479.png> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC6VE7YDQ3UKKLONTL3BVT3RHDLXFANCNFSM4KDBBESQ> .

[tomaae]

its part of this release. eventually I would like there to be an option, but kbps for now

[tomaae]

I think I solved this in 1.2.2

[hwmland]

THUMB UP. Thank you. Installing at once. BTW: this kbps (from last update) is really great. I'll report if anything wrong happens (I hope I'll not:-D) BR, Marek

…

On Fri, Mar 13, 2020 at 2:46 AM Tomaae ***@***.***> wrote: I think I solved this in 1.2.2 — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC6VE77A33R5DP6DJRWLS5TRHGGADANCNFSM4KDBBESQ> .

[tomaae]

It is related to #10
I would like to have a configurable option for bps/kbps/mbps eventually. Will have to take a look if it is possible in config flow.

[tomaae]

Found possible issue at one more place, its now fixed in master. It is related just to traffic section.
If you had this issue, please let me know if its fixed now.

[Dyllaann]

Hello!
I found this issue after having some problems with the integration too.
Whenever the integration crashes, it doesn't come up again.
I found two error logs around the time it happened:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/aiohttp/web_protocol.py", line 418, in start
    resp = await task
  File "/usr/local/lib/python3.7/site-packages/aiohttp/web_app.py", line 431, in _handle
    match_info = await self._router.resolve(request)
  File "/usr/local/lib/python3.7/site-packages/aiohttp/web_urldispatcher.py", line 953, in resolve
    match_dict, allowed = await resource.resolve(request)
  File "/usr/src/homeassistant/homeassistant/components/frontend/__init__.py", line 402, in resolve
    and request.url.parts[1] not in self.hass.data[DATA_PANELS]
IndexError: tuple index out of range
2020-03-20 07:23:04 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/config/custom_components/mikrotik_router/librouteros_custom/connections.py", line 27, in read
    tmp = self.sock.recv((length - len(data)))
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 77, in force_update
    await self.async_update()
  File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 167, in async_update
    await self.hass.async_add_executor_job(self.get_interface_client)
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 304, in get_interface_client
    self.update_bridge_hosts(mac2ip)
  File "/config/custom_components/mikrotik_router/mikrotik_controller.py", line 378, in update_bridge_hosts
    for entry in data:
  File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 103, in __iter__
    yield from self('print')
  File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 108, in __call__
    **kwargs,
  File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 28, in __call__
    yield from self.readResponse()
  File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 60, in readResponse
    reply_word, words = self.readSentence()
  File "/config/custom_components/mikrotik_router/librouteros_custom/api.py", line 46, in readSentence
    reply_word, words = self.protocol.readSentence()
  File "/config/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 189, in readSentence
    sentence = tuple(word for word in iter(self.readWord, ''))
  File "/config/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 189, in <genexpr>
    sentence = tuple(word for word in iter(self.readWord, ''))
  File "/config/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 205, in readWord
    word = self.transport.read(length)
  File "/config/custom_components/mikrotik_router/librouteros_custom/connections.py", line 29, in read
    raise ConnectionClosed('Socket recv failed.')
librouteros_custom.exceptions.ConnectionClosed: Socket recv failed.
2020-03-20 07:23:04 ERROR (SyncWorker_1) [custom_components.mikrotik_router.mikrotikapi] Mikrotik 192.168.1.1 error while path Socket recv failed.```

[tomaae]

hmm, not sure whats going on there.
It clearly captured an exception correctly but traceback appeared as well.
this is almost impossible to troubleshoot since I'm not able to reproduce this issue. I think I need to discard librouteros and write my own api for mikrotik to have full control.

[ivanpavlina]

Just got the same error

2020-03-25 17:30:44 ERROR (SyncWorker_2) [custom_components.mikrotik_router.mikrotikapi] Mikrotik 192.168.88.1 error while path Socket recv failed.
2020-03-25 17:30:44 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/connections.py", line 27, in read
    tmp = self.sock.recv((length - len(data)))
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 95, in force_update
    await self.async_update()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 185, in async_update
    await self.hass.async_add_executor_job(self.get_interface_client)
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 307, in get_interface_client
    mac2ip, bridge_used = self.update_arp(mac2ip, bridge_used)
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/mikrotik_controller.py", line 333, in update_arp
    for entry in data:
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 103, in __iter__
    yield from self('print')
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 108, in __call__
    **kwargs,
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 28, in __call__
    yield from self.readResponse()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 60, in readResponse
    reply_word, words = self.readSentence()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/api.py", line 46, in readSentence
    reply_word, words = self.protocol.readSentence()
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 189, in readSentence
    sentence = tuple(word for word in iter(self.readWord, ''))
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 189, in <genexpr>
    sentence = tuple(word for word in iter(self.readWord, ''))
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/protocol.py", line 205, in readWord
    word = self.transport.read(length)
  File "/home/homeassistant/.homeassistant/custom_components/mikrotik_router/librouteros_custom/connections.py", line 29, in read
    raise ConnectionClosed('Socket recv failed.')
librouteros_custom.exceptions.ConnectionClosed: Socket recv failed.

Something is not right with catching the exception, because you are expecting that exception would be handled in mikrotikapi.py #177, but in traceback there is no mention of it.
I'm not really proficient in generators, but i guess the problem is that mikrotikapi is returning generator object which does not contain actual data from mikrotik, only when the loop in mikrotik_controller.py #333 start there is an actual request towards the Mikrotik api

data = self.api.path("/ip/arp")  <-- Not here
if not data:
    return mac2ip, bridge_used
for entry in data:  <-- Here the actual network call is made

[tomaae]

There are actually 2 exceptions. Problem in this case is that I still need to rework get_interface_client.
I will have that done soon, just bit busy with my IRL work at the moment.

As workaround, you can uncheck "Show client MAC and IP" in integration configuration. That will disable the code which still needs reworking.

[ivanpavlina]

There is an underlying issue here, not explicitly with get_interface_client method.
So -
There is a call to mikrotikapi.path object, it builds the generator object as per supplied path:

try:
    response = self._connection.path(path)
except librouteros_custom.exceptions.ConnectionClosed:
    ....

And then there is a transformation to tuple object:

try:
    tuple(response)
except librouteros_custom.exceptions.ConnectionClosed as api_error:
    ...

I presume this is because in this transformation to tuple, api will start reiterating and yielding results from Mikrotik for supplied path - so live data from device is pulled here and this is a check if the connection is alive or not - if not it would be handeled gracefully.

Now, path method is returning that same generator object to the method which requested it, but that the problem - when there is an iteration on that generator object in some other method, the same sequence is repeated - live data is pulled from Mikrotik. And if the connection times out in this moment it will not be properly handled and component will crash - just like it's happening in above exceptions.

Fix (workaround) is to transform data into list for methods that need actual data in the path method and return that list to method which called it, so when there is a iteratation over results data would be stored in memory, it wont go to mikrotik again. This should be only done for methods which arent interested in doing another call to api (ex. run_script, update method are calling commands on generator object, so here the list would break them)

I did changes described above on my dev instance, I'll give it a day or two to see if it will crash without recovering again, it never took survived than a day.

[tomaae]

You are right. this is potential and probably a real issue for people who are affected.
I have been trying to fix this issue pretty much blindly, since I'm not able to reproduce the issue at all.
Let me know how it works out, hopefully together with rewriting of get_interface_client, this issue will be finally fixed.

[ivanpavlina]

Pull request is open, tests were really successful, it ran for 4 days without stopping once.

Also, timeout error didn't occur according to logs, my only explanation for this is that there were two of the same calls one after another for every request of the data from API, so lower-end devices could get stuck for a bit there. Probably that's why you didn't experience any problems, your Mikrotik is a beast :)

[tomaae]

ah, you may be right. My main mikrotik is rb4011 which is really an overkill in terms of cpu power and test ones I have dont have any real load.
I'm going to review your PR and test it on all my devices right away :)

[tomaae]

released v1.4 including this fix

[tomaae]

I did one more improvement in v1.4.1. Let me know in case the issue reappears.

[radikovo]

Great! Thanks.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.