davidfoliveira / homeswitch-api

An API for your home smart switches

homeswitch-api

An API for your home smart switches

Why is this Python2 in the first place?

Because I need it to run on a AR9330 OpenWRT router and the only thing I can get working there is py2.

homeswitch-api's Issues

Crash on EHOSTUNREACH

2020-11-14 19:14:30.370107Z: [INFO] Client 6 crashed
2020-11-14 19:14:30.370153Z: [DEBUG] Exception
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 83, in read
    obj.handle_read_event()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 449, in handle_read_event
    self.handle_read()
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 91, in handle_read
    self.request.push_data(data)
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 209, in push_data
    self._eat_raw_request()
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 234, in _eat_raw_request
    return self._eat_request_body(last_crlf)
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 253, in _eat_request_body
    self.emit('ready')
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit
    remove = [l for l in listeners if not l(*args, **kwargs)]
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__
    self.func(*args, **kwargs)
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 85, in <lambda>
    self.request.on('ready', lambda: self.emit('request', self.request, False))
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit
    remove = [l for l in listeners if not l(*args, **kwargs)]
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__
    self.func(*args, **kwargs)
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 72, in process_request
    self.server.emit('request', self, request, error)
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit
    remove = [l for l in listeners if not l(*args, **kwargs)]
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__
    self.func(*args, **kwargs)
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 50, in on_request
    return self.on_http_request(client, req, error)
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 74, in on_http_request
    self.devices[dev_id].update(discovery_status='online', hw_metadata=value)
  File "/Users/david/projects/homeswitch-api/homeswitch/device.py", line 62, in update
    self.hw.update(hw_metadata=hw_metadata)
  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 182, in update
    self.emit('_ip')
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit
    remove = [l for l in listeners if not l(*args, **kwargs)]
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__
    self.func(*args, **kwargs)
  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 131, in _on_ip_change
    self.get_status(origin='online')
  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 326, in get_status
    }, lambda err, reply: self._get_status_callback(err, reply, callback, origin))
  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 193, in send_command
    return self._connect()
  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 204, in _connect
    self.connection.connect(self.ip, self.port, timeout=self.socket_timeout)
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 47, in connect
    self.socket.start()
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 123, in start
    self.connect((self._host, self._port))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 356, in connect
    raise socket.error(err, errorcode[err])
error: [Errno 65] EHOSTUNREACH
2020-11-14 19:14:30.377572Z: [DEBUG] Stack
  File "bin/hsapid", line 7, in <module>
    main()
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 138, in main
    api.run()
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 129, in run
    asyncorepp.loop(use_poll=True)
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncorepp.py", line 23, in loop
    poll_fun(timeout, map)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 156, in poll
    read(obj)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 87, in read
    obj.handle_error()
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 100, in handle_error
    traceback.print_stack()

Check `Connection reset by peer` and handle it

2020-11-10 20:45:09.098546Z: [WARN] Socket (192.168.10.179:6668) error occurred: <class 'socket.error'> [Errno 54] Connection reset by peer <traceback object at 0x10531bb90>
2020-11-10 20:45:09.098956Z: [ERRO] Socket error on device's bf5d0abdb1e6210180duku connection: ['Traceback (most recent call last):\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 83, in read\n    obj.handle_read_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 446, in handle_read_event\n    self.handle_connect_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 454, in handle_connect_event\n    raise socket.error(err, _strerror(err))\n', 'error: [Errno 54] Connection reset by peer\n']

Happening at the same time as other instability events

Socket connect timeout seems to not be working

For some reason connecting to an ip that doesn't exist takes ages to fail

Fix or handle `Operation timed out` on connect

FIX this:

2020-11-10 20:44:42.755958Z: [WARN] Socket (192.168.10.179:6668) error occurred: <class 'socket.error'> [Errno 60] Operation timed out <traceback object at 0x1053ef710>
2020-11-10 20:44:42.758109Z: [ERRO] Socket error on device's bf5d0abdb1e6 connection: ['Traceback (most recent call last):\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 83, in read\n    obj.handle_read_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 446, in handle_read_event\n    self.handle_connect_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 454, in handle_connect_event\n    raise socket.error(err, _strerror(err))\n', 'error: [Errno 60] Operation timed out\n']

Possibly related to device instability / device down.

Fix slack hook issue - and Move hooks out of the API

2020-11-27 01:26:06.121609Z: [DBUG] Calling hook opentsdb on bf4a36374de48
2020-11-27 01:26:06.121769Z: [DBUG] Calling hook slack on bf4a36374de48
('SETTINGS: ', {u'off_template': u'{device} foi *desligada* por {who}', u'offline_template': u'{device} ficou fora de servi\xe7o', u'on_template': u'{device} foi *ligada* por {who}'})
STATUS UPDATE ON: bf4a36374de48
2020-11-27 01:26:06.122078Z: [ERRO] Exception running interval callback for timer 945cb2db-55d9-465e-8676-f7f89b125ec2:  'ascii' codec can't encode character u'\xe7' in position 26: ordinal not in range(128)
Traceback (most recent call last):
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncorepp.py", line 43, in _check_timers
    call()
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncorepp.py", line 53, in call_and_expire
    return callback()
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 58, in <lambda>
    self.timeout = set_timeout(lambda: self._on_failure({'error': 'timeout'}), timeout)
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 92, in _on_failure
    self.emit('failure', ex)
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit
    remove = [l for l in listeners if not l(*args, **kwargs)]
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__
    self.func(*args, **kwargs)
  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 186, in _on_dev_connection_failure
    None
  File "/Users/david/projects/homeswitch-api/homeswitch/syncproto.py", line 113, in flush
    cmd.reply(err, reply)
  File "/Users/david/projects/homeswitch-api/homeswitch/syncproto.py", line 195, in reply
    return callback(*args)
  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 250, in <lambda>
    }, lambda err, reply: self._get_status_callback(err, reply, callback, origin))
  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 255, in _get_status_callback
    return callback(err, None)
  File "/Users/david/projects/homeswitch-api/homeswitch/device.py", line 99, in <lambda>
    return self.hw.get_status(lambda err, status: self._get_status_callback(err, status, origin, collector_callback), origin)
  File "/Users/david/projects/homeswitch-api/homeswitch/device.py", line 104, in _get_status_callback
    self._check_error(err, origin)
  File "/Users/david/projects/homeswitch-api/homeswitch/device.py", line 153, in _check_error
    self._on_status_update(None, origin='{}.too_many_errors'.format(origin))
  File "/Users/david/projects/homeswitch-api/homeswitch/device.py", line 54, in _on_status_update
    self.emit('status_update', status, origin)
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit
    remove = [l for l in listeners if not l(*args, **kwargs)]
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__
    self.func(*args, **kwargs)
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 39, in <lambda>
    dev.on('status_update', lambda status, origin: self._run_hooks(dev_id, status, origin))
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 59, in _run_hooks
    hook.notify('status_update', dev_hook_settings, {'device': dev, 'status': status, 'origin': origin})
  File "/Users/david/projects/homeswitch-api/homeswitch/hooks/__init__.py", line 9, in notify
    self._hook.notify(notif_type, settings, data)
  File "/Users/david/projects/homeswitch-api/homeswitch/hooks/slack.py", line 55, in notify
    debug("INFO", "Sending slack notification with '{}'".format(notification_text))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe7' in position 26: ordinal not in range(128)

Fix or handle "timeout: timed out" on send()

FIX:

2020-11-10 20:44:44.764720Z: [WARN] Socket (192.168.10.179:6668) error occurred: <class 'socket.timeout'> timed out <traceback object at 0x1053e9998>
2020-11-10 20:44:44.843719Z: [ERRO] Socket error on device's bf5d0abdb1e6 connection: ['Traceback (most recent call last):\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 91, in write\n    obj.handle_write_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 467, in handle_write_event\n    self.handle_connect_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 455, in handle_connect_event\n    self.handle_connect()\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 104, in handle_connect\n    self.emit(\'connect\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 61, in _on_connect\n    self.emit(\'connect\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 201, in _on_dev_connect\n    self.emit(\'_next\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 127, in _on_can_send_next_command\n    self.connection.send(bin_message)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 52, in send\n    self.socket.send(data)\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 374, in send\n    result = self.socket.send(data)\n', 'timeout: timed out\n']

Possibly related to device or network instability.

Avoid Tuya connects while a connect is ongoing

2020-11-12 20:19:15.144826Z: [DBUG] [Client 5] Proto 3 Request: set {u'switches': {u'bf5d0abdb1e62': True}, u'method': u'set'}
2020-11-12 20:19:15.144918Z: [INFO] Setting device u'bf5d0abdb1e62 status to True
2020-11-12 20:19:15.144975Z: [DBUG] Getting device status to True (IP: 192.168.10.178, PORT: 6668, GW_ID: u'bf5d0abdb1e62, KEY: xxxxx)
2020-11-12 20:19:15.145028Z: [DBUG] IP: 192.168.10.178, PORT: 6668, GW_ID: u'bf5d0abdb1e62, KEY: xxxx
2020-11-12 20:19:15.145069Z: [INFO] Connecting to device u'bf5d0abdb1e62 at 192.168.10.178:6668...
2020-11-12 20:19:17.966686Z: [INFO] Client 5 has disconnected

2020-11-12 20:19:19.970068Z: [INFO] Updating device bf4a36374de status...
2020-11-12 20:19:19.970163Z: [INFO] Getting device bf4a36374de status
2020-11-12 20:19:19.970213Z: [DBUG] Getting device status (IP: 192.168.10.182, PORT: 6668, GW_ID: bf4a36374de, KEY: xxxx)
2020-11-12 20:19:19.970435Z: [DBUG] IP: 192.168.10.182, PORT: 6668, GW_ID: bf4a36374de, KEY: xxx
2020-11-12 20:19:19.970509Z: [INFO] Connecting to device bf4a36374de at 192.168.10.182:6668...
2020-11-12 20:19:20.024270Z: [INFO] Connected to 192.168.10.182:6668 !
2020-11-12 20:19:20.024374Z: [INFO] Connected to device bf4a36374de !
2020-11-12 20:19:20.024422Z: [DBUG] We can send next command for bf4a36374de!!!
2020-11-12 20:19:20.024447Z: [DBUG] Sending to bf4a36374de...
2020-11-12 20:19:20.026414Z: [DBUG] Sending 183 bytes to 192.168.10.182:6668 (fd: 5)

2020-11-12 20:19:20.026617Z: [INFO] Updating device u'bf5d0abdb1e62 status...
2020-11-12 20:19:20.026673Z: [INFO] Getting device u'bf5d0abdb1e62 status
2020-11-12 20:19:20.026720Z: [DBUG] Getting device status (IP: 192.168.10.178, PORT: 6668, GW_ID: bf4a36374de, KEY: xxx)
2020-11-12 20:19:20.026762Z: [DBUG] IP: 192.168.10.178, PORT: 6668, GW_ID: u'bf5d0abdb1e62, KEY: xxxx
2020-11-12 20:19:20.026787Z: [INFO] Connecting to device u'bf5d0abdb1e62 at 192.168.10.178:6668...
2020-11-12 20:19:20.027394Z: [ERRO] Exception running interval callback for timer 5e7c4c59-2596-47bd-aa01-6effadf35cb7:  [Errno 64] EHOSTDOWN

Fix attempt to send item that is not in `waiting` state

FIX (slow connect):

2020-11-08 22:08:24.408117Z: [DBUG] Sending to bf4a36374de48f83d8vd9g...
2020-11-08 22:08:24.408142Z: [WARN] I was told I can process the next queued item but the item is not in 'waiting' state

This is possible related to slow connects or reconnects.

Ignore certain send() exceptions

2020-11-13 06:50:24.368618Z: [INFO] Client 5 crashed
2020-11-13 06:50:24.368680Z: [DEBUG] Exception
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 83, in read
    obj.handle_read_event()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 449, in handle_read_event
    self.handle_read()
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 89, in handle_read
    self.request.push_data(data)
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 196, in push_data
    self._eat_raw_request()
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 221, in _eat_raw_request
    return self._eat_request_body(last_crlf)
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 240, in _eat_request_body
    self.emit('ready')
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit
    remove = [l for l in listeners if not l(*args, **kwargs)]
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__
    self.func(*args, **kwargs)
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 83, in <lambda>
    self.request.on('ready', lambda: self.emit('request', self.request, False))
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit
    remove = [l for l in listeners if not l(*args, **kwargs)]
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__
    self.func(*args, **kwargs)
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 70, in process_request
    self.server.emit('request', self, request, error)
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit
    remove = [l for l in listeners if not l(*args, **kwargs)]
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__
    self.func(*args, **kwargs)
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 49, in on_request
    return self.on_http_request(client, req, error)
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 79, in on_http_request
    client.message({'error': 'not_found'})
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 113, in message
    self.send_http(HTTP_STATUS_BY_ERROR.get(body.get('error'), 500), body)
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 132, in send_http
    self.send(response)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 550, in send
    self.initiate_send()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 537, in initiate_send
    num_sent = dispatcher.send(self, self.out_buffer[:512])
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 374, in send
    result = self.socket.send(data)
error: [Errno 41] Protocol wrong type for socket

Understand `Resource temporarily unavailable` error during socket read()

2020-11-14 10:42:12.306371Z: [INFO] Getting device bf5d0abdb1e62 status
2020-11-14 10:42:12.306439Z: [DBUG] Getting Tuya device status (IP: 192.168.10.178, PORT: 6668, GW_ID: bf5d0abdb1e62)
2020-11-14 10:42:12.306505Z: [INFO] Getting device bf4a36374de48 status
2020-11-14 10:42:12.306528Z: [WARN] Device bf4a36374de48 is not online. Cannot get its status
2020-11-14 10:42:12.306837Z: [ERRO] Error getting device statuses:  bf4a36374de48
2020-11-14 10:42:12.388960Z: [INFO] Connected to 192.168.10.178:6668 !
2020-11-14 10:42:12.389063Z: [INFO] Connected to Tuya device bf5d0abdb1e62 !
2020-11-14 10:42:12.389110Z: [DBUG] We can send next command for bf5d0abdb1e62!!!
2020-11-14 10:42:12.389172Z: [DBUG] Sending to bf5d0abdb1e62...
2020-11-14 10:42:12.391450Z: [DBUG] Sending 183 bytes to 192.168.10.178:6668 (fd: 6)
2020-11-14 10:42:12.916375Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-14 10:42:12.916468Z: [INFO] Socket for device's bf5d0abdb1e62 has data and should be read
2020-11-14 10:42:12.916557Z: [DBUG] Got header (prefix: 21930, seq: 0, cmd: 8, size: 75)
2020-11-14 10:42:12.917526Z: [DBUG] Got device bf5d0abdb1e62 status: {u'dps': {u'1': True}, u't': 1605350531}
2020-11-14 10:42:12.917623Z: [INFO] Got a status update about device bf5d0abdb1e62. Device status has NOT changed (True)
2020-11-14 10:42:12.917660Z: [INFO] Device bf5d0abdb1e62 status is True
2020-11-14 10:42:12.917713Z: [DBUG] We can send next command for bf5d0abdb1e62!!!
2020-11-14 10:42:12.917752Z: [DBUG] Sending to bf5d0abdb1e62...
2020-11-14 10:42:12.919566Z: [DBUG] Sending 183 bytes to 192.168.10.178:6668 (fd: 6)
2020-11-14 10:42:12.919728Z: [DBUG] Got header (prefix: 21930, seq: 0, cmd: 7, size: 12)
2020-11-14 10:42:12.919958Z: [WARN] Socket (192.168.10.178:6668) error occurred: <class 'socket.error'> [Errno 35] Resource temporarily unavailable <traceback object at 0x10df67d88>
2020-11-14 10:42:12.924052Z: [ERRO] Socket error on device's bf5d0abdb1e62 connection: ['Traceback (most recent call last):\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 83, in read\n    obj.handle_read_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 449, in handle_read_event\n    self.handle_read()\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 160, in handle_read\n    self.emit(\'read\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 107, in _on_read\n    self.emit(\'data\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 267, in _on_dev_data\n    reply = self._read_and_parse_message()\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 386, in _read_and_parse_message\n    header = self._read_need(16)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 465, in _read_need\n    data = self.connection.receive(num_bytes)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 70, in receive\n    return self.socket.recv(num_bytes)\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 387, in recv\n    data = self.socket.recv(buffer_size)\n', 'error: [Errno 35] Resource temporarily unavailable\n']
2020-11-14 10:42:12.924506Z: [INFO] Disconnecting from Tuya device bf5d0abdb1e62...

Event 'connect' called when connection is actually not established (the IP doesn't exist)

2020-11-13 06:23:22.694746Z: [INFO] Connection to 192.168.11.178:6668 was reset by peer.
2020-11-13 06:23:22.694818Z: [INFO] Device bf5d0abdb1e62 has disconnected.
2020-11-13 06:23:22.694856Z: [INFO] Reconnecting to bf5d0abdb1e62...
2020-11-13 06:23:22.694891Z: [INFO] Disconnecting from device bf5d0abdb1e62...
2020-11-13 06:23:22.694933Z: [DBUG] Closing connecting socket to 192.168.11.178:6668 (fd: 5)
2020-11-13 06:23:22.695017Z: [DBUG] IP: 192.168.11.178, PORT: 6668, GW_ID: bf5d0abdb1e62
2020-11-13 06:23:22.695056Z: [INFO] Connecting to device bf5d0abdb1e62 at 192.168.11.178:6668...
2020-11-13 06:23:22.695250Z: [DBUG] Setting socket timeout to 2
2020-11-13 06:23:24.698955Z: [INFO] Connected to 192.168.11.178:6668 !
2020-11-13 06:23:24.699072Z: [INFO] Connected to device bf5d0abdb1e62 !
2020-11-13 06:23:24.699140Z: [DBUG] We can send next command for bf5d0abdb1e62!!!
2020-11-13 06:23:24.699175Z: [DBUG] Sending to bf5d0abdb1e62...
2020-11-13 06:23:24.702780Z: [DBUG] Sending 183 bytes to 192.168.11.178:6668 (fd: 5)
2020-11-13 06:23:26.705013Z: [WARN] Socket (192.168.11.178:6668) error occurred: <class 'socket.timeout'> timed out <traceback object at 0x1036e9cb0>
2020-11-13 06:23:26.707105Z: [ERRO] Socket error on device's bf5d0abdb1e62 connection: ['Traceback (most recent call last):\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 91, in write\n    obj.handle_write_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 467, in handle_write_event\n    self.handle_connect_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 455, in handle_connect_event\n    self.handle_connect()\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 109, in handle_connect\n    self.emit(\'connect\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 65, in _on_connect\n    self.emit(\'connect\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 219, in _on_dev_connect\n    self.emit(\'_next\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 138, in _on_can_send_next_command\n    self.connection.send(bin_message)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 56, in send\n    self.socket.send(data)\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 374, in send\n    result = self.socket.send(data)\n', 'timeout: timed out\n']

Check issue with slow connects

2020-11-12 22:17:06.610376Z: [INFO] Updating device bf5d0abdb1e62 status...
2020-11-12 22:17:06.610463Z: [INFO] Getting device bf5d0abdb1e62 status
2020-11-12 22:17:06.610529Z: [DBUG] Getting device status (IP: 192.168.10.178, PORT: 6668, GW_ID: bf5d0abdb1e62)
...
2020-11-12 22:17:18.591902Z: [WARN] Socket (192.168.10.178:6668) error occurred: <class 'socket.error'> [Errno 60] Operation timed out <traceback object at 0x10c37bf38>
2020-11-12 22:17:18.592588Z: [ERRO] Socket error on device's bf5d0abdb1e62 connection: ['Traceback (most recent call last):\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 83, in read\n    obj.handle_read_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 446, in handle_read_event\n    self.handle_connect_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 454, in handle_connect_event\n    raise socket.error(err, _strerror(err))\n', 'error: [Errno 60] Operation timed out\n']
2020-11-12 22:17:18.592641Z: [INFO] Disconnecting from device bf5d0abdb1e62...
2020-11-12 22:17:18.592718Z: [INFO] Connected to 192.168.10.178:6668 !
2020-11-12 22:17:18.592753Z: [INFO] Connected to device bf5d0abdb1e62 !
2020-11-12 22:17:18.620466Z: [DBUG] We can send next command for bf5d0abdb1e62!!!
2020-11-12 22:17:18.620543Z: [DBUG] Sending to bf5d0abdb1e62...
2020-11-12 22:17:18.620581Z: [WARN] I was told I can process the next queued item but the item is not in 'waiting' state
2020-11-12 22:17:18.620821Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-12 22:17:18.620872Z: [INFO] Socket for device's bf5d0abdb1e62 has data and should be read
2020-11-12 22:17:18.620960Z: [INFO] Connection to 192.168.10.178:6668 was closed.
2020-11-12 22:17:18.620990Z: [INFO] Connection to 192.168.10.178:6668 was reset by peer.
2020-11-12 22:17:18.621037Z: [INFO] Device bf5d0abdb1e62 has disconnected.
2020-11-12 22:17:18.621066Z: [INFO] Reconnecting to bf5d0abdb1e62...
2020-11-12 22:17:18.621093Z: [INFO] Disconnecting from device bf5d0abdb1e62...
2020-11-12 22:17:18.621122Z: [DBUG] Closing connecting socket to 192.168.10.178:6668 (fd: 5)
2020-11-12 22:17:18.621188Z: [DBUG] IP: 192.168.10.178, PORT: 6668, GW_ID: bf5d0abdb1e62
2020-11-12 22:17:18.621218Z: [INFO] Connecting to device bf5d0abdb1e62 at 192.168.10.178:6668...
2020-11-12 22:17:20.625431Z: [INFO] Connected to 192.168.10.178:6668 !
2020-11-12 22:17:20.625551Z: [INFO] Connected to device bf5d0abdb1e62 !
2020-11-12 22:17:20.625616Z: [DBUG] We can send next command for bf5d0abdb1e62!!!
2020-11-12 22:17:20.625644Z: [DBUG] Sending to bf5d0abdb1e62...
2020-11-12 22:17:20.628311Z: [DBUG] Sending 183 bytes to 192.168.10.178:6668 (fd: 5)
2020-11-12 22:17:22.632660Z: [WARN] Socket (192.168.10.178:6668) error occurred: <class 'socket.timeout'> timed out <traceback object at 0x10c390908>
2020-11-12 22:17:22.633360Z: [ERRO] Socket error on device's bf5d0abdb1e62 connection: ['Traceback (most recent call last):\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 91, in write\n    obj.handle_write_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 467, in handle_write_event\n    self.handle_connect_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 455, in handle_connect_event\n    self.handle_connect()\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 115, in handle_connect\n    self.emit(\'connect\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 67, in _on_connect\n    self.emit(\'connect\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 214, in _on_dev_connect\n    self.emit(\'_next\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 138, in _on_can_send_next_command\n    self.connection.send(bin_message)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 58, in send\n    self.socket.send(data)\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 374, in send\n    result = self.socket.send(data)\n', 'timeout: timed out\n']
2020-11-12 22:17:22.670159Z: [INFO] Disconnecting from device bf5d0abdb1e62...
2020-11-12 22:17:22.670234Z: [DBUG] Closing connecting socket to 192.168.10.178:6668 (fd: 5)

Weird thing:

• Disconnecting..
• Connected!

Infinite loop on "Socket (...) read can happen"

2020-11-18 11:47:26.656414Z: [DBUG] We can send next command to 192.168.10.178:6668!!!
2020-11-18 11:47:26.656446Z: [DBUG] Sending '7' command to 192.168.10.178:6668...
2020-11-18 11:47:26.658734Z: [DBUG] Sending 183 bytes to 192.168.10.178:6668 (fd: 6)
...
2020-11-18 11:47:26.687311Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:26.687422Z: [DBUG] Got header (prefix: 21930, seq: 0, cmd: 7, size: 12)
...
2020-11-18 11:47:27.680932Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:27.681132Z: [INFO] Connection to 192.168.10.178:6668 was closed.
2020-11-18 11:47:27.681171Z: [INFO] Connection to 192.168.10.178:6668 was reset by peer.
2020-11-18 11:47:27.681230Z: [INFO] Device bf5d0abdb1e62 has disconnected.
2020-11-18 11:47:27.681368Z: [DBUG] Detected disconnect on socket 192.168.10.178:6668
2020-11-18 11:47:27.681527Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:27.681656Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:27.681778Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:27.681885Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:27.681989Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:27.682097Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:27.682199Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:27.682302Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:27.682407Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:27.682508Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:27.682617Z: [INFO] Socket (192.168.10.178:6668) read can happen
2020-11-18 11:47:27.682723Z: [INFO] Socket (192.168.10.178:6668) read can happen
...

Fix KeyError on HTTP client disconnect

2020-11-10 21:29:22.075109Z: [DBUG] [Client 14] Proto http Request: POST /api/device/sync
2020-11-10 21:29:22.075191Z: [DBUG] HTTP Request: POST /api/device/sync: {u'bf5d0abdb1e6': {u'ablilty': 0, u'encrypt': True, u'productKey': u'keyaqc', u'ip': u'192.168.10.179', u'gwId': u'bf5d0abdb1e6', u'version': u'3.3', u'active': 2}, u'bf4a36374de': {u'ablilty': 0, u'encrypt': True, u'productKey': u'keyaqu', u'ip': u'192.168.10.182', u'gwId': u'bf4a36374de', u'version': u'3.3', u'active': 2}}
2020-11-10 21:29:22.075708Z: [INFO] Client 14 has disconnected
2020-11-10 21:29:22.075788Z: [INFO] Client 14 has disconnected
error: uncaptured python exception, closing channel <homeswitch.hybridserver.HybridServerClient 127.0.0.1:64101 at 0x10ecd9c90> (<type 'exceptions.KeyError'>:14 [/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py|read|83] [/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py|handle_read_event|449] [/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py|handle_read|83] [/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py|push_data|178] [/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py|_eat_raw_request|203] [/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py|_eat_request_body|222] [/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py|emit|282] [/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py|__call__|310] [/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py|<lambda>|77] [/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py|emit|282] [/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py|__call__|310] [/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py|process_request|65] [/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py|emit|282] [/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py|__call__|310] [/Users/david/projects/homeswitch-api/homeswitch/api.py|on_request|49] [/Users/david/projects/homeswitch-api/homeswitch/api.py|on_http_request|79] [/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py|message|95] [/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py|send_http|115] [/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py|send|550] [/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py|initiate_send|537] [/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py|send|380] [/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py|handle_close|87] [/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py|remove_user|50])
2020-11-10 21:29:22.076166Z: [INFO] Client 14 has disconnected
Traceback (most recent call last):
  File "bin/hsapid", line 7, in <module>
    main()
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 127, in main
    api.run()
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 110, in run
    asyncorepp.loop()
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncorepp.py", line 22, in loop
    poll_fun(timeout, map)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 156, in poll
    read(obj)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 87, in read
    obj.handle_error()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 503, in handle_error
    self.handle_close()
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 87, in handle_close
    self.server.remove_user(self)
  File "/Users/david/projects/homeswitch-api/homeswitch/hybridserver.py", line 50, in remove_user
    del self.clients[client.id]
KeyError: 14

Understand and fix issue with socket error before connect which apparently causes issues with timers

2020-11-14 18:41:53.519334Z: [WARN] Socket (192.168.10.178:6668) error occurred: <class 'socket.error'> [Errno 60] Operation timed out <traceback object at 0x1076e2bd8>
2020-11-14 18:41:53.520179Z: [ERRO] Socket error on device's bf5d0abdb1e62 connection: ['Traceback (most recent call last):\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 91, in write\n    obj.handle_write_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 467, in handle_write_event\n    self.handle_connect_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 454, in handle_connect_event\n    raise socket.error(err, _strerror(err))\n', 'error: [Errno 60] Operation timed out\n']
2020-11-14 18:41:53.520249Z: [INFO] Disconnecting from Tuya device bf5d0abdb1e62...
2020-11-14 18:41:53.520426Z: [INFO] Connected to 192.168.10.178:6668 !
2020-11-14 18:41:53.520552Z: [WARN] Socket (192.168.10.178:6668) error occurred: <type 'exceptions.KeyError'> UUID('5cc6f0dd-63a9-4af0-bd81-036b8fd96cde') <traceback object at 0x107557200>
2020-11-14 18:41:53.523337Z: [ERRO] Socket error on device's bf5d0abdb1e62 connection: ['Traceback (most recent call last):\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 91, in write\n    obj.handle_write_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 467, in handle_write_event\n    self.handle_connect_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 455, in handle_connect_event\n    self.handle_connect()\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 141, in handle_connect\n    self.emit(\'connect\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 75, in _on_connect\n    cancel_timeout(self.timeout)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncorepp.py", line 125, in cancel_timeout\n    del TIMERS[timer_id]\n', "KeyError: UUID('5cc6f0dd-63a9-4af0-bd81-036b8fd96cde')\n"]

Cancel command timeout in case of socket error

2020-11-18 22:10:32.872914Z: [WARN] Socket (127.0.0.1:6668) error occurred: <class 'socket.error'> [Errno 61] Connection refused <traceback object at 0x102e605f0>
2020-11-18 22:10:32.873807Z: [ERRO] Socket error on device's bf5d0abdb1e62 connection: ['Traceback (most recent call last):\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 91, in write\n    obj.handle_write_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 467, in handle_write_event\n    self.handle_connect_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 454, in handle_connect_event\n    raise socket.error(err, _strerror(err))\n', 'error: [Errno 61] Connection refused\n']
2020-11-18 22:10:32.874096Z: [INFO] Disconnecting from Tuya device bf5d0abdb1e62...
.....
2020-11-18 22:10:35.939561Z: [WARN] Trying to cancel timeout timer d7a71a0f-97a5-4f48-9890-84cfd4e01116 which doesn't exist. This should be fixed
2020-11-18 22:10:35.939730Z: [ERRO] Error getting Tuya device bf5d0abdb1e62 status: {'descriptor': 'Waited too long for the device to respond', 'error': 'command_timeout'}

Check `[Errno 64] EHOSTDOWN` issue on connect()

CHECK:

2020-11-10 20:44:44.846828Z: [INFO] Connecting to device bf5d0abdb1e6210180duku at 192.168.10.179:6668...
2020-11-10 20:44:44.847335Z: [WARN] Socket (192.168.10.179:6668) error occurred: <class 'socket.error'> [Errno 64] EHOSTDOWN <traceback object at 0x10541b7e8>
2020-11-10 20:44:44.848645Z: [ERRO] Socket error on device's bf5d0abdb1e6210180duku connection: ['Traceback (most recent call last):\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 83, in read\n    obj.handle_read_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 446, in handle_read_event\n    self.handle_connect_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 455, in handle_connect_event\n    self.handle_connect()\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 104, in handle_connect\n    self.emit(\'connect\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 61, in _on_connect\n    self.emit(\'connect\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 201, in _on_dev_connect\n    self.emit(\'_next\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 127, in _on_can_send_next_command\n    self.connection.send(bin_message)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 52, in send\n    self.socket.send(data)\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 380, in send\n    self.handle_close()\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 108, in handle_close\n    self.emit(\'close\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 67, in _on_close\n    self.emit(\'break\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 207, in _on_dev_connection_break\n    self._reconnect()\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 196, in _reconnect\n    self._connect()\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 182, in _connect\n    self.connection.connect(self.ip, self.port, timeout=self.socket_timeout)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 42, in connect\n    self.socket.start()\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 100, in start\n    self.connect((self._host, self._port))\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 356, in connect\n    raise socket.error(err, errorcode[err])\n', 'error: [Errno 64] EHOSTDOWN\n']

Happening at the same time as other timeouts and instability events.

Fix issue with `_on_dev_reply()` wrong number of arguments

2020-11-30 06:33:04.416214Z: [WARN] Socket (192.168.10.182:6668) error occurred: <type 'exceptions.TypeError'> _on_dev_reply() takes exactly 3 arguments (2 given) <traceback object at 0x10836aab8>
2020-11-30 06:33:04.431310Z: [ERRO] Socket error on device's bf4a36374de48 connection: ['Traceback (most recent call last):\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 83, in read\n    obj.handle_read_event()\n', '  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/asyncore.py", line 449, in handle_read_event\n    self.handle_read()\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 179, in handle_read\n    self.emit(\'read\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 115, in _on_read\n    self.emit(\'data\')\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit\n    remove = [l for l in listeners if not l(*args, **kwargs)]\n', '  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__\n    self.func(*args, **kwargs)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/syncproto.py", line 59, in _on_data\n    cmd.reply(None, reply)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/syncproto.py", line 192, in reply\n    return callback(*args)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 251, in <lambda>\n    }, lambda err, reply: self._get_status_callback(err, reply, callback, ctx))\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 261, in _get_status_callback\n    return callback(None, status)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/device.py", line 101, in <lambda>\n    return self.hw.get_status(lambda err, status: self._get_status_callback(err, status, ctx, collector_callback), ctx)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/device.py", line 112, in _get_status_callback\n    return callback(None, status, ctx)\n', '  File "/Users/david/projects/homeswitch-api/homeswitch/device.py", line 179, in collector_callback\n    other_callback(err, status)\n', 'TypeError: _on_dev_reply() takes exactly 3 arguments (2 given)\n']

Fix double reply() attempt

2020-11-18 22:19:16.196427Z: [ERRO] Error connecting to 127.0.0.1:6668 {'error': 'timeout'}
2020-11-18 22:19:16.196707Z: [WARN] Failure while connecting to Tuya device bf5d0abdb1e62: {'error': 'timeout'}
2020-11-18 22:19:16.197045Z: [WARN] Command's 7f9f2a1d-08af-4435-8f51-cecf771e24d9 was already called. Stopping another reply here
DBUG: [2020-11-18 22:19:16.200020Z] FIRST CALL:
  File "bin/hsapid", line 7, in <module>
    main()
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 182, in main
    api.run()
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 173, in run
    asyncorepp.loop(use_poll=True)
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncorepp.py", line 24, in loop
    _check_timers()
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncorepp.py", line 43, in _check_timers
    call()
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncorepp.py", line 53, in call_and_expire
    return callback()
  File "/Users/david/projects/homeswitch-api/homeswitch/syncproto.py", line 169, in <lambda>
    self.timeout = set_timeout(lambda: self.reply({'error': 'command_timeout', 'descriptor': 'Waited too long for the device to respond'}, None), timeout)
  File "/Users/david/projects/homeswitch-api/homeswitch/syncproto.py", line 188, in reply
    self._responded = current_stack()
  File "/Users/david/projects/homeswitch-api/homeswitch/util.py", line 70, in current_stack
    return ''.join(traceback.format_stack())


SECOND CALL:
  File "bin/hsapid", line 7, in <module>
    main()
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 182, in main
    api.run()
  File "/Users/david/projects/homeswitch-api/homeswitch/api.py", line 173, in run
    asyncorepp.loop(use_poll=True)
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncorepp.py", line 24, in loop
    _check_timers()
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncorepp.py", line 43, in _check_timers
    call()
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncorepp.py", line 53, in call_and_expire
    return callback()
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 57, in <lambda>
    self.timeout = set_timeout(lambda: self._on_failure({'error': 'timeout'}), timeout)
  File "/Users/david/projects/homeswitch-api/homeswitch/asyncsocket/client.py", line 90, in _on_failure
    self.emit('failure', ex)
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 282, in emit
    remove = [l for l in listeners if not l(*args, **kwargs)]
  File "/Users/david/projects/homeswitch-api/env/lib/python2.7/site-packages/pymitter/__init__.py", line 310, in __call__
    self.func(*args, **kwargs)
  File "/Users/david/projects/homeswitch-api/homeswitch/hw/tuya.py", line 177, in _on_dev_connection_failure
    None
  File "/Users/david/projects/homeswitch-api/homeswitch/syncproto.py", line 113, in flush
    cmd.reply(err, reply)
  File "/Users/david/projects/homeswitch-api/homeswitch/syncproto.py", line 184, in reply
    dump("DBUG", "FIRST CALL:\n{}\n\nSECOND CALL:\n{}\n\n".format(self._responded, current_stack()))
  File "/Users/david/projects/homeswitch-api/homeswitch/util.py", line 70, in current_stack
    return ''.join(traceback.format_stack())

Fix infinite loop of "Should be read", Disconnect

Check how we can get to this situation and fix it

2020-11-10 20:45:19.520800Z: [INFO] Connected to 192.168.10.179:6668 !
2020-11-10 20:45:19.520859Z: [INFO] Connected to device bf5d0abdb1e62 !
2020-11-10 20:45:19.520902Z: [DBUG] We can send next command for bf5d0abdb1e62!!!
2020-11-10 20:45:19.520925Z: [DBUG] Command queue is empty...
2020-11-10 20:45:19.520945Z: [INFO] Disconnecting as command queue is empty
2020-11-10 20:45:19.520967Z: [INFO] Disconnecting from device bf5d0abdb1e62...
2020-11-10 20:45:19.521300Z: [INFO] Socket (192.168.10.179:6668) read can happen
2020-11-10 20:45:19.521345Z: [INFO] Socket for device's bf5d0abdb1e62 has data and should be read
2020-11-10 20:45:19.522604Z: [INFO] Connection to 192.168.10.179:6668 was closed.
2020-11-10 20:45:19.523180Z: [INFO] Successfully disconnected from device bf5d0abdb1e62
2020-11-10 20:45:19.523521Z: [INFO] Socket (192.168.10.179:6668) read can happen
2020-11-10 20:45:19.525884Z: [INFO] Socket for device's bf5d0abdb1e62 has data and should be read
2020-11-10 20:45:19.526014Z: [INFO] Connection to 192.168.10.179:6668 was closed.
2020-11-10 20:45:19.526076Z: [INFO] Successfully disconnected from device bf5d0abdb1e62
2020-11-10 20:45:19.526180Z: [INFO] Socket (192.168.10.179:6668) read can happen
2020-11-10 20:45:19.526228Z: [INFO] Socket for device's bf5d0abdb1e62 has data and should be read
2020-11-10 20:45:19.526304Z: [INFO] Connection to 192.168.10.179:6668 was closed.
2020-11-10 20:45:19.526349Z: [INFO] Successfully disconnected from device bf5d0abdb1e62
2020-11-10 20:45:19.526433Z: [INFO] Socket (192.168.10.179:6668) read can happen
2020-11-10 20:45:19.526474Z: [INFO] Socket for device's bf5d0abdb1e62 has data and should be read
2020-11-10 20:45:19.526542Z: [INFO] Connection to 192.168.10.179:6668 was closed.
2020-11-10 20:45:19.526584Z: [INFO] Successfully disconnected from device bf5d0abdb1e62
2020-11-10 20:45:19.526796Z: [INFO] Socket (192.168.10.179:6668) read can happen
2020-11-10 20:45:19.526843Z: [INFO] Socket for device's bf5d0abdb1e62 has data and should be read
2020-11-10 20:45:19.526911Z: [INFO] Connection to 192.168.10.179:6668 was closed.
2020-11-10 20:45:19.526952Z: [INFO] Successfully disconnected from device bf5d0abdb1e62
2020-11-10 20:45:19.527028Z: [INFO] Socket (192.168.10.179:6668) read can happen
2020-11-10 20:45:19.527071Z: [INFO] Socket for device's bf5d0abdb1e62 has data and should be read
2020-11-10 20:45:19.527137Z: [INFO] Connection to 192.168.10.179:6668 was closed.
...