kevinkk525 / pysmartnode Goto Github PK

Hi,
I am trying to run my first pysmartnode device in the wild and I am thinking about few enhancements:

• user serviceable watchdog: a component, that runs a timer (in a scale of hours e.g.) and subscribed to specific MQTT topic; if it not receive a message, it will proceed a soft reboot/ as lowest reintit of pysmartnode as it's possible... Reason: I tried to write my own sensor component BME280 (https://github.com/Nabla128k/pysmartnode/blob/dev/pysmartnode/components/sensors/bme280.py) (beginner's experiment)) and it's not reliable... partially maybe a fault of sensor board itself and outside nearly 100% humidity idk). And it's not easily accessible, I have to take a car, go 20 minutes, go thru garden, unlock cabinet... not forget a laptop in car...). So it would be nice to have a possibility to initiate reboot/reinitialization remotely as a first try.
  And a true independent hardware watchdog I think is an overkill.

• for the same reason - posibility to listen, e.g. for some time, a log trace from pysmartnode device via MQTT

I am not sure, if I am able to develop such enhancements, so I am posting it here just for consideration and/or maybe to get some guidance/opposing...

Hej Kevin! Alot of thanks for your upy smart nodes projects at first!
I think I am making a mistake because my English is not sufficient. Maybe you can help me.
I don´t understand the remoteconfig options,

At first i´ve run the smartserver. Logs looking good.

[2020-02-16 11:02:58,790] [INFO] [MQTT] [_keep_connected] Keeping connected
[2020-02-16 11:02:58,794] [INFO] [Main] [main] Starting main loop
[2020-02-16 11:03:00,089] [INFO] [MQTT] [_connected] Connection returned result: Connection Accepted.

I have create an firmware image with froozen files (also with docker, you can see this at my PR)
Frozen Files are:

• pysmartnode
• _testing
• external_modules to rootdir at the esp32
  • uasyncio
  • stats
  • micropython_mqtt_as

Then i have copy boot.py config.py and main.py to the esp32 rootdir with ampy.
MQTT_RECEIVE_CONFIG = True in config.py
When i start the node the Micropython log will show this:

FTP-Server active
Connected, local ip '192.168.178.42'
Synchronize time from NTP server ...
FTP Server started on 192.168.178.42
Set time to (2020, 2, 16, 13, 16, 42, 6, 47)
[2020-02-16 13:16:42] [RAM] [info] 92432
[2020-02-16 13:16:42] [MQTT] [info] (sysname='esp32', nodename='esp32', release='1.12.0', version='v1.12 on 2020-02-12', machine='ESP32 module with ESP32')
[2020-02-16 13:16:42] [MQTT] [info] Client version: 600
[2020-02-16 13:16:43] [config] [info] Added module WifiLED version 1.4 as component wifi_led
[2020-02-16 13:16:44] [config] [info] Added module remoteConfig version 0.8 as component None
received amount {'_order': []}
[2020-02-16 13:16:45] [MQTT] [error] Error executing mqtt topic 'home/login/3c71bfaaafd8': can't convert dict to int

At SmartServer i can see this:

[2020-02-16 11:03:45,241] [INFO] [Clients] [init] Created device '3c71bfaaafd8'
[2020-02-16 11:03:45,243] [INFO] [3c71bfaaafd8] [getLog] [MQTT] (sysname='esp32', nodename='esp32', release='1.12.0', version='v1.12 on 2020-02-12', machine='ESP32 module with ESP32')
[2020-02-16 11:03:45,525] [INFO] [3c71bfaaafd8] [getLog] [MQTT] Client version: 600
[2020-02-16 11:03:45,703] [INFO] [3c71bfaaafd8] [getLog] [config] Added module WifiLED version 1.4 as component wifi_led
[2020-02-16 11:03:46,466] [INFO] [3c71bfaaafd8] [getLog] [config] Added module remoteConfig version 0.8 as component None
[2020-02-16 11:03:46,826] [INFO] [Main] [sendConfig] Config request from 3c71bfaaafd8 version [600, 'esp32', 0.5] platform None
[2020-02-16 11:03:47,422] [ERROR] [3c71bfaaafd8] [getLog] [MQTT] Error executing mqtt topic 'home/login/3c71bfaaafd8': can't convert dict to int
[2020-02-16 11:06:39,980] [INFO] [3c71bfaaafd8] [getLog] [MQTT] (sysname='esp32', nodename='esp32', release='1.12.0', version='v1.12 on 2020-02-12', machine='ESP32 module with ESP32')
[2020-02-16 11:06:40,261] [INFO] [3c71bfaaafd8] [getLog] [MQTT] Client version: 600
[2020-02-16 11:06:41,017] [INFO] [3c71bfaaafd8] [getLog] [config] Added module WifiLED version 1.4 as component wifi_led
[2020-02-16 11:06:41,479] [INFO] [3c71bfaaafd8] [getLog] [config] Added module remoteConfig version 0.8 as component None
[2020-02-16 11:06:41,978] [INFO] [Main] [sendConfig] Config request from 3c71bfaaafd8 version [600, 'esp32', 0.5] platform None
[2020-02-16 11:06:42,440] [ERROR] [3c71bfaaafd8] [getLog] [MQTT] Error executing mqtt topic 'home/login/3c71bfaaafd8': can't convert dict to int

In MQTT:

home/login/3c71bfaaafd8
{"_order": []
home/log/error/3c71bfaaafd8
[MQTT] Error executing mqtt topic 'home/login/3c71bfaaafd8': can't convert dict to int

I think (but doesn´t know!) i must set up the components dict. But don´t understand this.
Do I have to send a mqtt message to /login//set? I ´ve test a lot Json payloads like this
{ "_order": ["ram"], "ram": { "package": ".machine.ram", "component": "ram", "constructor_args": { "interval": 60 } } }

Or must i setup an config.json or hjson at the smartserver?

I think this one is very relevant
**Added module remoteConfig version 0.8 as component None**

Maybe you can help me...

This occured one time last night...

[2020-04-12 02:03:53] [STATS0] [info] 67920
[2020-04-12 02:04:03] [STATS0] [info] 67920
[2020-04-12 02:04:14] [STATS0] [info] 67920
[2020-04-12 02:04:24] [STATS0] [info] 67920
[2020-04-12 02:04:35] [STATS0] [info] 67936
[2020-04-12 02:04:45] [STATS0] [info] 67920
[2020-04-12 02:04:56] [STATS0] [info] 67920
Guru Meditation Error: Core 1 panic'ed (Unhandled debug exception)
Debug exception reason: BREAK instr
Core 1 register dump:
PC : 0x400803c0 PS : 0x00060836 A0 : 0x00000000 A1 : 0x3ffcfe90
A2 : 0x3ffc1eb4 A3 : 0x3ffba28c A4 : 0x3ffe5990 A5 : 0x3ffd00a0
A6 : 0x00000002 A7 : 0x3ffd0060 A8 : 0x00060023 A9 : 0x00000000
A10 : 0x3ffc294c A11 : 0x3ffc2a00 A12 : 0x80091c3c A13 : 0x3ffbb260
A14 : 0x00000003 A15 : 0x00060023 SAR : 0x00000016 EXCCAUSE: 0x00000001
EXCVADDR: 0xffffffd0 LBEG : 0x4000c46c LEND : 0x4000c477 LCOUNT : 0x00000000

ELF file SHA256: 0000000000000000000000000000000000000000000000000000000000000000

Backtrace: 0x400803bd:0x3ffcfe90

Rebooting...
ets Jun 8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:5240
load:0x40078000,len:12880
ho 0 tail 12 room 4
load:0x40080400,len:3484
entry 0x40080630
I (551) cpu_start: Pro cpu up.
I (551) cpu_start: Application information:
I (551) cpu_start: Compile time: Apr 11 2020 18:24:42
I (554) cpu_start: ELF file SHA256: 0000000000000000...
I (560) cpu_start: ESP-IDF: v4.0
I (565) cpu_start: Starting app cpu, entry point is 0x40082e74
I (556) cpu_start: App cpu up.
I (575) heap_init: Initializing. RAM available for dynamic allocation:
I (582) heap_init: At 3FFAFF10 len 000000F0 (0 KiB): DRAM
I (588) heap_init: At 3FFB6388 len 00001C78 (7 KiB): DRAM
I (594) heap_init: At 3FFB9A20 len 00004108 (16 KiB): DRAM
I (601) heap_init: At 3FFBDB5C len 00000004 (0 KiB): DRAM
I (607) heap_init: At 3FFCCA70 len 00013590 (77 KiB): DRAM
I (613) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (619) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (626) heap_init: At 4009EA08 len 000015F8 (5 KiB): IRAM
I (632) cpu_start: Pro cpu start user code
I (650) spi_flash: detected chip: generic
I (651) spi_flash: flash io: dio
I (651) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
109952
105744
PySmartNode version 610 started
I (1390) modsocket: Initializing
I (1460) wifi: wifi driver task: 3ffd2ef4, prio:23, stack:3584, core=0
I (3580) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (3590) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (3620) wifi: wifi firmware version: 581f422
I (3620) wifi: config NVS flash: enabled
I (3620) wifi: config nano formating: disabled
I (3620) wifi: Init dynamic tx buffer num: 32
I (3620) wifi: Init data frame dynamic rx buffer num: 32
I (3630) wifi: Init management frame dynamic rx buffer num: 32
I (3630) wifi: Init management short buffer num: 32
I (3640) wifi: Init static rx buffer size: 1600
I (3640) wifi: Init static rx buffer num: 10
I (3640) wifi: Init dynamic rx buffer num: 32
I (3760) phy: phy_version: 4180, cb3948e, Sep 12 2019, 16:39:13, 0, 0
I (3760) wifi: mode : sta (4c:11:ae:71:39:94)
I (3760) wifi: STA_START
free ram 90112
[2020-04-12 02:04:20] [DS18] [info] Sensor DS18 will publish readings for temperature to topic breweryaiss/4c11ae713994/DS18
free ram 70848
Starting uasyncio loop
[2020-04-12 02:04:20] [config] [info] Added module STATS version 1.71 as component STATS
I (6050) wifi: new:<4,0>, old:<1,0>, ap:<255,255>, sta:<4,0>, prof:1
I (6900) wifi: state: init -> auth (b0)
I (6900) wifi: state: auth -> assoc (0)
I (6910) wifi: state: assoc -> run (10)
[2020-04-12 02:04:22] [DS18] [info] Found new ds18: 28F58C4592030282
I (7920) wifi: connected with ASUS_68_2G, aid = 1, channel 4, BW20, bssid = b0:6e:bf:de:6f:68
I (7920) wifi: security type: 3, phy: bgn, rssi: -23
I (7920) wifi: pm start, type: 1

I (7930) network: CONNECTED
I (7940) wifi: AP's beacon interval = 102400 us, DTIM period = 3
I (8510) tcpip_adapter: sta ip: 192.168.1.231, mask: 255.255.255.0, gw: 192.168.1.1
I (8510) network: GOT_IP
[2020-04-12 02:04:26] [STATS0] [info] 67792
[2020-04-12 02:04:29] [MQTT] [info] mqtt connected
[2020-04-12 02:04:29] [MQTT] [info] WIFI state True
FTP-Server active
Connected, local ip '192.168.1.231'
Synchronize time from NTP server ...
FTP Server started on 192.168.1.231
Set time to (2020, 4, 12, 2, 5, 19, 6, 103)
[2020-04-12 02:05:19] [MQTT] [info] (sysname='esp32', nodename='esp32', release='1.12.0', version='v1.12-351-gbd5633778-dirty on 2020-04-11', machine='ESP32 module with ESP32')
[2020-04-12 02:05:19] [config] [info] Added module DS18 version 3.4 as component ds18
[2020-04-12 02:05:20] [MQTT] [info] Client version: 610
[2020-04-12 02:05:20] [config] [info] Added module GPIO version 1.11 as component relay
[2020-04-12 02:05:23] [config] [info] Added module Climate version 0.92 as component fermenter
[2020-04-12 02:05:39] [STATS0] [info] 67920
[2020-04-12 02:05:50] [STATS0] [info] 67920
[2020-04-12 02:06:00] [STATS0] [info] 67920
[2020-04-12 02:06:10] [STATS0] [info] 67920

• I use the below method to handle UTC offset. I haven't fully grasped your architecture yet, where should I implement the below?

    def daylight_savings(utc, march, november): # daylight_savings(utc=(2020, 4, 9, 8, 44, 4, 4, 362, 0), march=-7 november=-8)
        year = time.localtime(utc)[0]  # get current year
        now = time.time()
        local_time = None
        HHMarch = time.mktime(
            (year, 3, (14-(int(5*year/4+1)) % 7), 1, 0, 0, 0, 0, 0))  # Time of 2nd Sunday of March change to PDT
        HHNovember = time.mktime(
            (year, 11, (7-(int(5*year/4+1)) % 7), 1, 0, 0, 0, 0, 0))  # Time of 1st Sunday of November change to PST
        if now < HHMarch:  # we are before last sunday of march
            local_time = now + 3600 * november
            pac = time.localtime(local_time)  # PDT:  UTC-8H
        elif now < HHNovember:  # we are before last sunday of october
            local_time = now + 3600 * march
            pac = time.localtime(local_time)  # PST: UTC-7H
        else:  # we are after last sunday of october
            local_time = now + 3600 * november
            pac = time.localtime(local_time)  # PDT:  UTC-8H
        return local_time

• I am in the US so I need to convert C to F

    def celsius_to_fahrenheit(c):
        n = float(9.0 / 5.0 * c + 32)
        # self.dprint("Temp: {}".format(str(n)))
        # normal_round
        if n - math.floor(n) < 0.5:
            return math.floor(n)
        return math.ceil(n)

Can your implementation watchdog.py component be used for esp32?

In module pysmartnode.components.sensors in file dht22.py line 64.
Argument list in call to super() is missing the third argument _unit_index. This causes a runtime error
to be thrown. Simply add the missing parameter which is defined locally to the call and all is well.

Not able to freeze mqtt.py, seems to fail because of "@micropython.native"

error compiling /home/fredrick/Development/micropython/micropython-1.12/lib/pysmartnode-8266/pysmartnode/networking/mqtt.py:
mpy-cross: ../py/asmbase.c:82: mp_asm_base_label_assign: Assertion `as->label_offsets[label] == as->code_offset' failed.

First, I want to say what you have done to implement this mqtt solution is fantastic. Thanks...
Can you explain what the state storage attributes are to be used for?

Thanks,
Rick

I'm having questions and issues.

• Following your instructions, copy(flash) "pysmartnode" directory after updating the submodules.
  Are the libraries and associated readme, license and image files required?

• The networking/mqtt.py is importing class Lock however it is not contained in the mqtt_as.py file or any other library contained in your project after updating submodules.
  from micropython_mqtt_as.mqtt_as import MQTTClient, Lock
  Traceback (most recent call last):
  File "main.py", line 8, in
  File "pysmartnode/main.py", line 12, in
  File "pysmartnode/config.py", line 54, in
  File "pysmartnode/networking/mqtt.py", line 20, in
  ImportError: cannot import name Lock

I'm using the gpio component and it looks like it is subscribing to the topic before mqtt is connected.

component.py

from pysmartnode import config
import gc

gc.collect()
from pysmartnode.components.sensors.ds18 import DS18
gc.collect()

from pysmartnode.components.switches.gpio import GPIO
gc.collect()

ds18 = DS18(pin=27)
config.addComponent("ds18", ds18)
gc.collect()

gpio = GPIO(pin=15)
config.addComponent("relay", gpio)
gc.collect()

MPY: soft reboot
109952
104784
PySmartNode version 610 started
free ram 59232
[2020-04-10 12:16:27] [DS18] [info] Sensor DS18 will publish readings for temperature to topic breweryaiss/4c11ae713994/DS18
free ram 40304
Starting uasyncio loop
[2020-04-10 12:16:27] [config] [info] Added module STATS version 1.71 as component STATS
[2020-04-10 12:16:27] [MQTT] [error] Error subscribing, lost connection: breweryaiss/4c11ae713994/GPIO/15
[2020-04-10 12:16:27] [DS18] [info] Found new ds18: 28F58C4592030282
[2020-04-10 12:16:32] [STATS0] [info] 37216
[2020-04-10 12:16:32] [MQTT] [info] mqtt connected
[2020-04-10 12:16:32] [MQTT] [info] WIFI state True
FTP-Server active
Connected, local ip '192.168.1.231'
Synchronize time from NTP server ...
FTP Server started on 192.168.1.231
Set time to (2020, 4, 10, 12, 16, 33, 4, 101)
[2020-04-10 12:16:33] [config] [info] Added module DS18 version 3.4 as component ds18
[2020-04-10 12:16:33] [MQTT] [info] (sysname='esp32', nodename='esp32', release='1.12.0', version='v1.12-351-gbd5633778-dirty on 2020-04-08', machine='ESP32 module with ESP32')
[2020-04-10 12:16:33] [config] [info] Added module GPIO version 1.11 as component relay
[2020-04-10 12:16:33] [MQTT] [info] Client version: 610
[2020-04-10 12:16:53] [STATS0] [info] 32144

Hi Kevin,
I'm trying to use your pysmartnode but seem like day light saving is not offset for country which locate near equator such as Thailand (UTC+7)

Your below current code under Networking directory are all not covering day light saving condition. Could you kindly reconsider?
tm = tm[0:3] + (0,) + (tm[3] + config.RTC_TIMEZONE_OFFSET,) + tm[4:6] + (0,)

Hello,

I'm having a couple of issues with the unix port (a freshly built one: MicroPython v1.14-113-ga9140ab09 on 2021-03-15; linux version):

[2021-03-15 09:32:26] [main] [error] Traceback (most recent call last):
File "/home/renaud/.micropython/lib/uasyncio/core.py", line 176, in run_until_complete
File "/home/renaud/pysmartnode/pysmartnode/main.py", line 86, in _resetReason
AttributeError: 'module' object has no attribute 'listdir'

[2021-03-15 09:32:26] [main] [error] Traceback (most recent call last):
File "/home/renaud/.micropython/lib/uasyncio/core.py", line 176, in run_until_complete
File "/home/renaud/pysmartnode/pysmartnode/networking/mqtt.py", line 165, in _connected_handler
AttributeError: 'module' object has no attribute 'name'

Both are pretty easy to workaround as I don't need those features but I would like to make sure I am using pysmartnode correcly. Is is expecting a previous version of Micropython or maybe a custom os library?

Hi, I don't know what I am doing wrong, but it seems for me, that interval_publish / interval_reading has no efect in component configuration. I have DS18B20 sensor and ESP8622, pysmartnode downloaded from Github (latest build).
In components.py there's just
COMPONENTS = { "_order": ["ds18"], "ds18": { "package": ".sensors.ds18", "component": "DS18", "constructor_args": { "rom":"102624ED020800F5", "interval_publish": 30, "interval_reading": 30, "auto_detect": False, "precision_temp": 2, "discover": True, "friendly_name": "Krnsko vodomerna sachta", "expose_intervals": True, }, }, }

I tried also without rom: .... and with auto_detect: True.... and cannot make it to accept my interval of read and publish... it also uses values from config.py:

INTERVAL_SEND_SENSOR = const(300) # I don't know if this is from older verison? INTERVAL_SENSOR_PUBLISH = const(300) INTERVAL_SENSOR_READ = const(300)

If I comment out these settings it I think using 10 minutes interval....

What I am doing wrong?

I believe there is a small logic error in the abutton.Pushbutton.buttoncheck coro:

ticks_diff(ticks_ms, ticks_change) returns an erroneous result on the first button press, because t_change has not been changed from None to a ticks value. This causes the long press function to be triggered directly, when the button is pressed for the first time (i.e. before the release and even before the long press timeout has passed).

The solution is, I believe to set t_change before the while loop is entered to a sufficiently high number, e.g.

t_change = time.ticks_ms() + time.ticks_add(0, -1)

See the example for http://docs.micropython.org/en/latest/library/utime.html#utime.ticks_add

# Find out TICKS_MAX used by this port
print(ticks_add(0, -1))