mycontroller-org / 2mqtt Goto Github PK
View Code? Open in Web Editor NEWMQTT Bridge
License: Apache License 2.0
MQTT Bridge
License: Apache License 2.0
Hello @jkandasa,
As mentioned in the forum post https://forum.mycontroller.org/post/2990 I file here the bug about the high CPU usage when using 2mqtt as serial adapter.
I get something like this with htop:
My environment was on a Raspberry Pi with latest Raspberry Pi OS based on Debian Buster and using Mosquitto 1.5.7 from apt-get.
The config file was set like this, where /dev/serial0
is a symlink to the actual PL011 UART /dev/ttyAMA0
connected to the MySensors serial gateway node:
logger:
mode: development
encoding: console
level: info
adapters:
- name: adapter1
enabled: true
reconnect_delay: 20s
provider: mysensors_v2
source:
type: serial
port: /dev/serial0
baud_rate: 115200
transmit_pre_delay: 10ms
mqtt:
broker: tcp://127.0.0.1:1883
insecure_skip_verify: false
username:
password:
subscribe: mysensors-in/#
publish: mysensors-out
qos: 0
transmit_pre_delay: 0s
reconnect_delay: 5s
- name: adapter2
enabled: false
reconnect_delay: 20s
provider: mysensors_v2
source:
type: ethernet
server: tcp://192.168.10.21:5003
transmit_pre_delay: 10ms
mqtt:
broker: tcp://192.168.10.21:1883
insecure_skip_verify: false
username:
password:
subscribe: in_nrf24/#
publish: out_nrf24
qos: 0
transmit_pre_delay: 0s
reconnect_delay: 5s
- name: adapter3
enabled: false
reconnect_delay: 20s
provider: raw
source:
type: serial
port: /dev/ttyUSB0
baud_rate: 115200
transmit_pre_delay: 10ms
message_splitter: 10
mqtt:
broker: tcp://192.168.10.21:1883
insecure_skip_verify: false
username:
password:
subscribe: in/my_topic
publish: out/my_topic
qos: 0
transmit_pre_delay: 0s
reconnect_delay: 5s
Thanks in advance.
It seems messages are put into a queue and held for up to 1 second. I see this in the debug logs received
received
received
before I see about to send a message
about to send a message
about to send a message
no matter what the rate from the serial device to 2mqtt it is holding those messages and shipping them off every 1 second.
2023-10-27T13:36:58.894+0530 debug adapter/service.go:172 received a message from source device {"message": "{data:{\"message\":\"metrics\",\"uptime\":46028069,\"altitude\":1072.49,\"pressure\":97675.58,\"temp\":71.66,\"heading\":83.83,\"accelX\":26.47,\"accelY\":-63.31,\"accelZ\":993.08,\"gyroX\":26.47,\"gyroY\":-63.31,\"gyroZ\":993.08}\r, others:map[], timestamp:2023-10-27 13:36:58.894284234 +0530 IST m=+2.317788558", "adapterName": "adapter1", "provider": "raw"}
2023-10-27T13:36:59.225+0530 debug adapter/service.go:172 received a message from source device {"message": "{data:{\"message\":\"metrics\",\"uptime\":46028402,\"altitude\":1072.62,\"pressure\":97675.14,\"temp\":71.66,\"heading\":83.98,\"accelX\":26.35,\"accelY\":-63.56,\"accelZ\":993.32,\"gyroX\":26.35,\"gyroY\":-63.56,\"gyroZ\":993.32}\r, others:map[], timestamp:2023-10-27 13:36:59.224778851 +0530 IST m=+2.648283101", "adapterName": "adapter1", "provider": "raw"}
2023-10-27T13:36:59.556+0530 debug adapter/service.go:172 received a message from source device {"message": "{data:{\"message\":\"metrics\",\"uptime\":46028735,\"altitude\":1072.85,\"pressure\":97675.38,\"temp\":71.66,\"heading\":83.46,\"accelX\":26.35,\"accelY\":-63.68,\"accelZ\":993.32,\"gyroX\":26.35,\"gyroY\":-63.68,\"gyroZ\":993.32}\r, others:map[], timestamp:2023-10-27 13:36:59.555906296 +0530 IST m=+2.979410564", "adapterName": "adapter1", "provider": "raw"}
2023-10-27T13:36:59.682+0530 debug mqtt/device.go:134 about to send a message {"adapterName": "adapter1", "message": "{data:{\"message\":\"metrics\",\"uptime\":46028069,\"altitude\":1072.49,\"pressure\":97675.58,\"temp\":71.66,\"heading\":83.83,\"accelX\":26.47,\"accelY\":-63.31,\"accelZ\":993.08,\"gyroX\":26.47,\"gyroY\":-63.31,\"gyroZ\":993.08}\r, others:map[mqtt_qos:0 mqtt_topic:], timestamp:2023-10-27 13:36:58.894284234 +0530 IST m=+2.317788558"}
2023-10-27T13:36:59.682+0530 debug mqtt/device.go:134 about to send a message {"adapterName": "adapter1", "message": "{data:{\"message\":\"metrics\",\"uptime\":46028402,\"altitude\":1072.62,\"pressure\":97675.14,\"temp\":71.66,\"heading\":83.98,\"accelX\":26.35,\"accelY\":-63.56,\"accelZ\":993.32,\"gyroX\":26.35,\"gyroY\":-63.56,\"gyroZ\":993.32}\r, others:map[mqtt_qos:0 mqtt_topic:], timestamp:2023-10-27 13:36:59.224778851 +0530 IST m=+2.648283101"}
2023-10-27T13:36:59.683+0530 debug mqtt/device.go:134 about to send a message {"adapterName": "adapter1", "message": "{data:{\"message\":\"metrics\",\"uptime\":46028735,\"altitude\":1072.85,\"pressure\":97675.38,\"temp\":71.66,\"heading\":83.46,\"accelX\":26.35,\"accelY\":-63.68,\"accelZ\":993.32,\"gyroX\":26.35,\"gyroY\":-63.68,\"gyroZ\":993.32}\r, others:map[mqtt_qos:0 mqtt_topic:], timestamp:2023-10-27 13:36:59.555906296 +0530 IST m=+2.979410564"}
I have tried setting transmit_pre_delay: 10ms
and also 0s
on the MQTT side, so I don't think this is where the delay is being introduced.
Is any way to disable this queueing and ship then immediately?
Thanks!
I can't find the message format for using the bridge correctly.
I was successfully using another project but it's bridge can't receive MQTT messages from Serial (reverse way), only publish to a broker.
Here is a quick log from your application:
vd@vd1:~/downloads/2mqtt-1.3-linux-arm64$ ./2mqtt
2023-02-17T11:53:01.168Z info start/logger.go:14 welcome to the 2mqtt adapter server :)
2023-02-17T11:53:01.168Z info start/logger.go:15 server detail {"version": {"version":"1.3","gitCommit":"0667010e106bc840db78c3b8d157498d8e116856","buildDate":"2022-06-08T10:20:20+00:00","goLang":"go1.18.2","platform":"linux","arch":"arm64"}, "loggerConfig": {"Mode":"development","Encoding":"console","Level":"debug"}}
2023-02-17T11:53:01.169Z debug mqtt/device.go:60 mqtt config {"adapterName": "adapter1", "config": {"Name":"","Broker":"tcp://127.0.0.1:1883","InsecureSkipVerify":true,"Username":"","Subscribe":"vallydroid/#","Publish":"vallydroid/","QoS":0,"TransmitPreDelay":"0s","ReconnectDelay":"5s"}}
2023-02-17T11:53:01.170Z debug mqtt/device.go:94 mqtt client connected successfully {"adapterName": "adapter1", "timeTaken": "1.579829ms", "clientConfig": {"Name":"","Broker":"tcp://127.0.0.1:1883","InsecureSkipVerify":true,"Username":"","Subscribe":"vallydroid/#","Publish":"vallydroid/","QoS":0,"TransmitPreDelay":"0s","ReconnectDelay":"5s"}}
2023-02-17T11:53:01.170Z info adapter/service.go:238 connected to the mqtt broker {"adapterName": "adapter1"}
2023-02-17T11:53:01.170Z debug mqtt/device.go:103 mqtt connection success {"adapterName": "adapter1"}
2023-02-17T11:53:01.171Z debug serial/device.go:58 generated config {"config": {"Port":"/dev/ttyUSB0","BaudRate":115200,"MessageSplitter":10,"TransmitPreDelay":"10ms"}}
2023-02-17T11:53:01.171Z info serial/device.go:62 opening a serial port {"adapterName": "adapter1", "port": "/dev/ttyUSB0"}
2023-02-17T11:53:01.171Z debug mqtt/device.go:186 subscribed a topic {"adapterName": "adapter1", "topic": "vallydroid/#"}
2023-02-17T11:53:01.177Z info adapter/service.go:270 connected to the source device {"adapterName": "adapter1"}
2023-02-17T11:53:01.177Z info start/start.go:26 services started {"timeTaken": "10.175326ms"}
2023-02-17T11:53:08.511Z debug adapter/service.go:172 received a message from source device {"message": "{data:[1,\"vallydroid/sensors/24AE0525BF58/send\",\"JK4FJb9YAQ0BFwEH1SoCnwDKAAAQsBA=\"]\r, others:map[], timestamp:2023-02-17 11:53:08.511578315 +0000 GMT m=+7.347275376"}
2023-02-17T11:53:09.178Z debug mqtt/device.go:134 about to send a message {"adapterName": "adapter1", "message": "{data:[1,\"vallydroid/sensors/24AE0525BF58/send\",\"JK4FJb9YAQ0BFwEH1SoCnwDKAAAQsBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:], timestamp:2023-02-17 11:53:08.511578315 +0000 GMT m=+7.347275376"}
2023-02-17T11:53:09.181Z debug adapter/service.go:162 received a mqtt message {"message": "{data:[1,\"vallydroid/sensors/24AE0525BF58/send\",\"JK4FJb9YAQ0BFwEH1SoCnwDKAAAQsBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:09.181021406 +0000 GMT m=+8.016718560"}
2023-02-17T11:53:09.194Z debug adapter/service.go:162 received a mqtt message {"message": "{data:[1,\"vallydroid/sensors/24AE0525BF58/send\",\"JK4FJb9YAQ0BFwEH1SoCnwDKAAAQsBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:09.19393239 +0000 GMT m=+8.029629562"}
2023-02-17T11:53:09.992Z debug adapter/service.go:172 received a message from source device {"message": "{data:[1,\"vallydroid/sensors/E04066F23A08/send\",\"4EBm8joIAQ0BGgEM1SkCcwC/AAAQrhA=\"]\r, others:map[], timestamp:2023-02-17 11:53:09.992556235 +0000 GMT m=+8.828253407"}
2023-02-17T11:53:10.178Z debug adapter/service.go:148 posting a message to source device {"message": "{data:[1,\"vallydroid/sensors/24AE0525BF58/send\",\"JK4FJb9YAQ0BFwEH1SoCnwDKAAAQsBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:09.181021406 +0000 GMT m=+8.016718560"}
2023-02-17T11:53:10.178Z debug mqtt/device.go:134 about to send a message {"adapterName": "adapter1", "message": "{data:[1,\"vallydroid/sensors/E04066F23A08/send\",\"4EBm8joIAQ0BGgEM1SkCcwC/AAAQrhA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:], timestamp:2023-02-17 11:53:09.992556235 +0000 GMT m=+8.828253407"}
2023-02-17T11:53:10.179Z debug adapter/service.go:162 received a mqtt message {"message": "{data:[1,\"vallydroid/sensors/E04066F23A08/send\",\"4EBm8joIAQ0BGgEM1SkCcwC/AAAQrhA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:10.179059469 +0000 GMT m=+9.014756512"}
2023-02-17T11:53:10.186Z debug adapter/service.go:162 received a mqtt message {"message": "{data:[1,\"vallydroid/sensors/E04066F23A08/send\",\"4EBm8joIAQ0BGgEM1SkCcwC/AAAQrhA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:10.186665782 +0000 GMT m=+9.022362825"}
2023-02-17T11:53:10.188Z debug adapter/service.go:148 posting a message to source device {"message": "{data:[1,\"vallydroid/sensors/24AE0525BF58/send\",\"JK4FJb9YAQ0BFwEH1SoCnwDKAAAQsBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:09.19393239 +0000 GMT m=+8.029629562"}
2023-02-17T11:53:10.199Z debug adapter/service.go:148 posting a message to source device {"message": "{data:[1,\"vallydroid/sensors/E04066F23A08/send\",\"4EBm8joIAQ0BGgEM1SkCcwC/AAAQrhA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:10.179059469 +0000 GMT m=+9.014756512"}
2023-02-17T11:53:10.209Z debug adapter/service.go:148 posting a message to source device {"message": "{data:[1,\"vallydroid/sensors/E04066F23A08/send\",\"4EBm8joIAQ0BGgEM1SkCcwC/AAAQrhA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:10.186665782 +0000 GMT m=+9.022362825"}
2023-02-17T11:53:30.259Z debug adapter/service.go:172 received a message from source device {"message": "{data:[1,\"vallydroid/sensors/184166F23A08/send\",\"GEFm8joIAQ8BHQEL1SoCiwDIAAAQuBA=\"]\r, others:map[], timestamp:2023-02-17 11:53:30.25971624 +0000 GMT m=+29.095413264"}
2023-02-17T11:53:31.178Z debug mqtt/device.go:134 about to send a message {"adapterName": "adapter1", "message": "{data:[1,\"vallydroid/sensors/184166F23A08/send\",\"GEFm8joIAQ8BHQEL1SoCiwDIAAAQuBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:], timestamp:2023-02-17 11:53:30.25971624 +0000 GMT m=+29.095413264"}
2023-02-17T11:53:31.178Z debug adapter/service.go:162 received a mqtt message {"message": "{data:[1,\"vallydroid/sensors/184166F23A08/send\",\"GEFm8joIAQ8BHQEL1SoCiwDIAAAQuBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:31.178879574 +0000 GMT m=+30.014576617"}
2023-02-17T11:53:31.185Z debug adapter/service.go:162 received a mqtt message {"message": "{data:[1,\"vallydroid/sensors/184166F23A08/send\",\"GEFm8joIAQ8BHQEL1SoCiwDIAAAQuBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:31.185833108 +0000 GMT m=+30.021530151"}
2023-02-17T11:53:32.177Z debug adapter/service.go:148 posting a message to source device {"message": "{data:[1,\"vallydroid/sensors/184166F23A08/send\",\"GEFm8joIAQ8BHQEL1SoCiwDIAAAQuBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:31.178879574 +0000 GMT m=+30.014576617"}
2023-02-17T11:53:32.188Z debug adapter/service.go:148 posting a message to source device {"message": "{data:[1,\"vallydroid/sensors/184166F23A08/send\",\"GEFm8joIAQ8BHQEL1SoCiwDIAAAQuBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:31.185833108 +0000 GMT m=+30.021530151"}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.