Comments (9)
Hello @joonas-fi - I think if I'm understanding you correctly - the problem is you need to do a lot more to enable messages from sensors. shimmeringbee/zstack only really handles the basic managment of the zigbee stack.
The default configuration of zstack does not register any clusters against zcl endpoints and advertise them out, as such devices that join the network wont sent messages to the coordinator. You'll need to use the RegisterAdapterEndpoint function to register the endpoints, as well as adding any cluster IDs that might be needed (depending on the device, though not often needed).
if err := zigbeeComms.RegisterAdapterEndpoint(z.context, 1, zigbee.ProfileHomeAutomation, 1, 1, []zigbee.ClusterID{}, []zigbee.ClusterID{}); err != nil {
}
You're going to find using the zstack library is indepth, you'll need to understand a lot about zigbee and the ZCL. shimmeringbee/zstack
is very much like the adapter directory in zigbee-steward
- it provides a lot of low level functionality.
I'd like to bring your attention to two higher level components in this stack.
shimmeringbee/zda
which is a generic abstraction on top of zigbee/zstack, but it handles endpoints, subscriptions of clusters, device enumeration, etc. This is closer to zigbee-steward
- but it's very opinionated in its interfaces (kind of needed with Go) - it expects devices to behave according to the ZCL specification - if they don't, then vendor specific code is needed in zda.
There's also shimmeringbee/controller
which is more towards the zigbee2mqtt
end - though at current it's only input/output is http. I am considering adding mqtt to it. I finished adding level and color control to it for another user over Christmas. MQTT might be an easy add.
from zstack.
Thanks for the quick reply and explanation! :)
I was suspecting something like that, that maybe I need to "register" to receive events, and I noticed the RegisterAdapterEndpoint()
, I was actually writing that function call to test, but I glanced through shimmeringbee/controller
and dyrkin/zigbee-steward
to see if they send any messages to the Zigbee nodes to "register to receive events" and I didn't find any.
Granted, shimmeringbee/controller
is composed of many layers of abstractions so I just might've missed that there are eventual function calls somewhere. Or like you mentioned, there's this "ZDA" that I haven't taken a look at. :)
I took a somewhat thorough look at dyrkin/zigbee-steward
and upon a device joining the Zigbee network, I didn't see any code that communicates with the just-joined-node to "register" to receive events - and I still receive the events that I didn't see with your zstack
. It does interrogate the device for its endpoints, but IIUC it's only for metadata use for the computer-side database of nodes? https://github.com/dyrkin/zigbee-steward/blob/a1dc32edfb8560969ea4aedb6f85d05d8279cd7a/steward.go#L93
Is this "registration to receive events" really a Zigbee-level concept or is this a concept internal to shimmeringbee
suite? I'm asking because zigbee-steward
looks like it doesn't send nothing but interrogation messages to the Zigbee network after the device has joined, and I get sensor data from the devices just fine with zigbee-steward
.
I'm actually trying to replace Zigbee2MQTT in my home automation with code built in Go because I don't like JavaScript and I want something that I can debug (Go is very familiar for me). My ideal abstraction would just give me temperature etc. data that the sensors send, but I'd also love to be able to hook into raw Zigbee RX/TX frames for debugging purposes in cases where it's needed. I noticed zstack
already limits my options in some aspects due to it hiding the implementation details like internally initializing ZNP here. Would you be open to maybe having an additional constructor where I can give the ZNP from my code, so users like me can interface with the ZNP if need to - to have more control?
from zstack.
I've not looked at drykin/zigbee-steward
in a while - I started there myself, this whole suite came out of me wanting to update it and I couldn't get hold of the guy.
All zigbee devices have a Node Description contains a list of endpoints, and the clusters they support - other zigbee nodes are allowed to request this for any another node. What you're technically meant to do, if you need to say announce Temperature, is look up on the destination node which endpoint to send it to. So you have to have an open endpoint with the temperature cluster.
Now many zigbee nodes don't do this, and just send messages blind to the coordinator. What I suspect is that Z-Stack (the firmware) is dropping messages to nodes that aren't registered on it.
Registration occurs:
zigbee-herdsmen
https://github.com/Koenkk/zigbee-herdsman/blob/master/src/adapter/z-stack/adapter/startZnp.ts#L35
https://github.com/Koenkk/zigbee-herdsman/blob/master/src/adapter/z-stack/adapter/startZnp.ts#L146
shimmeringbee/zda
https://github.com/shimmeringbee/zda/blob/master/gateway.go#L158
drykin/zigbee-steward
https://github.com/dyrkin/zigbee-steward/blob/master/coordinator/coordinator.go#L493
Re: Zigbee TX/RX - the line you linked is actually the message library for talking z-stack via unpi... you're probably not interested in that... it's not great to work with. I have never once needed to spy on that link once it was working, that'd be for debugging why the CCXXXX didn't do what it was meant to do.
For Zigbee/ZCL debugging, you'd probably be more interested in:
https://github.com/shimmeringbee/zstack/blob/master/node_send_message.go#L12
and
https://github.com/shimmeringbee/zstack/blob/master/node_receive_message.go#L10
So... here me out :D - my suggestion to you would be to look at shimmeringbee/zda
(or the controller) so you can stop worrying about ZCL/Zigbee for the most part... but... when you do need to debug...
What you can do, is create a shim that implements the zigbee.Provider interface (zstack also does), that would allow you to intercept and debug sent/received messages. By adding logic in the shim on these two interfaces, you can capture messages coming from the Zigbee network, and messages being sent by the ZDA.
https://github.com/shimmeringbee/zigbee/blob/master/provider.go
type NodeSender interface {
SendApplicationMessageToNode(ctx context.Context, destinationAddress IEEEAddress, message ApplicationMessage, requireAck bool) error
}
type EventReceiver interface {
ReadEvent(ctx context.Context) (interface{}, error)
}
One more thing... I have wanted to add a capability into ZDA to allow message capture for debug purposesit's just been low on the priority list. zda does have acceess to those raw messages, and I'd like to expose them.
https://github.com/shimmeringbee/da/blob/master/capabilities/message_capture_debug.go
That was the intended interface - some of the comments are wrong (ignore the comments about buffering), but the intent was you'd enable Capturing on a device, you'd receive in/out messages via the zda ReadEvent function, and then you'd be able to disable it. That might make what you're after a little easier.
from zstack.
Oh wow, thank you so much for the helpful information and for taking the time to write it! 👏
I totally missed that one in zigbee-steward
- I was looking for node-specific registration, i.e. I was thinking I need to register for each device that joins the network (after node join) so that's why I was looking in the wrong place, but it seems like one needs to inform the coordinator to register for some information categories, i.e. unrelated to individual nodes.
I'm still wrapping my head around all this.. I think I'm understanding Zigbee topology with coordinator/routers/end devices and understand basic concepts like clusters (IMO stupid word for what is basically a feature/capability/interface, and Zigbee is networking and "cluster" is an already established term in networking..) but am still a bit lost on the concept of an endpoint and these registrations.
I could probably make do with the NodeSender
/EventReceiver
interfaces for debugging, thanks for that tip!
Anyways, so this issue is clearly a user error. I thank you for helping me understand, and I'll need to study zda
if that's a better abstraction for me. As I said, I'm currently trying to replace Zigbee2mqtt and currently shimmeringbee/controller is too high level for me with too many features to understand. I was turned off by zda not yet supporting all the capabilities I need, but that of course is quite understandable as it's work in progress. I however want to move forward now and I was thinking I use a JS script to export cluster definitions from https://github.com/Koenkk/zigbee-herdsman/blob/30aff314345dcc0dbc25c2935462a5e0a894a5b6/src/zcl/definition/cluster.ts to Go structs. This autogeneration approach, if works, would be great for piggybacking off of the massive user base of zigbee2mqtt.
I'm currently using zigbee-steward with success, but with this advice of yours I could re-try switching to either zstack or zda and if I manage to cobble up something that would help your projects, I'll ping you. Let me know if you're open to this idea of autogen-from-zigbee2mqtt.
from zstack.
Completely understand, many of the ZCL cluster definitions are present in shimmeringbee already (https://github.com/shimmeringbee/zcl) - I'm a fan of going back to the source material (the ZCL Specification document), rather than zigbee2mqtt. I wonder if there's something that could be done there (sadly a PDF) - https://github.com/shimmeringbee/zcl/tree/master/commands/local
The ZCL doesn't change that often, ZCL 7 is the latest and hasn't been updated since the launch of Zigbee 3.
Personally, I think the true value of zigbee2mqtt is the implemented knowledge of all the vendor specific work arounds, but much of that is implementation of custom sending of packets or branches in code, often not within the ZCL spec. Not a nice structure to translate sadly for that.
Out of interest which Clusters are you interested in?
from zstack.
Out of interest which Clusters are you interested in?
Not sure about the clusters yet, but I'd like to get all my Zigbee devices to work at least with parity to what Zigbee2mqtt had (battery levels / heartbeats for each device). I have:
- IKEA Trådfri bulbs
- Xiaomi Aqara water leak sensor
- Xiaomi Aqara vibration sensor (dining room chair, post box)
- Xiaomi Aqara temperature/humidity/pressure sensor
- Xiaomi Aqara one-button wall switch
- Xiaomi Aqara two-button wall switch
- Xiaomi Aqara movement/presence sensor
- Xiaomi Aqara door/window sensors
from zstack.
Okay - I have to ask.... Dining Room Chair vibration sensor? 😄
A good selection, and just as well you're using IKEA bulbs for the Xiaomi stuff, their Zigbee low level stuff... isn't 100% compliant and can be difficult to get to work with some routers devices, they thankfully do work well with the Trådfri suite.
I have the movement/presence/ilumination sensors and door/window sensors on my pile to play with - if I had to guess the vibration, movement/presence and door/window sensors are part of the IAS Zone cluster.
tests door sensor
"00158d0004798d07-00": {
"Metadata": {},
"Identifier": "00158d0004798d07-00",
"Capabilities": {
"DeviceRemoval": {},
"EnumerateDevice": {
"Enumerating": false
},
"HasProductInformation": {
"Name": "lumi.sensor_magnet.aq2",
"Manufacturer": "LUMI"
},
"OnOff": {
"State": false
},
"PowerSupply": {
"Battery": [
{
"Voltage": 3.3,
"MaximumVoltage": 3,
"MinimumVoltage": 2,
"Remaining": 1,
"Available": true
}
]
}
},
"Gateway": "cc2531"
}
Nope... they are misusing the On/Off cluster... of course they are. The OnOff attribute changes depending if the magnet is close to them. It wouldn't suprise me if vibration and water are the same - these should really be in the IAS Zone cluster.
Movement/Presence/Illumination do use the correct dedicated clusters (I remember that from my initial playing with zigbee2mqtt).
The buttons/remotes are always a nightmare... there's no real standard between vendors as to how they implement them. Phillips is a vendor specific Cluster, Trådfri just send the OnOff/Level/Color commands to the device based on the button. The Aqara stuff is ironically the closest to being ideal - they use the Input/Output/Value Clusters, I have one of those Cube controllers to play around with and messages come in via those Clusters.
So anyway... good to know... thank you. I appreciate you have your immediate direction you want to go - given this is the shimmeringbee issue repo, I should point out that the only devices that probably wouldn't work out of the box with shimmeringbee\zda
are:
Xiaomi Aqara one-button wall switch
Xiaomi Aqara two-button wall switch
Xiaomi Aqara movement/presence sensor
Battery reporting works, I've just altered the custom rule file for what I believe should be all Xiaomi Aqara devices:
shimmeringbee/zda-rules@928ae54
from zstack.
Okay - I have to ask.... Dining Room Chair vibration sensor?
lol, I could've been more clear, yes that sounds naughty but it isn't :D So I have 6 dining room chairs, but we have one favourite chair so if one of us is sitting in the table there's this one chair that is always used. That chair has a vibration sensor, so when you pull the chair from under the table to sit at the table, the chair vibrates enough to act as a "someone just sat at the dinner table" presence signal. That triggers the light above the table to turn on.
(In this case I couldn't have used movement sensor because the aparment is open concept and thus just walking around would trigger the dining room light.)
A good selection, and just as well you're using IKEA bulbs for the Xiaomi stuff, their Zigbee low level stuff... isn't 100% compliant and can be difficult to get to work with some routers devices, they thankfully do work well with the Trådfri suite.
I've actually seen someone say IKEA Trådfri and Xiaomi Aqara cause problems when running in the same network (can't remember where I saw that), and that's been my experience as well. Either that or I'm hitting some scaling limits (I've got ~ 30 Zigbee nodes). It's really annoying every now and then having to re-pair a device because it just stops communicating in the network.
It doesn't really matter if it's Trådfri+Aqara incompatibility or scaling limits, because both get solved because I just bought two new CC2531 dongles (with actual antennas instead of PCB antenna traces) to have two separate networks, one for IKEA stuff and the other for Aqara stuff. I wanted to revamp my network "for good" by also taking control of the software side (to be able to understand and dig in deep if/when stuff don't work), because I'm not convinced on the debugging size of Zigbee2mqtt, there are "undefined" strings in its logs, I dislike JavaScript-based software and I'm just much more comfortable operating Go-based software.
I'll get back to you when I get time to re-test using shimmeringbee components, you have certainly gone above and beyond to help me (thanks again!), both generally and to convince me that shimmeringbee components can help me. Have a great day!
from zstack.
No worries, happy to have discussions about Zigbee - we also have a Discord if you have that particular service (https://discord.gg/99rPz2h) - warning on the CC2531's - they can only have so many direct connected devices (5 I think), so you probably will need a router. You can flash the CC2531's as pure routers - and they seem to support many more connections.
As your network gets larger, I totally recommend a CC2652 based stick (https://electrolama.com/projects/zig-a-zig-ah/), I have that with two CC2351 routers supporting 26 Aqara and Heiman devices at the moment.
I love your idea for automatically turning on the lights about the table... time to order a couple of vibration sensors.
from zstack.
Related Issues (20)
- Add support for receiving ZCL data from zigbee devices
- Convert Node based commands to accept IEEE Address rather than Network Address
- Node Table can be accessed by two different Go routines
- If End Devices parent is the Coordinator, it may have issues after restart HOT 1
- Could not QueryNodeDescription and QueryNodeEndpoints HOT 7
- Implement persistable network address and IEEE address cache
- Support CC2652 adapter
- Fix Channel Selection
- Initialisation fails on linux/arm7 HOT 1
- Initialize() fails when config is changed (or just generated), but works on second try HOT 3
- Z-Stack appears to crash on CC2652R HOT 5
- Update to use `persistence` layer for storing node table.
- Add support for registering Endpoints.
- Add support for binding and unbinding. HOT 2
- Add support for querying Active Endpoints from remote devices.
- Add support for querying endpoints.
- Haromnise names of ZStack message structs
- Provide Usage Examples for ZStack Layer HOT 2
- Add support for sending ZCL data to zigbee devices
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zstack.