Might just be me... Null Client about rpi-reporter-mqtt2ha-daemon

[ironsheep]

@suburbanidiot Based on your description (well described, too) it appears to be something I've seen before.
Like all more intricate systems, I have seen this just frequently enough to know that I've solved it. Easy fix if it's the same problem. But also just NOT often enough for me to remember the issue or fix thereof offhand. Sigh...

So, let's look at each of your config.ini files for each of the two. I'll contrast them against my own to see if there is anything obvious. Generally, this happens when the two systems somehow appear to be the same to the broker/HA. Let's see if we can see why.

(If this is the same it's happening often enough now that it will warrant an adjustment to my instructions.)

[suburbanidiot]

@ironsheep, thanks for the quick reply!

Here on the uncommented out configs for both Pi A & B:

Pi A:
hostname = {MQTT Broker IP}
port = 1883
base_topic = home/nodes
sensor_name = rip-{PiA}
username = {HA User}
password = {HA User PW}

Pi B:
hostname = {MQTT Broker IP}
port = 1883
base_topic = home/nodes
sensor_name = rip-{PiB}
username = {HA User}
password = {HA User PW}

Is there a way to define the Client Name Hostname?

[suburbanidiot]

Other things I have tried this morning:

• Adding FQDN localdomain (at VLAN DCHP & on etc/hosts)
• Changing the base_topic = local/node on one Pi & leaving home/topic on the other
• Coffee (sadly didn't work)

[ironsheep]

@suburbanidiot oh, sorry to hear about the Coffee not working! ;-) Sadly, it doesn't always help.

Ok, I'll see what mine say and report back. Meanwhile, Are you familiar with MQTTBox or MQTT Explorer?
They are both free apps and either one of them can help us figure this out.

After you get the app installed and connected to your MQTT broker then you can explore the traffic being sent from your two RPis. By comparing the json data being sent you should be able to see why they look identical to HA. (Oh and when inspecting the json can look compact and hard to review. I use this site JSON PrettyPrint to help me make sense of it all.

Let me know what you find out while inspecting the traffic from the two RPis.

You are going to want to compare both the advertisement from each RPi and the actual report from each.

Once we see why they are identical then we'll make changes in your config.ini files to keep them from looking identical.

[suburbanidiot]

@ironsheep - thinking the same & the coffee finally kicked in... Been playing with MQTT Explorer today.

I see 1 node tree, with 2 sensors (is this what I should expect?)

Pi 1:

{
"info": {
"timestamp": "2022-01-26T19:28:56+00:00",
"timestamp": "2022-01-26T19:33:56+00:00",
"rpi_model": "RPi 4 Model B r1.2",
"ifaces": "e,w,b",
"host_name": "rhubarb",
"fqdn": "rhubarb.localdomain",
"ux_release": "focal\nfocal-backports\nfocal-security\nfocal-updates",
"ux_version": "5.4.0-1050-raspi",
"up_time": " 6:04",
"up_time": " 6:09",
"last_update": "2022-01-26T08:02:28+00:00",
"fs_total_gb": 128,
"fs_free_prcnt": 3,
"networking": {
"": {}
},
"drives": {
"dev": {
"size_gb": 2,
"used_prcnt": 0,
"device": "udev",
"mount_pt": "/dev"
},
"root": {
"size_gb": 128,
"used_prcnt": 3,
"device": "/dev/mmcblk0p2",
"mount_pt": "/"
},
"snap-core18-2127": {
"size_gb": 0,
"used_prcnt": 100,
"device": "/dev/loop0",
"mount_pt": "/snap/core18/2127"
},
"snap-core18-2289": {

    "size_gb": 0,
    "used_prcnt": 100,
    "device": "/dev/loop1",
    "mount_pt": "/snap/core18/2289"
  },
  "snap-core20-1274": {
    "size_gb": 0,
    "used_prcnt": 100,
    "device": "/dev/loop2",
    "mount_pt": "/snap/core20/1274"
  },
  "snap-snapd-12707": {
    "size_gb": 0,
    "used_prcnt": 100,
    "device": "/dev/loop4",
    "mount_pt": "/snap/snapd/12707"
  },
  "snap-lxd-21843": {
    "size_gb": 0,
    "used_prcnt": 100,
    "device": "/dev/loop3",
    "mount_pt": "/snap/lxd/21843"
  },
  "snap-lxd-21032": {
    "size_gb": 0,
    "used_prcnt": 100,
    "device": "/dev/loop5",
    "mount_pt": "/snap/lxd/21032"
  },
  "snap-snapd-14553": {
    "size_gb": 0,
    "used_prcnt": 100,
    "device": "/dev/loop6",
    "mount_pt": "/snap/snapd/14553"
  }
},
"memory": {
  "size_mb": "3793.293",
  "free_mb": "3522.500"
  "free_mb": "3521.168"
},
"cpu": {
  "hardware": "BCM2835",
  "model": "",
  "number_cores": 4,
  "bogo_mips": "432.00",
  "serial": "100000002b74a79f",
  "load_1min_prcnt": 0,
  "load_5min_prcnt": 0,
  "load_15min_prcnt": 0
},
"throttle": [
  "Under-voltage has occurred",
  "Throttling has occurred"
],
"temperature_c": 46.2,
"temp_gpu_c": 46.2,
"temp_cpu_c": 46.7,
"temp_cpu_c": 45.8,
"reporter": "ISP-RPi-mqtt-daemon v1.6.0",
"report_interval": 5

}
}

Pi 2:

{
"info": {
"timestamp": "2022-01-26T19:28:45+00:00",
"timestamp": "2022-01-26T19:33:45+00:00",
"rpi_model": "RPi 4 Model B r1.4",
"ifaces": "e,w,b",
"host_name": "shepherd",
"fqdn": "shepherd.localdomain",
"ux_release": "focal\nfocal-backports\nfocal-security\nfocal-updates\nstable",
"ux_version": "5.4.0-1050-raspi",
"up_time": " 5:37",
"up_time": " 5:42",
"last_update": "2022-01-26T13:43:17+00:00",
"fs_total_gb": 64,
"fs_free_prcnt": 9,
"networking": {
"": {}
},
"drives": {
"dev": {
"size_gb": 1,
"used_prcnt": 0,
"device": "udev",
"mount_pt": "/dev"
},
"root": {
"size_gb": 64,
"used_prcnt": 9,
"device": "/dev/mmcblk0p2",
"mount_pt": "/"
},
"snap-core18-2127": {
"size_gb": 0,
"used_prcnt": 100,
"device": "/dev/loop0",
"mount_pt": "/snap/core18/2127"
},
"snap-lxd-21032": {
"size_gb": 0,
"used_prcnt": 100,
"device": "/dev/loop1",
"mount_pt": "/snap/lxd/21032"
},
"snap-lxd-21843": {
"size_gb": 0,
"used_prcnt": 100,
"device": "/dev/loop3",
"mount_pt": "/snap/lxd/21843"
},
"snap-core20-1274": {
"size_gb": 0,
"used_prcnt": 100,
"device": "/dev/loop2",
"mount_pt": "/snap/core20/1274"
},
"snap-snapd-14296": {
"size_gb": 0,
"used_prcnt": 100,
"device": "/dev/loop4",
"mount_pt": "/snap/snapd/14296"
},
"snap-core18-2289": {
"size_gb": 0,
"used_prcnt": 100,
"device": "/dev/loop5",
"mount_pt": "/snap/core18/2289"
},
"snap-snapd-14553": {
"size_gb": 0,
"used_prcnt": 100,
"device": "/dev/loop6",
"mount_pt": "/snap/snapd/14553"
}
},
"memory": {
"size_mb": "1848.176",
"free_mb": "1457.344"
"free_mb": "1456.742"
},
"cpu": {
"hardware": "BCM2835",
"model": "",
"number_cores": 4,
"bogo_mips": "432.00",
"serial": "100000008de523f0",
"load_1min_prcnt": 0.2,
"load_5min_prcnt": 0.5,
"load_1min_prcnt": 0,
"load_5min_prcnt": 0,
"load_15min_prcnt": 0
},
"throttle": [
"Under-voltage has occurred",
"Throttling has occurred"
],
"temperature_c": 48.7,
"temp_gpu_c": 48.7,
"temp_cpu_c": 47.7,
"temp_cpu_c": 48.2,
"reporter": "ISP-RPi-mqtt-daemon v1.6.0",
"report_interval": 5
}
}

[suburbanidiot]

@ironsheep - Further down the rabbit hole... After a lot of deleting in MQTTExplorer it looks good to me.

But alas, only 1 device listed in HA:

With mismatched client to sensor in HA

I'm not sure it is a daemon issue anymore... Ugh. Thoughts?

[ironsheep]

Ok, looking better! Let's also look at the /homeassistant/sensor/rpi-rhubarb/monitor and /homeassistant/sensor/rpi-shepherd/monitor json values as it is these that tell HA what to make available...

[ironsheep]

@suburbanidiot ^^^

[suburbanidiot]

@ironsheep - Here is the monitor json for each. The problem is HA still only sees it as one device. Since my last message, I deleted the MQTT Integration, uninstalled the Mosquitto Broker, rebooted the entire system. Installed & started the broker (did not configure the integration) to hopefully see the MQTT traffic clean:

{"name": "Rpi Monitor Shepherd", "uniq_id": "RPi-Mon_monitor", "dev_cla": "timestamp", "stat_t": "/monitor", "val_tpl": "{{ value_json.info.timestamp }}", "": "home/nodes/sensor/rpi-shepherd", "pl_avail": "online", "pl_not_avail": "offline", "ic": "mdi:raspberry-pi", "avty_t": "/status", "json_attr_t": "/monitor", "json_attr_tpl": "{{ value_json.info | tojson }}", "dev": {"identifiers": ["RPi-Mon"], "manufacturer": "Raspberry Pi (Trading) Ltd.", "name": "RPi-shepherd.localdomain", "model": "RPi 4 Model B r1.4", "sw_version": "focal\nfocal-backports\nfocal-security\nfocal-updates\nstable 5.4.0-1050-raspi"}}

{"name": "Rpi Monitor Rhubarb", "uniq_id": "RPi-Mon_monitor", "dev_cla": "timestamp", "stat_t": "/monitor", "val_tpl": "{{ value_json.info.timestamp }}", "": "home/nodes/sensor/rpi-rhubarb", "pl_avail": "online", "pl_not_avail": "offline", "ic": "mdi:raspberry-pi", "avty_t": "/status", "json_attr_t": "/monitor", "json_attr_tpl": "{{ value_json.info | tojson }}", "dev": {"identifiers": ["RPi-Mon"], "manufacturer": "Raspberry Pi (Trading) Ltd.", "name": "RPi-rhubarb.localdomain", "model": "RPi 4 Model B r1.2", "sw_version": "focal\nfocal-backports\nfocal-security\nfocal-updates 5.4.0-1050-raspi"}}

[suburbanidiot]

@ironsheep - additional info from the broker...

[ironsheep]

@suburbanidiot You pointed it out!

"uniq_id": "RPi-Mon_monitor" this must be unique to each RPi (it's the same on both of yours) Hence they both ID to HA as the same device.

And the problem is even more insidious in your case. The unique ID is created from your RPi mac values for active ethernet/wifi connections. Your RPi's are not reported as having any connections... New problem. Why aren't they?

"networking": {
    "": {}
},

[suburbanidiot]

@ironsheep - not sure what to do next. What do you need me to do, show, or change?

[suburbanidiot]

@ironsheep - I guess I could build a new MQTT broker or a new HA instance... Or are you saying the daemon is not reporting correctly in my use case? Thanks for all the support!

[suburbanidiot]

@ironsheep - welp it's not HA (well it could be the new v7.2 HAOS). I had a spare sd card laying around, so I tested a clean build of HAOS in record time. Same exact problem with the "uniq_id": "RPi-Mon_monitor".

[ironsheep]

@suburbanidiot ok, I wasn't clear, I'm sure.

The problem is that "uniq_id": "xxx" MUST be unique for each RPi on your network! At the moment both of your RPis have the exact same value so HA things that they are both the one RPi. Make Sense?

So the first problem is that the value for "uniq_id" is turning out to be "RPi-Mon_monitor" twice on your network.

On a working RPi it would look like this: "RPi-b827ebMonc21081_monitor"

Where the b827eb and c21081 parts are part of the MAC address for the RPi.

The script on your RPi's doesn't appear to be locating the MAC address correctly. This is also part of the problem in your case.

(1) Please run an ifconfig on your RPi's and capture the output so we can look at why the script is failing to get the MAC address.

(2) Another thing you could do to help is to run the daemon script by hand and capture the initial output so we can look in detail at what it is seeing.

run like `python3 ISP-RPi-mqtt-daemon.py -d -v' which will cause debug and verbose output to be generated.

Here's a run from one of my older RPis:

[2022-01-26 22:21:42] - ISP-RPi-mqtt-daemon.py v1.6.0
[2022-01-26 22:21:42] - Verbose enabled
[2022-01-26 22:21:42] - (DBG): Debug enabled
[2022-01-26 22:21:42] - (DBG): * init mqtt_client_connected=[False]
[2022-01-26 22:21:42] - (DBG): fqdn_raw=[pinode1]
[2022-01-26 22:21:42] - (DBG): rpi_fqdn=[pinode1.home]
[2022-01-26 22:21:42] - (DBG): rpi_hostname=[pinode1]
[2022-01-26 22:21:42] - (DBG): rpi_model_raw=[Raspberry Pi 2 Model B Rev 1.1]
[2022-01-26 22:21:42] - (DBG): rpi_model=[RPi 2 Model B r1.1]
[2022-01-26 22:21:42] - (DBG): rpi_connections=[e]
[2022-01-26 22:21:43] - (DBG): cpu_loads_raw=[['0.14', '0.04', '0.01', '1/253', '23034']]
[2022-01-26 22:21:43] - (DBG): rpi_cpu_tuple=[('BCM2835', 'ARMv7 Processor rev 5 (v7l)', 4, 230.4, '00000000bab09c3b', 3.5, 1.0, 0.2)]
[2022-01-26 22:21:43] - (DBG): rpi_linux_release=[bullseye]
[2022-01-26 22:21:43] - (DBG): rpi_linux_version=[5.10.92-v7+]
[2022-01-26 22:21:43] - (DBG): getFileSystemDrives() trimmedLines=[['/dev/root          29646  3607     24784  13% /']]
[2022-01-26 22:21:43] - (DBG): lineParts(6)=[['/dev/root', '29646', '3607', '24784', '13%', '/']]
[2022-01-26 22:21:43] - (DBG): percent_field_index=[4]
[2022-01-26 22:21:43] - (DBG): device=[/dev/root]
[2022-01-26 22:21:43] - (DBG): mount_point=[/]
[2022-01-26 22:21:43] - (DBG): newTuple=[('32', '13', '/', '/dev/root')]
[2022-01-26 22:21:43] - (DBG): rpi_filesystem_space=[32GB]
[2022-01-26 22:21:43] - (DBG): rpi_filesystem_percent=[13]
[2022-01-26 22:21:43] - (DBG): rpi_filesystem=[[('32', '13', '/', '/dev/root')]]
[2022-01-26 22:21:43] - Connecting to MQTT broker ...
[2022-01-26 22:21:43] - (DBG): * Wait on mqtt_client_connected=[False]
[2022-01-26 22:21:43] * MQTT connection established
[2022-01-26 22:21:43] 
[2022-01-26 22:21:43] - (DBG): on_connect() mqtt_client_connected=[True]
[2022-01-26 22:21:44] - (DBG): - stopped MQTT timer
[2022-01-26 22:21:44] - (DBG): - started MQTT timer - every 60 seconds
[2022-01-26 22:21:44] - (DBG): Found IP(8)=[/bin/ip]
[2022-01-26 22:21:44] - (DBG): interfaceNames=[['eth0']]
[2022-01-26 22:21:44] - (DBG): IF=[eth0], lineParts=[['inet', '192.168.100.42', 'netmask', '255.255.255.0', 'broadcast', '192.168.100.255']]
[2022-01-26 22:21:44] - (DBG): newTuple=[('eth0', 'IP', '192.168.100.42')]
[2022-01-26 22:21:44] - (DBG): IF=[eth0], lineParts=[['ether', 'b8:27:eb:b0:9c:3b', 'txqueuelen', '1000', '(Ethernet)']]
[2022-01-26 22:21:44] - (DBG): rpi_mac=[b8:27:eb:b0:9c:3b]
[2022-01-26 22:21:44] - (DBG): newTuple=[('eth0', 'mac', 'b8:27:eb:b0:9c:3b')]
[2022-01-26 22:21:44] - (DBG): rpi_interfaces=[[('eth0', 'IP', '192.168.100.42'), ('eth0', 'mac', 'b8:27:eb:b0:9c:3b')]]
[2022-01-26 22:21:44] - (DBG): rpi_mac=[b8:27:eb:b0:9c:3b]
[2022-01-26 22:21:44] - (DBG): mac lt=[b827eb], rt=[b09c3b], mac=[b827ebb09c3b]
[2022-01-26 22:21:44] Announcing RPi Monitoring device to MQTT broker for auto-discovery ...

You can see that it is seeing my eth0 interface on this RPi.

Hopefully, your output will show it having some sort of problem trying to get network info?

But, in the end, we need to get the 'uniq_id' values to really be unique on each of your RPis!

[suburbanidiot]

@ironsheep - problem solved! I was missing net-tools install on both RPis. Once installed, the json reports correctly with a unique id. I did one RPi before the other to validate.

Not a bug. Thanks again!

[ironsheep]

@suburbanidiot Congratulations! I'm so glad this is working for you and glad that I could help in some way.

I'll definitely add a section to our README that addresses RPi's showing up with non-unique ID's and how to trouble shoot.

Is there anything else I missed in the install info that would have helped prevent this?

Also, what OS are you running on the RPi?

[suburbanidiot]

Thanks again @ironsheep!

This ordeal was associated to a clean disk image of 20.04 Ubuntu. I’ve run it on Debian and RPiOS as well but no logs to know if they had the “issue”. My guess is why I didn’t notice is because only 1 RPi was running this build of Ubuntu, so since it was still unique (sans MAC address) I
The issue didn’t surface…

As a suggestion, I would include running ifconfig (as a verification step) in the setup?

Again, thank you!!

[ironsheep]

@suburbanidiot ok, great. I've updated our install instructions and added a troubleshooting section near the bottom.
Let me know if you would like anything more.

Great working with you!

[Webguyatwork]

I had an issue with the unique ID as well running on an straight Ubuntu container in Proxmox. My network device came up like this:

2: eth0@if77: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000

To get around the issue, I had to split the interface name on the 'at' symbol and take the first part.

In getNetworkIFsUsingIP, I had to replace:

interfaceName = lineParts[1].replace(':', '')

with:

interfaceName = lineParts[1].replace(':', '').split('@')[0]

This fix worked perfectly. My only issue now is that Ubuntu doesn't have a /proc/device-tree/model file to read the device information, but everything else is working now.

[ironsheep]

@suburbanidiot I've added the verification step to the README.md. (It turns out that people are still stumbling on this even after the troubleshooting section was added. )

[ironsheep]

@Webguyatwork hmm.... looks like running in the container is causing the ifconfig output to show the interfaces differently?
It should be simple enough to add the additional code you suggest.

What other parts aren't working for you?

[ironsheep]

@Webguyatwork removing all after '@' in next version...

[ironsheep]

@Webguyatwork well... thinking twice about this... so didn't make the latest release. You said the problem was it wasn't finding your ether address and you tracked it to the container-shaped network device name (one with '@' in it?)