gcgarner / iotstack Goto Github PK

Awesome work with this project ! Absolutely great way to help people start using containers.
It is my first time working with containers and it is just great !
I have a little project that I have built based on influxdb, grafana and Zwave.
I am trying to learn how to build a container for Zwave for Raspberry but it is a lot of information out there that does not help to understand how to tailor the image build for arm processors and also I have no idea what base image should I put. Is there any guide you have followed to make this images you have included in this git ? If so, can you share it ? I would be happy to try and put together a Z-wave container that will work in this project as I am sure there will be may out there needing it .

BR,
Misujr

https://github.com/gcgarner/IOTstack/blob/master/menu.sh#L234 remove extra clone param

Hi gcgarner,

thanks a lot for creating the project. I found found it on youtube - the guy with the swiss accent. :-)
one nice option would be proxying all the services, to make them reachable via {url}/grafana, {url}/nodered, ...
one possible source of inspiration could be from:
https://github.com/janvda/balena-node-red-mqtt-nginx-TIG-stack

have a great day
echteler

Hi!

I came across this awesome project through Andreas Spiess' video and I was wondering if I can use this on an older 2B board.

It would really help me since I don't have to go out and buy a new one.

Thanks!

I have installed IOTstack on Virtual Machine to check how it works (used instruction from github) and when trying to start docker I get:
ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?
If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

I have restarted and recreated yaml but without any better result... Is IOTstack possible to run only on RPi?

Just for understanding it right: Will it be enough to have log2ram on the root system?
Won't each container log for itself?

Hey
I think there's a twisted letter in the templates/zigbee2mqtt/service.yml.
I had to change
/dev/ttyAMC0:/dev/ttyACM0
to
/dev/ttyACM0:/dev/ttyACM0

If Openhab and Adminer are selected, Adminer will get port 8080 instead of openhab.
The https for Opehab(8443) does work, however. Will test Opehab further and report.
Maybe consider changing the default port to Adminer for another port?
Wish List:

• Consider Loading SAMBA with sharing the /home/pi/IOTstack/volumes/openhab/conf folder as these get edited fairly often ( It would then be similar to Openhabian)
• Alternative Owner, Currently it has 9001 as the folder Owner this could be changed so that Pi could be the Owner.

The rest of Openhab performed as expected. I will run some more Tests but Well Done and Thank you Graham!

• rpieasy needs testing
• ha-dockermin

OS - Raspbian Buster Lite (Clean Image)
Followed the Steps in the Manual
When adding the various extra nodes for node red I got the following errors

Status: Downloaded newer image for nodered/node-red:latest
---> b04b04ad7675
Step 2/2 : RUN for addonnodes in node-red-node-pi-gpiod node-red-dashboard node-red-node-google node-r ed-node-emoncms node-red-node-ping node-red-node-random node-red-node-smooth node-red-node-darksky nod e-red-contrib-influxdb node-red-contrib-config node-red-contrib-grove node-red-contrib-diode node-red- contrib-bigtimer node-red-contrib-esplogin node-red-contrib-timeout node-red-contrib-moment node-red-c ontrib-particle node-red-contrib-web-worldmap node-red-contrib-isonline node-red-contrib-npm node-red- contrib-file-function node-red-contrib-boolean-logic node-red-contrib-blynk-ws node-red-contrib-owntra cks ; do npm install ${addonnodes} ;done;
---> Running in d40bf44e3aa7
npm notice created a lockfile as package-lock.json. You should commit this file.

• [email protected]
  added 4 packages from 2 contributors and audited 1375 packages in 14.243s
  found 2 vulnerabilities (1 low, 1 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 47 packages from 42 contributors and audited 1491 packages in 18.853s
  found 2 vulnerabilities (1 low, 1 high)
  run npm audit fix to fix them, or npm audit for details
  npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
  npm WARN deprecated [email protected]: This module moved to @hapi/hawk. Please make sure to switch over as th is distribution is no longer supported and may contain bugs and critical security issues.
  npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support p olicy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older vers ions (hapi.im/commercial).
  npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi supp ort policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes , and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
  npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support p olicy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older vers ions (hapi.im/commercial).
  npm WARN deprecated [email protected]: This module moved to @hapi/sntp. Please make sure to switch over as th is distribution is no longer supported and may contain bugs and critical security issues.
  npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support p olicy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older vers ions (hapi.im/commercial).
• [email protected]
  added 22 packages from 34 contributors and audited 1570 packages in 17.227s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 1 package from 1 contributor and audited 1571 packages in 12.65s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 1 package from 1 contributor and audited 1572 packages in 12.846s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 1 package from 1 contributor and audited 1573 packages in 12.87s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 1 package from 2 contributors and audited 1574 packages in 12.849s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 1 package from 1 contributor and audited 1575 packages in 12.946s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 3 packages from 6 contributors and audited 1578 packages in 15.649s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 1 package from 1 contributor and audited 1579 packages in 12.72s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 1 package from 1 contributor and audited 1580 packages in 14.564s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 1 package from 1 contributor and audited 1581 packages in 12.936s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 2 packages from 2 contributors and audited 1583 packages in 14.463s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 1 package from 1 contributor and audited 1585 packages in 13.069s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 1 package from 1 contributor and audited 1586 packages in 13.22s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 24 packages from 9 contributors and audited 1621 packages in 17.463s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
  npm WARN deprecated [email protected]: Please note that v5.0.1+ of superagent removes User-Agent header by defaul t, therefore you may need to add it yourself (e.g. GitHub blocks requests without a User-Agent header). This no tice will go away with v5.0.2+ once it is released.

[email protected] postinstall /usr/src/node-red/node_modules/core-js
node postinstall || echo "ignore"

Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!

The project needs your help! Please consider supporting of core-js on Open Collective or Patreon:

https://opencollective.com/core-js
https://www.patreon.com/zloirock

Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -)

• [email protected]
  added 13 packages from 22 contributors in 31.433s
• [email protected]
  added 14 packages from 19 contributors and audited 1819 packages in 21.177s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 2 packages from 2 contributors and audited 1821 packages in 14.828s
  found 3 vulnerabilities (1 low, 2 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 433 packages from 845 contributors and audited 13808 packages in 42.328s
  found 24 vulnerabilities (1 low, 23 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 1 package from 1 contributor and audited 13809 packages in 24.462s
  found 24 vulnerabilities (1 low, 23 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 1 package from 1 contributor and audited 13810 packages in 24.221s
  found 24 vulnerabilities (1 low, 23 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 4 packages from 5 contributors and audited 13822 packages in 27.912s
  found 25 vulnerabilities (1 low, 24 high)
  run npm audit fix to fix them, or npm audit for details
• [email protected]
  added 3 packages from 3 contributors and audited 13825 packages in 27.031s
  found 25 vulnerabilities (1 low, 24 high)
  run npm audit fix to fix them, or npm audit for details
  Removing intermediate container d40bf44e3aa7
  ---> a836c509eefe

Hope it hellps

Not sure if I am missing something but I have attempted to run the Script on Jessie, Strech and Buster and I am getting the same error :

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
chown: cannot access '/etc/openvpn/easy-rsa': No such file or directory
mkdir: cannot create directory ‘/etc/openvpn/easy-rsa/pki’: No such file or directory
main: line 825: cd: /etc/openvpn/easy-rsa: No such file or directory

What I have noticed is that the "/etc/openvpn/easy-rsa" file gets deleted if you mannually copy it ti the directory so maybe the script?

seems to be set to UTC and not the host time. host time correct when checked with $ date

• openHAB
• Home Assistant
• zigbee2mqtt
• Pi-Hole
• Telegraf
• tasmoadmin
• EspurinoHub
• Nextcloud
• Motioneye #76
• Mozilla webthings gateway #79
• Mariadb
• Python
• ownCloud
• nginx reverse proxy (busy with testing branch)
• lamp stack
• phpliteadmin + sqlite
• SAMBA/FTP server
• Z-wave
• ttn (amd64)
• loraserver (amd64)
• Gitea
• VictoriaMetrics #71
• IObroker
• domoticz
• homebridge
• unifi
• https://github.com/alexreinert/piVCCU
• https://github.com/mpromonet/webrtc-streamer

Can you add also Rtl_433 thanks

First a thanks for your IOTstack efforts.

By default hassio is a combination of a Home Assistant Docker container running on HassOS. It is available as downloadable (SD card) image from GitHub via https://www.home-assistant.io/hassio/installation/. Included in hassio's UI is the possibility to download add-ons from the hassio add-on store.

If I understand correctly IOTstack's Home Assistant Docker container is not hassio.
If so then the container name 'hassio' is very confusing and should therefore be renamed to 'homeassistant' instead.

According to https://www.home-assistant.io/hassio/installation/ and
https://github.com/home-assistant/hassio-installer/blob/master/hassio_install.sh
it should be possible to do an alternative install of hassio on Raspbian.
But because IOTstack's Home Assistant container has the standard Home Assistant UI and not hassio's UI (no add-on store in menu) my assumption is that IOTstack currently does not use hassio.

Hi, first of all thank you for the awesome work you have done!

My suggestions are that would be nice to have the following containers:

• NAS ( probably Samba?)
• Plex media server

airplay:
restart: unless-stopped
image: echteler/shairport:arm32v7
container_name: airplay
environment:
AIRPLAY_NAME: DockerOffice
privileged: true
devices:
- "/dev/snd:/dev/snd"
volumes:
- shairportConf:/etc
network_mode: host

I always use self build images, based on alpine... the original is orbsmiv/shairport-sync-rpi

Hi @gcgarner, what a great job!!

When creating the Telegraf container, an error occurs when mounting volumes:

ERROR: for telegraf Cannot start service telegraf: b'OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \\"rootfs_linux.go:58: mounting \\\\\\"/home/pi/IOTstack/services/telegraf/telegraf.conf\\\\\\" to rootfs \\\\\\"/var/lib/docker/overlay2/391d10a5f71b6968d5917f777ed56d92e1561f8db14d4c319c3fff27aaa439d9/merged\\\\\\" at \\\\\\"/var/lib/docker/overlay2/391d10a5f71b6968d5917f777ed56d92e1561f8db14d4c319c3fff27aaa439d9/merged/etc/telegraf/telegraf.conf\\\\\\" caused \\\\\\"not a directory\\\\\\"\\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type'

In my case, the error disappears if I map the directory ./services/telegraf in the volumes section of the service in docker compese.yml

The Telegraf service at docker-compose.yml would be:

telegraf:
    container_name: telegraf
    image: telegraf
    volumes:
      - ./services/telegraf:/etc/telegraf
      - ./services/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro
    depends_on:
      - influxdb
      - mosquitto

Thanks for your effort,

Ok , I am pretty sure this is more of an IO issue but how do I run a command from Node-red Exec on the host?

I build a small app in Node red and need run a nmap command "nmap -sn 192.168.4.1-254".

Could I please make a request for you to add Tasmoadmin (https://github.com/reloxx13/TasmoAdmin) as an optional container? I have tried the existing containers I can find for ARM but none seem to run for more than a few seconds on the Pi without stopping.

Thanks

Well done on the hard work!!
Can you add Openhab to the Mix as well?
https://hub.docker.com/r/openhab/openhab/

Doing some more Testing I have found the Node Red does not seem to be able to read files within the home file. I have created a quick flow to test and if you select Create the directory if it doesn't exist it seems to create the new file but I cannot find it in the location where I added it.
Where do I locate a file that is written in Node-Red and how can I link a file in Node-Red?

would be adding the admin auth to influx make sense?
you could add this to the env file for influx...

  INFLUXDB_HTTP_AUTH_ENABLED: "true"
  INFLUXDB_ADMIN_USER: admin
  INFLUXDB_ADMIN_PASSWORD: adminPW

If possible: What is the best way to go to call "sudo reboot" with an exec-node?

Consider Adding a Lampserver or alternative as default.
One could then have a Default page with all the Docker Apps Links on it as they are installed.

My Home Assistant configuration is using two devices connected via USB (USB to Serial).
According to docker-compose documentation (https://docs.docker.com/compose/compose-file/) ttyUSB* host devices can be made available in a container via the 'devices:' option as follows:

 hassio:
    image: "homeassistant/home-assistant:latest"
    container_name: "hassio"
    restart: unless-stopped
    network_mode: "host"
    #depends_on:
    #  - influxdb
    #  - mosquitto
    ports:
      - 8123:8123
    volumes:
      - ./volumes/hassio/config:/config
      - /etc/localtime:/etc/localtime:ro
    devices:
      - "/dev/ttyUSB0:/dev/ttyUSB0"
      - "/dev/ttyUSB1:/dev/ttyUSB1"

I have tried using both with and without the surrounding double quotes but both without success.
(I restarted the stack both times after updating docker-compose.yml and hassio service.yml.)

My Home Assistance setup is configured to use /dev/ttyUSB0 and /dev/ttyUSB1 and home-assistant.log shows that files /dev/ttyUSB0 and /dev/ttyUSB1 do not exist.
(This was not an issue previously when my Home Assistant was not running in a container.)

When opening a terminal to the hassio container (from within Portainer) and doing an ls /dev, the ttyUSB0 and ttyUSB1 devices are not visible (which is consistent with Home Assistant's error messages in the log).

How can ttyUSB0 and ttyUSB1 be made available in the Home Assistant container?

(It would be useful to add this information to the IOTstack documentation.)

Hi,

Just want to say thank you for the amazing work you have done on this project.

I'm not sure this is an issue so much as hoping you experienced the same during setup. The problem I am having is my dropbox back up. Running the scrips manually works well. Using the cron tab and outputting to a log file, shows that it is working and uploading, but it never arrives on dropbox. Yet the log says uploading DONE.
This is only using crontab -e that does not work?
Any ideas?

Could you explain how to add custom images to your project?
maybe looking at your files I can do that but it would be nice to have your wiki form for people to add more options.
I wanted to add Plex and Pi Hole.

Thanks

If you change the hostname before running the docker install command it will not be able to install docker.
I assume that this could be part of the read me?

• rclone google drive
• gdrive not under "active maintenance"

When I try to access my ha page at 192.168.1.53:8123 from a different pc or phone, I'm asked for a user name and password. IThere is no option to set up user name/ password when installing via docker, so I asssume these must have hard coded in the container?

Hi,
I installed your repo on my new 4G Raspberry and it works! Thank you.
Now I would like to add all my "customization". But of course, I do not want to lose it when I stop the container. Can you help me here?
Regards
Andreas

Hi I have installed and configured the stack with excellent results. All items seem to work I even set the password for mosquitto but I can’t seem to gain access to the interface.
Tried localhost:1883 and several other ports.
What am I missing to get into mosquitto
Thanks

Apple Siri server https://github.com/nfarina/homebridge (Docker image at https://github.com/marcoraddatz/homebridge-docker which I haven't tested).

A config file that can automatically select the things to install. Could also be configured to bypass the menu completely.

Not urgent or anything, just a suggestion :)

Hi All

I've created a branch called haas if anyone is able to test please verify that you can connect to local devices. Home Assistant uses port 8123 for the web ui

I've followed as instructed, but Hass.io isn't a menu choice. Please advise?

Can you please bundle RPIEasy home automation framework ? RPIEasy covers most of common sensors.By including it will be perfect standalone or network solution for home automation. Link to RPIEasy is below:

https://github.com/enesbcs/rpieasy

Thanks.

and one more :-)
you are using the standard images so far.
this might use a lot of space.
for most containers, there are alpine based ones out there.

could that be a possible switch at the beginning of your menu?

It may be my finger trouble but I followed the instructions to generate and activate a user/password combo for mosquitto and had difficulty after enabling.
The helper instructions "mosquitto_passwd -c /mosquitto/config/passwd MYUSER" put the new password file in passwd
BUT the last line in mosquitto.conf pointed to pwfile
I suspect the helper instruction should have been "mosquitto_passwd -c /mosquitto/config/pwfile MYUSER"

I'm using a Pi 4, bluetooth is working correctly.
I'm able to use bluetooth dependant nodes from within Node-RED running natively on the Pi, however attempting the same from Node-RED running in the IOTStack container fails and causes the container to restart.

Is it possible to access the bluetooth device from within the container?

I installed pigpiod and started it up. By default it starts locally and the node-red container cannot see port 8888 of the PI. If I change the startup to global (-g) pigpiod fails to start.

Having trouble installing and activating configurator with the docker set up.

I have run into a large number of compiler warnings (~3800 lines) which appear to be related to my decision to install node-red-node-sqlite via ./menu.sh

Initially I thought it might be related to my starting point ("Raspbian Buster with desktop" 2019-09-26) not having sqlite3 installed by default so I blew everything away and started over from scratch, including a:

sudo apt-get install sqlite3

along the way but I got the same long list of compiler warnings.

A test flow using the sqlite node definitely works so these warnings may be no more than an irritation. I'm only raising this issue to document the problem in case you have not been seeing the same thing in your testing.

I have not tried starting over but omitting node-red-node-sqlite at the ./menu.sh stage, then adding it later via Node-Red's "Manage Palette" command.

For the record, I'm using a brand new RPi4B with 4GB, booting from a 16GB SD card but running from a 440GB USB-3 SSD. The Pi is running "headless". The only other "add-on" prior to starting the Docker install is the VNC service.

SecondInstallSQLiteWarnings.txt.zip

I have attempted to install this stack and I can not get past this one section.

New install of Raspbian Buster Lite as I figure I don't need the GUI.
pulled the Stack and followed the directions.

I get this:

pi@raspberrypi:~/IOTstack $ ./menu.sh 
Adding portainer container
Adding Node-RED container
Adding influxdb container
Adding Grafana
Adding Mosquitto
docker-compose successfully created
run \'docker-compose up -d\' to start the stack
pi@raspberrypi:~/IOTstack $ docker-compose up -d
Building nodered
Step 1/2 : FROM nodered/node-red:latest
 ---> b04b04ad7675
Step 2/2 : RUN for addonnodes in node-red-node-pi-gpiod node-red-dashboard node-red-node-ping node-red-contrib-influxdb ; do npm install ${addonnodes} ;done;
 ---> Running in 537f7080658e
ERROR: Service 'nodered' failed to build: The command '/bin/sh -c for addonnodes in node-red-node-pi-gpiod node-red-dashboard node-red-node-ping node-red-contrib-influxdb ; do npm install ${addonnodes} ;done;' returned a non-zero code: 139
pi@raspberrypi:~/IOTstack $ 

If I copy the offending line and remove the '/bin/sh -c' it seems to work, but that does not get me past the docker-compose up command.

I did have to install npm prior to this as it was not in the initial OS load.
Are there dependencies missing that we are not aware of or am I doing something silly?

Hi @gcgarner

I think this was discussed over in Discord chat, it would be ideal to have date/time appended to the Dropbox backup file to avoid it being overwritten each time.

PS - I'm sure you have a solution so feel free to ignore the below ramblings:

Surprisingly it raises some issues in my inexperienced brain:

I had a look into docker_backup.sh, it would be simple enough to rename the file before uploading; sudo cp -a ~/IOTstack/backups/docker.tar.gz "backup-$(date +"%Y-%m-%d-%H%M%S").tar.gz"
Or even better create the timestamped name when compressing the files.

But how would the script know the filename to upload as it keeps changing with the time? Maybe just appending the date which would fix this issue. And how to remove the file from the backups folder after uploading, to save space? Maybe some thing like sudo rm backup*

And then on the Dropbox side of things, if doing daily backups this will add up to a lot of space over time.

Leaving it as it is solves all of the above issue, as it just overwrites with the latest copy. But if there is an issue and 11pm swings by, your last good backup is overwritten. However Dropbox does have a 30day 'rewind' feature so that could be used to recover a recent backup.

I'll be interested to see how this could be approached, or if it's not worth implementing.

Thanks!