GithubHelp home page GithubHelp logo

a-bali / janitor Goto Github PK

View Code? Open in Web Editor NEW
79.0 1.0 6.0 627 KB

Availability monitoring and alerting for IOT devices

License: GNU General Public License v3.0

Go 69.21% HTML 28.76% Dockerfile 0.60% Makefile 1.42%
mqtt gotify ping iot-devices telegram iot monitoring homeassistant hassio homeautomation

janitor's Introduction

janitor

Objective

Janitor is a standalone tool that monitors the availability of your IOT devices and alerts you in case a device goes missing or stops transmitting data. This is particulary useful if you have many sensors, possibly with unstable hardware or connection, so you can take action in case of any issues and monitor the stability of your devices.

Janitor does not aim to implement any additional functionalities, therefore is not an alternative to your other home automation software (e.g. HASS). Focusing on solely this functionality will enable to keep this tool simple and efficient.

Janitor currently supports the following monitoring methods:

  • MQTT: Janitor will subscribe to predefined MQTT topics and monitor incoming messages. An average transmit frequency will be calculated for each channel and in case no new messages are received within this interval, Janitor will alert you (the threshold can be configured as a multiple of the average frequency or as absolute values per topic). This method is particulary useful for any kind of sensors submitting data regularly via MQTT (e.g. temperature).
  • Ping: Janitor will ping predefined hosts with a predefined frequency (configurable on a per host basis) and will alert you in case of no reply (the threshold used for consecutively missed pings can be configured). This method is useful for any kind of IOT devices e.g. sensors, cameras etc.
  • HTTP: Janitor will send a HTTP GET request to predefined addresses and check for reply, and, optionally, whether the reply contains a predefined string. Janitor will alert you in case of consecutively unsuccessful requests above the configured threshold. The frequency and timeout are also configurable per address. This method is useful for any kind of services with a web interface (e.g. APIs, hosted services etc.).
  • Exec: Janitor will execute a preconfigured command and check for its exit code. Janitor will alert you in case of consecutively unsuccessful executions above the configured threshold. The frequency and timeout are also configurable per command. With this method you can implement any kind of custom monitoring.

Janitor currently supports the following alert methods:

  • Telegram: Janitor will send a message to a predefined Telegram channel.
  • Gotify: Janitor will send a push message to Gotify.
  • MQTT: Janitor will publish a message to a preconfigured topic on a preconfigured MQTT server. The message will contain a JSON payload (see sample config for example). This is suitable for automations e.g. in HASS.
  • Exec: Janitor will execute a preconfigured command. This enables creating any type of custom alerting method.

Additionally, Janitor has a web interface where you can see the current status and historical data, remove items, change timeouts, intervals and thresholds and reload the configuration file (see screenshot below).

Finally, Janitor includes a simple JSON api with the following endpoints:

  • /api/data provides a snapshot of all monitoring related data.
  • /api/stats provides the count of monitoring targets in functional/dysfunctional state.
  • /api/metrics provides target statistics in Prometheus metrics format.

Screenshot

Screenshot

Building and installing

Janitor is written in Go and will compile to a single standalone binary. Janitor should compile and work both on Linux and on Windows.

For compiling, use at least Go version 1.16 and execute the following commands to clone the repository and build the binary:

$ git clone https://github.com/a-bali/janitor.git
$ cd janitor
$ go build

This will create the standalone binary named janitor that you can place anywhere you like. Pre-built binaries for releases are available on Github.

Configuration and usage

For configuration, a YAML formatted file is required. Please use the sample configuration file and change it according to your needs, following the comments in the file. Most of the variables are optional and have reasonable defaults, for details please see the comments.

A minimal but already operational configuration can be as short as follows (assuming Janitor's web interface will be available on its default port which is 8080):

monitor:
  mqtt:
    server: mymqtt.server
    targets:
    - topic: "/sensors/#"
alert:
  gotify:
    server: "http://mygotify.server:1234"
    token: gotify_token

Once you created a configuration file, Janitor can be launched as follows:

$ janitor path/to/your/configfile.yml

Janitor will log to standard output. The log is viewable on the web interface as well, where you can delete monitored targets and reload the configuration file (e.g. in case you added new targets or changed any of the settings).

Janitor will not daemonize itself. It is recommended to create a systemd service for janitor in case you want it running continuously.

Running with Docker

The latest version of Janitor is available on Docker Hub abali/janitor. To use this, map your configuration file to /janitor/config.yml:

$ docker run -v $(pwd)/config.yml:/janitor/config.yml -p 8080:8080 abali/janitor

Alternatively, you can use the supplied Dockerfile to build a container yourself :

$ git clone https://github.com/a-bali/janitor.git
$ cd janitor
$ docker build . -t janitor
$ docker run -v $(pwd)/config.yml:/janitor/config.yml -p 8080:8080 janitor

Future plans and contributing

Janitor's objective is clear and simple: to monitor the availability and operation of IOT devices and alert in case if any issues. Any future improvements should follow this objective and thus either add new ways of monitoring, or add new ways of alerting.

Janitor is open source software and you are encouraged to send pull requests via Github that improve the software.

License

Janitor is licensed under GPL 3.0.

janitor's People

Contributors

a-bali avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

janitor's Issues

When configuring topic with single level wildcard "+", timeout does not work

Hello, fist thanks for a great tool!

Issue is, when i have signle level wildcard "+" like this in my config:

monitor:
  mqtt:
    server: 192.168.x.x.
    port: 1883
    user: "username"
    password: "pass"
    targets:
    - topic: "zigbee2mqtt/sensor_out/#"
      timeout: 2000
    - topic: "other_gateway/+/1/#"
      timeout: 1200
    - topic: "other_gateway/+/2/#"
      timeout: 1200

then I can see topics like "other_gateway/15/1/2", "other_gateway/17/1/1" ... etc are monitored, but timeout 1200 does not apply for them. Instead, only standardtimeout is applied.

I need this filter because othergateway use this topic structure:
other_gateway/SENSOR_ID/CHILD_ID/MESSAGE_ID/

And I need to monitor all sensors, but just specific child_ids.

As a workaround I can specify all topics separately, but there are quite a lot of them, and each time new sensor is added, I need to modify this config.

Thank You.
pd.

How to setup MQTT-connection?

Hello,
Thank you for this very clean app; great programming.

Just a question regarding connecting to my MQTT-server for HomeAssistant.

I have HA and Janitor up and running; evrything works fine with Wifi and Ping but the MQTT-flow doesn't show anything.
Can you complete the example at the HomeAssistant-side to get all the necessary info (espacially: "/sensors/#" and "/stats/#".

I presume that this is done in the configuration.yaml off HA? Via MQTT-stream?
I found a link here: https://www.home-assistant.io/integrations/mqtt_statestream/

Thnx again!

Wrong time duration logged when error is resolved

In case of Exec/Ping/HTTP checks, LastError attribute gets updated every cycle, therefore the message logged when the error is resolved contains wrong time duration. E.g.:

[2024-03-12 13:44:58] ⚠ Exec ERROR for cam2 timelapse recent, last seen 2m0s ago
[2024-03-12 13:45:58] ⚠ Exec ERROR for cam1 timelapse recent, last seen 2m0s ago
[2024-03-12 13:59:00] ✓ Exec OK for cam1 timelapse recent, in error since 1m0s ago
[2024-03-12 13:59:01] ✓ Exec OK for cam2 timelapse recent, in error since 1m0s ago

Devices not shown for topic with v1.1

Thank you for this! I was looking for an easy way to monitor my MQTT sensors and alert me.

However, I am currently not able to get this to work, even though I don't run a special setup..

I am running this as docker. Log seems to be ok, it's connecting to my MQTT server alright - however, there are no sensors being detected. Checking this now for some time and didn't know why it won't work - maybe someone can help?

Janitor config

{
	"Debug": true,
	"LogSize": 1000,
	"Web": {
		"Host": "",
		"Port": 8080
	},
	"Alert": {
		"Telegram": {
			"Token": "",
			"Chat": 0
		},
		"Gotify": {
			"Token": "",
			"Server": ""
		},
		"Exec": "",
		"MQTT": {
			"Server": "mqtt://hostip",
			"Port": 1883,
			"User": "sensor",
			"Password": "sensorpass",
			"Topic": "/janitor/alerts"
		}
	},
	"Monitor": {
		"MQTT": {
			"Server": "mqtt://hostip",
			"Port": 1883,
			"User": "sensor",
			"Password": "sensorpass",
			"History": 10,
			"StandardTimeout": 1.5,
			"Targets": [
				{
					"Topic": "/zigbee2mqtt/#",
					"Name": "",
					"Timeout": 0
				}
			]
		},
		"Ping": {
			"Interval": 60,
			"Threshold": 2,
			"Targets": null
		},
		"HTTP": {
			"Interval": 60,
			"Timeout": 5000,
			"Threshold": 2,
			"Targets": null
		},
		"Exec": {
			"Interval": 60,
			"Timeout": 5000,
			"Threshold": 2,
			"Targets": null
		}
	}
}

docker compose file

  janitor:
    restart: always
    depends_on:
      - zigbee2mqtt
      - mosquitto
    volumes:
        - "/etc/localtime:/etc/localtime:ro"
        - "[...redacted]/janitor/config.yml:/janitor/config.yml"
    ports:
      - "7200:8080"
    container_name: janitor
    networks:
       - [...redacted]
    image: abali/janitor:v1.1

Logfile

2023-10-15T20:51:10.016978953Z [2023-10-15 22:51:10] Starting Janitor v1.1 (build date 2023-09-04T14:49:49+0000, 199fceb)
2023-10-15T20:51:10.017028395Z [2023-10-15 22:51:10] (Loaded config: [...redacted...]
2023-10-15T20:51:10.018699157Z [2023-10-15 22:51:10] Connected to MQTT server for monitoring at mqtt://localip:1883
2023-10-15T20:51:10.019611551Z [2023-10-15 22:51:10] Connected to MQTT server for alerting at mqtt://localip:1883
2023-10-15T20:51:10.019627822Z [2023-10-15 22:51:10] (Entering monitoring loop)
2023-10-15T20:51:10.019632393Z [2023-10-15 22:51:10] Launching web server at :8080
2023-10-15T20:51:10.019636255Z [2023-10-15 22:51:10] Subscribed to MQTT topics: /zigbee2mqtt/#

mqtt-explorer shows sensors with the relevant topic (zigbee2mqtt)
image

MQTT topics: LWT data

I would like to monitor a Topic to see if a particular payload is there.
e.g. on an LWT topic, my devices publish 'online', 'offline','restarting' etc.
I would like to catch the 'offline' state.

Thanks for a great app!

Support for mqtts

Hello,

Update: I've raised a PR #12

I'm wondering if you could add support for specifying the URL scheme in the config.yml? I've been running into an issue where my MQTT broker is setup to use SSL and thus the mqtts:// scheme (like https://) but it seems that Janitor doesn't support this?

I've built the Dockerfile locally and hardcoded the scheme in main.go which works for me, for now - the %s before seems to suggest it is supported but doesn't have a corresponding config key? I don't use golang though, so not sure.

// Original:
opts.AddBroker(fmt.Sprintf("%s://%s:%d", getConfig().Monitor.MQTT.Server, getConfig().Monitor.MQTT.Port))

// My change:
opts.AddBroker(fmt.Sprintf("mqtts://%s:%d", getConfig().Monitor.MQTT.Server, getConfig().Monitor.MQTT.Port))

If I don't specify mqtts, I get this error on startup of Janitor:

Unable to connect to MQTT for monitoring: network Error : EOF

If I specify mqtts in the server hostname in config.yml then I get the below error:

monitor:

  # MQTT monitoring
  mqtt:
    # Server connection details
    # Host name
    server: mqtts://mosquitto.int.price.gb.net
    # Port number (default: 1883)
    port: 8883
Unable to connect to MQTT for monitoring: network Error : dial tcp: lookup mqtts on 127.0.0.11:53: no such host

I don't use Golang at all, so don't know enough to propose a PR, but wondering if you could add support for this to be specified in config.yml at some point. Thanks for the great work you've done on Janitor - it's extremely helpful to me!

ESPHome integration possible?

Hey !

this is excactly what I trying to build inside home-assistant and failed with it. So grateful that you share your code. One question I like to monitor my Xiaomi MiFlora devices as the battery sensor is not working. Is it possible to integrate the ESPHome devices using the native ESPHome API?

Would you mind sharing a config snippet? Thank you very much!

Release tags and docker tags need

Hello! 1st, Janitor is a great tool! Thank you very much for this!

One notice: it could be better to use tags for git and docker images. It's a common practice (in DevOps world) which allows controlling about what current version is running on your host. Using "latest" tag is definitely bad practice.

Packr installation instructions no longer work

Installation guide appears to be out of date.

go get github.com/gobuffalo/packr

cannot find package "github.com/gobuffalo/packr/packr" in any of:
        /usr/lib/go-1.15/src/github.com/gobuffalo/packr/packr (from $GOROOT)
        /root/go/src/github.com/gobuffalo/packr/packr (from $GOPATH)
# golang.org/x/sys/unix
/root/go/src/golang.org/x/sys/unix/syscall.go:83:16: undefined: unsafe.Slice
/root/go/src/golang.org/x/sys/unix/syscall_linux.go:2256:9: undefined: unsafe.Slice
/root/go/src/golang.org/x/sys/unix/syscall_unix.go:118:7: undefined: unsafe.Slice
/root/go/src/golang.org/x/sys/unix/sysvshm_unix.go:33:7: undefined: unsafe.Slice

Custom error thresholds

Less of an issue, more of a feature request.

Would it be possible to have custom error thresholds for some devices; things like motion and door/window sensors which may see clusters of activity during the day, but nothing overnight. This pattern really throws the average out and outs them in an error state when they are actually performing correctly.

Obviously possible to get round this with devices that also send temp or some other consistent output, but not with ones that only have one binary trigger.

Feature Request | Ability to save name in config for MQTT Entries

Howdy,

I use mysensors which does not have a way to provide pretty names in its MQTT topics. I would like to request the ability to tag a name directly on a topic in the config so its easier to know what is broken. And to use that name in the notifications.

  • topic: "/mysgw-out/93/3/1/0/0"
    name: "Office Door Temperature"

docker container with ports?

Hello,
i have docker containers with different ports with same IP how do i add them?, is this kind of setup is
supported? i tried adding

 http:
  - name: container_name
    address: http://x.x.x.x:port
  - name: container_name2
    address: http://x.x.x.x:port2

error:net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

Refactor goroutines

Currently goroutines are launched at startup for the different types of monitoring, therefore the types of monitoring cannot be changed during runtime (via reloading the config file), only the contents for the different types.

Refactor code so that goroutines are stopped and (re)started upon reloading configuration. Use channels to signal to running goroutines.

Support for Personal Certificate Authority?

Does Janitor have support for specifying a personal certificate authority? My Mosquitto server is protected with a custom CA, and Janitor can't connect.

For example:

  mqtt:
    # Server connection details
    # Host name
    server: mqtts://mosquitto.int.price.gb.net
    # Port number (default: 1883)
    port: 8883
    # User and password if required
    user: admin
    password: ewpojfewhfiosdfsdojsdpoew
    ca: /certs/ca.crt

Notify issues via MQTT

Would if be possible to report the fact that an IoT device is down via MQTT? I use HomeAssistant (seen you in the forums there), and would like to be able to trigger things when a sensor stops sending, and this would be an easy way to inform HA.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.