GithubHelp home page GithubHelp logo

balerter / balerter Goto Github PK

View Code? Open in Web Editor NEW
297.0 10.0 18.0 31.26 MB

Script Based Alerting Manager

Home Page: https://balerter.com

License: MIT License

Go 93.90% Dockerfile 0.14% Lua 5.62% Makefile 0.34%
lua alert alertmanager golang go monitoring

balerter's Introduction

GitHub release (latest by date) Go Report Card Test codecov

logo.png

A Project in active development. Features may have breaking changes at any time before v1.0.0 version

Balerter is a scripts based alerting system.

In your script you may:

  • obtain needed data from different data sources (prometheus, clickhouse, postgres, external HTTP API etc.)
  • analyze data and make a decision about alert status
  • change Alerts statuses and receive notifications about it

In the example bellow we create one Clickhouse datasource, one scripts source and one alert channel. In the script we run query to clickhouse, check the value and fire the alert (or switch off it)

Notification channels

  • Slack
  • Telegram
  • Syslog
  • Desktop Notify
  • Email
  • Discord
  • Webhook
  • Prometheus Alertmanager
  • Prometheus AlertmanagerReceiver
  • Twilio Voice (phone calls)

Datasources

  • Clickhouse
  • Prometheus
  • Postgres
  • MySQL
  • Loki
  • Any external API with http lua module

Full documentation available on https://balerter.com

Example

docker pull balerter/balerter
docker run \
    -v /path/to/config.yml:/opt/config.yml \
    -v /path/to/scripts:/opt/scripts \ 
    -v /path/to/cert.crt:/home/user/db.crt \
    balerter/balerter -config=/opt/config.yml

Config file config.yml

scripts:
  folder:
    - name: debug-folder
      path: /opt/scripts
      mask: '*.lua'

datasources:
  clickhouse:
    - name: ch1
      host: localhost
      port: 6440
      username: default
      password: secret
      database: default
      sslMode: verified_full
      sslCertPath: /home/user/db.crt

channels:
  slack:
    - name: slack1
      url: https://hooks.slack.com/services/hash

Sample script rps.lua

-- @cron */10 * * * * *
-- @name script1

local minRequestsRPS = 100

local log = require("log")
local ch1 = require("datasource.clickhouse.ch1")

local res, err = ch1.query("SELECT sum(requests) AS rps FROM some_table WHERE date = now()")
if err ~= nil then
    log.error("clickhouse 'ch1' query error: " .. err)
    return
end

local resultRPS = res[1].rps

if resultRPS < minResultRPS then
    alert.error("rps-min-limit", "Requests RPS are very small: " .. tostring(resultRPS))
else
    alert.success("rps-min-limit", "Requests RPS ok")
end 

Also, you can to write tests!

An example:

-- @test script1
-- @name script1-test

test = require('test')

local resp = {
    {
        rps = 10
    }
} 

test.datasource('clickhouse.ch1').on('query', 'SELECT sum(requests) AS rps FROM some_table WHERE date = now()').response(resp)

test.alert().assertCalled('error', 'rps-min-limit', 'Requests RPS are very small: 10')
test.alert().assertNotCalled('success', 'rps-min-limit', 'Requests RPS ok')

See a documentation on https://balerter.com

balerter's People

Contributors

dgolubets avatar maxim-usikov avatar negasus avatar negbie avatar tdakkota avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

balerter's Issues

Implement support for elastic/opensearch

Adding support for Elastic / OpenSearch would open up a wide range of use cases for folks currently using elastalert or elastalert2.

Work in progress

I've started sketching support in this branch using OpenSearch and the Go client from @olivere. These were selected to avoid potential issues with Elastic's viral licensing changes. Currently it supports connecting to a datastore and simple query_string based queries.

There is a bit more work to do before it's ready for a PR, including fleshing out the tests, sorting out dependencies, and support for various auth mechanisms.

Interested in feedback on general approach and how to make this best fit into the balerter design ethos.

Startup up containers and add a document

$ docker-compose up
....
$  curl -H "Content-Type: application/json" -XPOST "http://localhost:9200/testindex/testtype" -d "{ \"foo\" : \"bar\"}"               [20:01:09]
{"_index":"testindex","_type":"testtype","_id":"iM0nAXwBZaWvkaRHbMYd","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}%                              

Define a simple script and run it

-- @cron * * * * * *

local log = require("log")
local alert = require("alert")

local es1 = require("datasource.elastic.es1")

local res, err = es1.query("testindex", "foo: bar")
if err ~= nil then
    log.error("elastic 'es1' query error: " .. err)
    return
end

if #res >0 then
  log.info("Got more than one hit")
else
  log.info("Got no hits")
end
scripts {
  updateInterval = 5000
  folder "scripts"{
    path = "balerter-test/scripts"
    mask = "*.lua"
  }
}

datasources {
  elastic "es1" {
    host = "127.0.0.1"
    port = 9200
    scheme = "http"
    sniff = "false"
    timeout = "10000"
  }
}


channels {
  notify "notify1" {}
}
$ balerter -config balerter.hcl                                                                                                                                                         [20:02:32]
{"level":"info","ts":1632106954.4304147,"msg":"balerter start","version":"undefined"}
{"level":"info","ts":1632106954.4307108,"msg":"init scripts manager"}
{"level":"info","ts":1632106954.4307454,"msg":"init datasources manager"}
{"level":"info","ts":1632106954.442129,"msg":"init upload storages manager"}
{"level":"info","ts":1632106954.442164,"msg":"init core storages manager"}
{"level":"info","ts":1632106954.4421866,"msg":"init channels manager"}
{"level":"info","ts":1632106954.4422402,"msg":"init runner"}
{"level":"info","ts":1632106954.4422848,"msg":"run runner"}
{"level":"info","ts":1632106955.067425,"msg":"Got more than one hit","scriptName":"folder.scripts.test"}
...
^C{"level":"info","ts":1632106956.6963909,"msg":"got os signal","signal":"interrupt"}
{"level":"info","ts":1632106956.6964293,"msg":"stop jobs"}
{"level":"info","ts":1632106956.6964521,"msg":"terminate"}

Discuss how to write docs

The current way of documentation is done via github pages which is quite inconvenient for new contributors.

Maybe a better way would be to use something like https://readthedocs.org/ and add a link on the homepage? What do you think or do you have a suggestion for something else?

Charts as attachement

Mby it's useful to be able to send charts as attachement aswell. What do you think?

Make `logLevel` and `debug` flag values known to lua scripts

It is a common case that many diagnostic output statements like print() or log.debug() statements are used in script while lua script is under development and not needed in production version.
Also, some fragments of script are often only required during debug runs and have to be turned off somehow in production code.

Currently lua scripts are unaware of the values for logLevel and debug flags used when running balerter. If would be useful to make those values available in scripts at runtime.
Then one can write code like this:

if runtime.logLevel == 'INFO' then print('info') end
if runtime.debug then <some debug related code here> end

Implement support for parameters within messages

It would be quite helpful for a number of use cases if messages could contain an optional set of key value pairs. This would allow for scripts to extract values and dispatch them for later use. For example, webhooks could use these values in the body of the Post payload. Or slack/discord messages could embed them to add more context to alerts.

Elastalert supports something similar to this whereby an alert can extract field values and then use them to add context to alerts.

Example

include:
  - match_body.host.hostname
  - match_body.user.name
  - match_body.source.ip

alert_subject: "SSH abuse (repeat offender) on <{}> | <{}|Show Dashboard>"
alert_subject_args:
  - match_body.host.hostname
  - kibana_link

Systems notifier

Problem

Wrong behavior notify channel

Actual

I tried use a channel Notify for monitoring my private service availability.
I have config:

$ cat config.yaml
scripts:
  sources:
    update_interval: 5s
    folder:
    - name: pinger-dns
      path: /opt/scripts
      mask: '*pinger.lua'
channels:
  notify:
  - name: balerter-notify

And script in /opt/scripts/

cat /opt/scripts/dns-pinger.lua
-- @interval 1s
-- @name notify

local command = "ping -c 1 my.private.domain"
local handle = io.popen(command)
local result = handle:read("*a")
handle:close()

local alert = require('alert')
if string.find(result, '1 received') then
	alert.success('OK', 'VPN available')
else
	alert.error('ERROR', 'VPN NOT available')
end

And when I changed status my vpn connect I had in log this:

{"level":"info","ts":1584042065.1926973,"caller":"balerter/main.go:64","msg":"balerter start","version":"undefined"}
{"level":"debug","ts":1584042065.1929371,"caller":"balerter/main.go:72","msg":"loaded configuration","config":{"scripts":{"sources":{"folder":[{"update_interval":0,"name":"pinger-dns","path":"/opt/scripts","mask":"*pinger.lua"}]}},"datasources":{"clickhouse":null,"prometheus":null,"postgres":null,"mysql":null},"channels":{"slack":null,"telegram":null,"syslog":null,"notify":[{"name":"balerter-notify","icons":{"success":"","error":"","warning":""}}]},"storages":{"s3":null},"global":{"send_start_notification":null,"send_stop_notification":null,"api":{"address":"","metrics":false}}}}
{"level":"info","ts":1584042065.1932344,"caller":"balerter/main.go:75","msg":"init scripts manager"}
{"level":"info","ts":1584042065.1932533,"caller":"balerter/main.go:83","msg":"init datasources manager"}
{"level":"info","ts":1584042065.1932702,"caller":"balerter/main.go:91","msg":"init storages manager"}
{"level":"info","ts":1584042065.1932845,"caller":"balerter/main.go:104","msg":"init alert manager"}
{"level":"info","ts":1584042065.1933174,"caller":"balerter/main.go:168","msg":"init runner"}
{"level":"info","ts":1584042065.1933365,"caller":"balerter/main.go:171","msg":"run runner"}
{"level":"debug","ts":1584042065.1935132,"caller":"runner/runner.go:79","msg":"update scripts","count":1}
{"level":"info","ts":1584042065.193549,"caller":"manager/api.go:59","msg":"serve api server","address":""}
{"level":"debug","ts":1584042065.1935747,"caller":"runner/runner.go:98","msg":"run script job","hash":"f09a77009f537d896dd2ed432946a9a737b20528","script name":"/opt/scripts/dns-pinger.lua","interval":1}
{"level":"debug","ts":1584042065.1936605,"caller":"runner/job.go:35","msg":"run job loop","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042065.1936796,"caller":"runner/job.go:56","msg":"create job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042065.1939332,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042065.2264009,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"OK","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN available","alertLevel":1,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042066.2267144,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042066.2611897,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"OK","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN available","alertLevel":1,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042067.2614403,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042067.295141,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"OK","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN available","alertLevel":1,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042068.2953045,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042068.3274243,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"OK","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN available","alertLevel":1,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042069.3275747,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042069.3516433,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042070.3579571,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042070.3841815,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042071.3843913,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042071.4058106,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042072.406013,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042072.4383116,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042073.4384856,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042073.4672143,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042074.467391,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042074.501433,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042075.5015755,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042075.5350842,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"OK","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN available","alertLevel":1,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042076.5353124,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042076.5726438,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"OK","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN available","alertLevel":1,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042077.572808,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042077.6034443,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"OK","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN available","alertLevel":1,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042078.6035912,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042078.6391623,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"OK","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN available","alertLevel":1,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042079.6393101,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042079.6690571,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"OK","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN available","alertLevel":1,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042080.66921,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042080.7021093,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"OK","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN available","alertLevel":1,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042081.7022893,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042081.7284563,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042082.7286816,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042082.7485447,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042083.7487128,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042083.7749538,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042084.7751884,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042084.7984426,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042085.1938145,"caller":"runner/runner.go:79","msg":"update scripts","count":1}
{"level":"debug","ts":1584042085.1939087,"caller":"runner/runner.go:94","msg":"script already running","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042085.798663,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042085.819705,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042086.8201966,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042086.8463368,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042087.846516,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042087.874271,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"ERROR","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN NOT available","alertLevel":3,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042088.8746927,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042088.924336,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"OK","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN available","alertLevel":1,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"debug","ts":1584042089.9246237,"caller":"runner/job.go:41","msg":"run job","name":"/opt/scripts/dns-pinger.lua"}
{"level":"debug","ts":1584042089.9738853,"caller":"manager/lua_call.go:86","msg":"call alert luaCall","alertName":"OK","scriptName":"/opt/scripts/dns-pinger.lua","alertText":"VPN available","alertLevel":1,"alertOptions":{"Fields":null,"Channels":null,"Quiet":false,"Repeat":0,"Image":""}}
{"level":"info","ts":1584042090.357878,"caller":"balerter/main.go:186","msg":"got os signal","signal":"interrupt"}
{"level":"debug","ts":1584042090.3579376,"caller":"runner/runner.go:124","msg":"stop jobs"}
{"level":"debug","ts":1584042090.3579574,"caller":"runner/runner.go:130","msg":"stop script job","hash":"f09a77009f537d896dd2ed432946a9a737b20528","script name":"/opt/scripts/dns-pinger.lua"}
{"level":"info","ts":1584042090.3579805,"caller":"manager/api.go:68","msg":"shutdown api server"}
{"level":"info","ts":1584042090.3580391,"caller":"balerter/main.go:200","msg":"terminate"}

When status was changed from OK to ERROR first time, I had an error notification. For the next changes from the ERROR to OK and visa versa I didn't have any notifications.

EXPECTED

For the each changes status from OK to ERROR and visa versa I expect notification message.

/build/balerter/internal/runner/job.go:54 error run job

2020-11-23T06:43:07.728Z        DEBUG   /build/balerter/internal/alert/provider/telegram/send.go:10     tg send message
2020-11-23T06:43:07.728Z        ERROR   /build/balerter/internal/runner/job.go:54       error run job   {"script name": "disk-low-space-ubd", "error": "runtime error: invalid memory address or nil pointer dereference\nstack traceback:\n\t[G]: in function 'error'\n\t<string>:5: in main chunk\n\t[G]: ?"}
github.com/balerter/balerter/internal/runner.(*Runner).runJob
        /build/balerter/internal/runner/job.go:54

Embed default lua modules into the binary or make a flag for it

Currently you use
defaultLuaModulesPath = "./?.lua;./modules/?.lua;./modules/?/init.lua"

which isn't really portable. Often users move their binaries just under /opt and don't even know that they need the modules directory. Maybe it's better to create a new flag so the user can specify this directory itself or you embed the default lua modules into the binary. There are some libs for that like

https://github.com/aletheia-icu/broccoli
https://github.com/UnnoTed/fileb0x/tree/master/_example/simple

Create some examples

Hi @negasus
what do you think about adding a example folder with some more examples? Currently I play with following:

Use yahoo finance API to fetch some predefined stocks and save the value into KV store. Next API call I check if the returned value is a predefined amount of % lower than the previous one and fire an alert.

Не возможно использовать webhook chanel для алетов

Попытался использовать webhook chanel для передачи алертов в свой кастомный апи для телеграмм ботов.
curl --header 'Content-Type: application/json' --request 'POST' --data '{"chat_id":"-11111111","text":"Тестовая отправка сообщения"}' http://proxy-api-url:8080/sendMessage
но в https://balerter.com/0.9.1/configuration/channels.html#type-webhook
Я сделал пытался сделать по аналогии с curl

channels:
  webhook:
    - name: wh1
      settings:
        url: http://proxy-api-url:8080/sendMessage
        method: POST
        payload:
          queryParams:
            chat_id: "-11111111"
            text: Алерт <----- Нельзя менять из скрипта
          body: body content <----- Нельзя менять из скрипта
        timeout: 5000
        headers:
            Content-Type: application/json
      ignore: false

нельзя менять текст параметра text....

sqllite storage not working

When SQLite selected as storage for KV and alerts, Balerter falls with error:

error create core storages manager, error create file storage, Binary was compiled with 'CGO_ENABLED=0', go-sqlite3 requires cgo to work. This is a stub

Balerter version: v0.9.0

Inject time of previous job run into lua state

Inject the time of previous cron execution into context of script execution. This would allow for alerts to run queries between the execution time and the time of prior execution. Perhaps create a meta module to expose these?

local meta = require("meta")
log.info("time of prior execution: " .. meta.priorExecutionTime)

Maybe there are some other useful bits of context to make available to scripts? Perhaps cron location?

Write log when sending message to telegram not successful

For example, in a situation where no chat id is found Telegram will return:

{
  "ok": false,
  "error_code": 404,
  "description": "Not Found"
}

Response currently not being processed:

func (api *API) sendMessage(body []byte, method string) error {
	req, err := http.NewRequest(http.MethodPost, api.endpoint+method, bytes.NewReader(body))
	if err != nil {
		return fmt.Errorf("error generate request to telegram, %w", err)
	}
	req.Header.Add("Content-type", "application/json")

	res, err := api.httpClient.Do(req)
	if err != nil {
		return fmt.Errorf("error send request, %w", err)
	}

	res.Body.Close()

	return nil
}

Mattermost/Slack incoming webhooks

Do you think it makes sense to add Mattermost/Slack webhooks or do you mean something else in the project description with channel: webhook?

Additional cron based runJob

The current watcher is Interval (time.Duration) based. A cron based watcher would allow some kind of reporting functionality. What do you think?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.