Warning: These handlers are intended for use by Advanced sensu users. Do not use them if you are setting up Sensu for the first time. Use standard handlers from the community plugins repo
These work best with the Yelp monitoring_check or the pysensu-yelp
python library to make checks that these handlers act upon.
To Repeat: these handlers are special and require special event
data to work. If the special event data (like team) is not provided,
these handlers will do nothing.
The base handler is the only handler necissary to use. It is the default. All other handler behavior is derived from the event data.
This allows checks to use one handler, and we can add new features or deprecate old ones without changing client-side configuration.
The base handler also handles advanced filtering. It respects the following tunables:
alert_after- Seconds to wait before any handler is activated. Defaults to
realert_every- Integer which filters out repeat events (uses "mod").realert_every => 2would filter every other event. Defaults to-1which is treated as a special input and does exponential backoff.
This handler also provides many helping functions to extract team data, etc.
All other handlers inherit the base handler.
Uses the nodebot tool to send IRC notifications. Nodebot is helpful here as it retains a persistent connection to the IRC server, which can be expensive to setup.
- Sends notification to the
pages_irc_channelor${team_name}-pagesif the alert haspage => true - Sends notification IRC messages to the array of
irc_channelsspecified by the check, otherwise sends to thenotifications_irc_channelspecified in the team data. - If out of all that there are no channels, then no notifications will be sent.
Modification of the sensu-community-plugins mailer that can route emails to different destinations depending on the circumstance.
- Sends an email to the
notification_emaildestination if specified in the check. - Otherwise it uses the
notification_emailspecified by the team. - Will refuse to send any email if
notification_email => false.
Modification of the sensu-community-plugins handler that can open events on different Pagerduty services depending on the inputs.
- Only activates if the
pageboolean key in the event data is set to true - Only creates an incident on "Critical" levels (Does not page for warnings)
- Uses the
pagerduty_api_keyconfig set to theteamto determine which service to open or close an event in. - Tries to provide maximum context in the pagerduty event details
- Automatically closes events that are resolved.
This handler can make a JIRA ticket for an alert.
- The alert must have
ticket => true - Derives the Project to make the ticket in from the
projectkey set in the event data - Falls back to the default project for the
teamif unset. - WARNING: The Jira project must not have special required fields
- WARNING: Jira has special "transition" states in order to close tickets, this handler won't work if you have some custom "workflow"? (specifically, it won't be ble to close/fix/done issues. Patches welcome)
- WARNING: Be sure to use exponential backoff in order to not overload your Jira server.
There are other handlers included here that are not yelp-specific in the sense
that they do not use the team construct, and are included out of convenience.
If you are using the module itself, it can deploy the handlers and configure them.
class sensu_handlers {
# See the teams section
$teams => $team_data,
}See the inline docstrings in init.pp for parameter documentation.
The Sensu handlers must have the team declarations available for consumption. This data must be in hiera because currently the monitoring_check module also utilizes it.
On the plus side, hiera allows you to describe your team configuration easily:
sensu_handlers::teams:
dev1:
pagerduty_api_key: 1234
pages_irc_channel: 'dev1-pages'
notifications_irc_channel: 'dev1'
dev2:
pagerduty_api_key: 4567
pages_irc_channel: 'dev2-pages'
notifications_irc_channel: 'dev2'
frontend:
# The frontend team doesn't use pagerduty yet, just emails
notifications_irc_channel: 'frontend'
pages_irc_channel: 'frontend'
notification_email: 'frontend+pages@localhost'
project: WWW
ops:
pagerduty_api_key: 78923
pages_irc_channel: 'ops-pages'
notifications_irc_channel: 'operations-notifications'
notification_email: 'operations@localhost'
project: OPS
hardware:
# Uses the ops Pagerduty service for page-worhty events,
# but otherwise just jira tickets
pagerduty_api_key: 78923
project: METAL
This is a very important aspect of the configuration of these sensu handlers. The team syntax determines the default behavior of the handlers, given an input team.
Warning: If you typo a team name, the Sensu handlers will not know how to associate an alert with the right outputs. This is a common source of mistakes.
Lets look at the team syntax in more detail:
sensu_handlers::teams:
ops:
pagerduty_api_key: 78923
pages_irc_channel: 'ops-pages'
notifications_irc_channel: 'operations-notifications'
notification_email: 'operations@localhost'
project: OPS
sensu_handlers::teams:- Normal puppet-hiera lookup name. Matches 1:1 with the sensu_handlers module, teams parameter. This is a hashops:- Team name. This is the primary lookup keypagerduty_api_key: deadbeef- In pagerduty, this corresponds to a "service". That service must use the "generic" or "sensu" api format. Sharing the api key with a "Nagios" service will NOT workpages_irc_channel: ops-pages- If there is an event with page=>true, a notification will go to this channel. This parameter defaults to $team-pages. It can take an array of channels. No need to have the leading "#".notifications_irc_channel: 'operations-notifications'- Non-paging events will appear here. If ommited, defaults to $team-notifications. This also can accept an array, and does not need a leading "#"notification_email: 'operations@localhost'- If set, the handler will send emails for every event to this address. If ommited it will send no emails. You can send the email to multiple destinations by using comma separated list (like any email client)project: OPS- Used by the JIRA handler. If a event comes in that hasticket=>true, the jira handler will open a ticket on this project. There no default for this parameter. Special considerations have to be made for the JIRA project to enable auto-opening and auto-closing of tickets, see the docs on the jira handler.
You can manually invoke these handlers in order to test them, ensuring that (for example) a JIRA ticket is correctly raised. Simply pipe the Sensu alert in JSON into one of the handlers, and it should parse it as if it were a fresh alert.
$ grep 'failed' /var/log/sensu/sensu-server.log | tail -n 1 | jq .event > last_failed_event.json
$ cat last_failed_event | sudo -u sensu ruby jira.rb
Please open a github issue for support.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent