Hellper - Your best friend in times of crisis

Hellper bot aims to orchestrate the process and resolution of incidents, reducing the time spent with manual tasks and ensuring that the necessary steps are fulfilled in the right order. Also, it facilitates the measurement of impact and response rate through metrics.

A chance to help explore and develop a bot written in Go, integrated with multiple external platforms and tools.

Help us expand incident processes’ and understand the needs of other companies that may benefit from Hellper bot.

You’re just one PR away from joining the developing team of Hellper! Contribute

Contents
Getting Started
- Prerequisites
- Installing
  - Variables explanation
Running the Tests
Running the application
Deployment
- Setup database
Optional Setup
How to use
- Commands
- Metrics
- Alerts
Contributing
Code of Conduct
Need help?
License

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Prerequisites

Installing

Clone this repo

git clone [email protected]:ResultadosDigitais/hellper.git

Configure Slack
Configure Google
Make a copy from configuration example

cp development.env.example development.env

Variables explanation

Variable	Explanation	Default value
HELLPER_BIND_ADDRESS	Hellper local bind address	`:8080`
HELLPER_DATABASE	Database provider (supported values: postgres)	`postgres`
HELLPER_DSN	Your Data Source Name	---
HELLPER_ENVIRONMENT	Current environment (supported values: production, staging)	---
HELLPER_GOOGLE_CREDENTIALS	Google Credentials	---
HELLPER_GOOGLE_DRIVE_TOKEN	Google Drive Token
HELLPER_GOOGLE_DRIVE_FILE_ID	Google Drive FileId to your post-mortem template	---
HELLPER_GOOGLE_CALENDAR_TOKEN	Google Calendar Token
HELLPER_GOOGLE_CALENDAR_ID	Google Calendar Id to schedule your post-mortem
HELLPER_POSTMORTEM_GAP_DAYS	Gap in days between resolve and postmortem event, by dafault the gap is 5 days if there is no variable	`5`
HELLPER_MATRIX_HOST	Matrix URL host	---
HELLPER_PRODUCT_CHANNEL_ID	The Product channel id used to notify new incidents	---
HELLPER_NOTIFY_ON_RESOLVE	Notify the Product channel when resolve the incident	`true`
HELLPER_NOTIFY_ON_CLOSE	Notify the Product channel when close the incident	`true`
HELLPER_NOTIFY_ON_CANCEL	Notify the Product channel when cancel the incident	`true`
HELLPER_SUPPORT_TEAM	Support team identifier to notify	---
HELLPER_PRODUCT_LIST	List of all products splitted by semicolon	`Product A;Product B;Product C;Product D`
HELLPER_REMINDER_OPEN_STATUS_SECONDS	Contains the time for the stat reminder to be triggered in open incidents, by default the time is 2 hours if there is no variable	`7200`
HELLPER_REMINDER_RESOLVED_STATUS_SECONDS	Contains the time for the stat reminder to be triggered in resolved incidents, by default the time is 24 hours if there is no variable	`86400`
HELLPER_REMINDER_OPEN_NOTIFY_MSG	Notify message when status is open	`Incident Status: Open - Update the status of this incident, just pin a message with status on the channel.`
HELLPER_REMINDER_RESOLVED_NOTIFY_MSG	Notify message when status is resolved	`Incident Status: Resolved - Update the status of this incident, just pin a message with status on the channel.`
HELLPER_OAUTH_TOKEN	Slack token to exeucte bot user actions	---
HELLPER_SLACK_SIGNING_SECRET	Slack token to verify external requests	---
FILE_STORAGE	Hellper file storage for postmortem document	`google_drive`
TIMEZONE	Timezone for Post Mortem Meeting	`America/Sao_Paulo`
HELLPER_SLA_HOURS_TO_CLOSE	Number of hours between the incident resolution and Hellper reminder to close the incident.	`168`

Running the Tests

make test

Running the application

make run

Deployment

Setup database

Run this command and copy the address:

heroku config:get DATABASE_URL

Run this command and past it on the YOUR_DATABASE_URL:

heroku config:set HELLPER_DSN=YOUR_DATABASE_URL

Import the scheema changing YOUR_HEROKU_APP_NAME by your application name:

heroku pg:psql --app YOUR_HEROKU_APP_NAME < internal/model/sql/postgres/schema/hellper.sql

Configure your environment variables

Optional Setup

Ngrok (To receive events from Slack)

Download Ngrok https://ngrok.com/ and create an account
sudo cp ngrok /usr/local/bin
ngrok http 8080
Copy your public address. You'll need this to Configure Slack API

Golang

Install golang

Install gvm
Follow gvm post install instructions
Install go 1.14 as default

Database

psql $HELLPER_DSN -f "./internal/model/sql/postgres/schema/hellper.sql"

How to use

Commands

After Configuring Slack you can use the commands created. The commands are as it follows:

Command	Short Description
`/hellper_incident`	Starts Incident
`/hellper_status`	Show all pinned messages
`/hellper_close`	Closes Incident
`/hellper_resolve`	Resolves Incident
`/hellper_cancel`	Cancels Incident
`/hellper_pause_notify`	Pauses incident notification
`/hellper_update_dates`	Updates the dates for an incident

The first command /hellper_incident can be use at any channel and/or conversation on Slack. It will open a pop-up for the user to set and start an Incident, creating the channel, meeting room link and post-mortem doc.

The remaining commands must be used only on the Incident's channel since they act on the specific incident that is open.

Metrics

This metrics came from metrics view table, they are calculated by the following formulas:

Metric	Description	Formula
start_ts	Date and time when the incident is started	Date and time in UTC from db
identification_ts	Date and time when the incident is identified	Date and time in UTC from db
end_ts	Date and time when the incident is resolved	Date and time in UTC from db
acknowledgetime	Time To Acknowledge	`identification_ts` - `start_ts`
solutiontime	Time To Solution	`end_ts` - `identification_ts`
downtime	Time in an incident	`end_ts` - `start_ts`
MTTA	Mean Time To Acknowledge	`total acknowledgetime` / `total incidents`
MTTS	Mean Time To Solution	`total solutiontime` / `total incidents`
MTTR	Mean Time To Recovery	`total downtime` / `total incidents`

Alerts

Alerts are useful for notifying the status of incidents. Notifications can be used in different situations, such as requesting an update of the incident status or finalizing the post-mortem. For this you can use the CLI notify on your CronJob service.

To use the CLI you need to build the binary file:

go build -o notify cmd/notify/main.go

Example of use

SHELL=/bin/bash
BASH_ENV=/app/.env

# At 16:30 on every week-day, from Monday through Friday, it sends a report to a selected channel with all incidents not closed
30 16 * * 1-5 root /app/notify --type=report --to=YOUR_SLACK_CHANNEL_ID --status=all

#  Every 30th minute it sends a status update request alert for all open incidents
0/30 * * * * root /app/notify --type=channels --status=open

# At 13:30 on every week-day, from Monday through Friday, sends a post-mortem request alert for all resolved incidents
30 13 * * 1-5 root /app/notify --type=channels --status=resolved

Contributing

Thanks for being interested in contributing! We’re so glad you want to help! Please take a little bit of your time and look at our contributing guidelines. All type of contributions are welcome, such as bug fixes, issues or feature requests.

Code of Conduct

Everyone interacting in the Hellper project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.

Need help?

If you need help with Hellper, feel free to open an issue with a description of the problem you're facing.

License

The Hellper is available as open source under the terms of the MIT License.

cristiantm / hellper Goto Github PK

hellper's Introduction