status-im / status-github-bot

A bot for github

statusbot

statusbot is a chat bot built on the Probot framework. A wiki is available here.

This README will help you get started. It will be updated and improved to discuss user-specific instances, usage and deployment, the functionality available, and more!

What does the bot do?

• Background management in GitHub:
  • Assign new PRs to the Pipeline for QA project board (REVIEW column).
  • Move existing PRs to the correct Pipeline for QA project board column (REVIEW/IN TEST), depending on whether or not the required conditions are met: is mergeable, at least two reviewers have approved it, and there is no request for changes.
  • Assign issues that are labeled bounty-awaiting-approval to the Status SOB Swarm project board (bounty-awaiting-approval column).
  • Welcome users who post their first PR in a project.
  • Check whether all commits are GPG signed, and set the PR status accordingly.
  • Unfurl links on Issues and Pull Request discussions.
  • Disallow merging of PRs containing WIP in the title.
  • Mention repo collaborators on Slack when a GHI is assigned the bounty-awaiting-approval label.
  • After a PR is moved to the column specified in automated-tests/kickoff-column-name and the build has passed successfully, the bot will kick a test automation build in Jenkins (retry periodically if the PR build is still running).
  • New functionality will be added in the future, and the wishlist is being tracked here).

The project board's names, column names, welcome message, and other values are stored in the .github/github-bot.yml file. It can be overriden for each specific repository by adding a file in the same path on the respective repository (see probot-config).

Development

To get your environment set up, go through the following steps:

1. Run yarn install.

2. Populate .env.

   cp .env.example .env213804
   
   # edit .env file to contain proper config

After this, you can start the bot by running the following:

yarn start

Creating the Slack Bot Integration

1. Go to https://my.slack.com/services/new/bot.
2. Add a bot integration.
3. Note the bot token starting with xoxb-, and put it into .env.

Creating the bot GitHub App

This bot is meant to be packaged as a GitHub App. There are two steps to it: creating the app, and installing the app. The app needs to be created only once and can be made public and reused for any number of repositories and organizations.

See the official docs for deployment.

1. Create the GitHub App:
   1. In GitHub, go to Settings/Developer settings/GitHub Apps and click on New GitHub App
   2. Enter the bot name in GitHub App name, e.g. Status GitHub Bot
   3. In Homepage URL, enter the /ping endpoint of the service, e.g. https://5e63b0ab.ngrok.io/ping
   4. In Webhook URL, enter the root endpoint of the service, e.g. https://5e63b0ab.ngrok.io/
   5. In Webhook secret (optional), enter a string of characters that matches the value passed in the in the WEBHOOK_SECRET environment variable.
   6. This app requires these Permissions & events for the GitHub App:
      • Commit statuses - Read & write
      • Issues - Read & Write
        • Check the box for Issue comment events
        • Check the box for Issues events
      • Pull requests - Read & Write
        • Check the box for Pull request events
        • Check the box for Pull request review events
        • Check the box for Pull request review comment events
      • Repository contents - Read-only
        • Check the box for Push events
      • Repository projects - Read & Write
        • Check the box for Project for repository projects events
        • Check the box for Project card for repository projects events
      • Organization projects - Read-only
        • Check the box for Project for organization projects events
      • Single File - Read-only
        • Path: .github/github-bot.yml
   7. 🔍 Verify that you have ticked 9 boxes.
   8. Generate a private key pass and save it.
2. Installing the bot service:
   1. Deploy the bot to the cloud.
   2. Set the APP_ID environment variable to value reported when the GitHub App was created.
   3. Set the WEBHOOK_SECRET environment variable to the value configured in the GitHub App.
   4. Set the PRIVATE_KEY environment variable to the contents of the .pem file.
   5. Set the SLACK_BOT_TOKEN environment variable to the value reported for the bot in Slack.
3. Install the GitHub App in an account:
   1. Select the repositories where the bot should work (e.g. status-react).

Customizing the bot

The bot gets its settings from a per-repo file located at .github/github-bot.yml. That file extends the base file at the status-im/probot-settings repo.

Examples of settings that can be configured:

• github-team/slug: Slug of the team that owns the respective repository

• welcome-bot/message-template: First time contributor welcome message template. Examples of template values allowed:

  • {user}: Replaced by the PR submitter's user name
  • {repo-name}: Replaced by the PR's target repository name
  • {pr-number}: Replaced by the PR number

• slack/notification/room: Slack room used for notifications (e.g. status-probot)

• Repository project board settings:

  • project-board/name: Name of the QA pipeline project board
  • project-board/contributor-column-name: Name of the column in the project board to group issues that are being worked on by a contributor
  • project-board/review-column-name: Name of the column in the project board to group issues that are up for review
  • project-board/test-column-name: Name of the column in the project board to group issues that up for testing by QA

• Bounty project board settings:

  • bounty-project-board/name: Name of the bounty project board in GitHub
  • bounty-project-board/owner: GitHub username of the maintainer of the bounty project board (used to e.g. send Slack notifications)
  • bounty-project-board/awaiting-approval-column-name: Name of the column in the bounty project board to group issues that are awaiting for bounty approval
  • bounty-project-board/awaiting-approval-label-name: Name of the label used in issues to declare that an issue is awaiting approval to become a bounty
  • bounty-project-board/bounty-label-name: Name of the label used in issues to declare that an issue is a bounty
  • bounty-project-board/bounty-size-label-name-regex: Regular expression that matches the bounty size label and returns a group containing the size itself
  • bounty-project-board/post-approved-bounties-to-slack-room: Name of the Slack room where to cross-post approved bounties

• Automated tests settings:

  • automated-tests/repo-full-name: Full name of the repo to watch in project cards in order to automatically run automated tests CI job (e.g. status-im/status-react)
  • automated-tests/job-full-name: Full name of the CI job to run automated tests (e.g. end-to-end-tests/status-app-end-to-end-tests)

Restart the bot

You may want to get comfortable with heroku logs and heroku restart if you're having issues.

status-github-bot's Issues

Finalize hubot deployment

For the lack of a better place, I'm capturing the TODOs in an issue.

Hubot as been deployed, but a few issues remain due to the fact that we're using a free Heroku plan. Namely:

• The bot will be put to sleep by Heroku after 30 minutes of inactivity. A keep-alive is in place, but I believe that this can only help for 16 hours. After that Heroku will terminate the app.
• The memory quota for the free Heroku plan (512MB) is exceeded.
• By default the app does not save data across restarts (which causes it to lose the GitHub authentication token). To solve this we need a paid plan with Redis. Currently I've configured it to use my personal account in Azure.
• The GitHub App is currently being hosted on my GitHub account. We could move it to the status-im github org.

Notify reviewers of requested reviews through Slack

User Story

As a developer, I want to have clear visibility of when a pull request review is asked from me. Pull request reviews sometimes fall through the cracks due to the volume of emails/GitHub notifications.

Description

A better way to address this issue would be to leverage the new GitHub bot (written on the Probot framework) to listen to PR comment events for a specific command (e.g. /review @PombeirP @divan), and then act on it by telling GH to create the review request, and notifying the respective reviewers on Slack (either through a DM or a mention on a given room).

Since GitHub user names don't necessarily match Slack user names, the bot needs to use the cache held in index.js.

Documentation for the bot is available in the wiki.

Unnecessary comment edit

Bot edited this comment but I am not really sure why and what was changed. It seems like unnecessary change.

Make an MVP for a Github bot

The current flow around https://github.com/status-im/status-react/projects/7 is suboptimal for several groups of people: Core Contributors, QA team, Community, and External contributors. It requires a lot of manual effort and things easily gets dropped.

To solve that issue we would like to have a bot that automates some basic maintenance tasks in our repos.

It could be done using the hubot framework or preferably if it is possible an open source library using the GitHub rest API such as tentacles (would need to be forked and updated).

There will be more issues with more use cases. This one has a larger bounty because it includes the set up of the initial project and the eventual library that has to be developed/updated.

Use case

Put newly open PRs in status-react repo to pipeline for QA project and move them to the REVIEW column

Acceptance criterias

the bot should:

• have clear manual instructions for installation
• include the use case above
• be testable, you can set it up on one of your own repo to demonstrate it
• be extensible, it's design should allow implementation of new use cases with minimal effort

Example of Facebook GitHub bot

Examples

• Automated interaction for detailed issue reporting: facebook/react-native#5178 (comment)
• Identify potential reviewers based on change history: facebook/react-native#9160 (comment)
• Notification about updated PR facebook/react-native#9160 (comment)

Allow separate configuration of stale timeout for PRs

Currently, there's a single parameter (stale/daysUntilStale) which applies to both issues and PRs. We need to add a separate override parameter for PRs (stale/daysUntilPullRequestStale). If present, it will override stale/daysUntilStale for PRs.

Have bot ping Status Open Bounty funders when three thumbs up are added to a bounty issue

User Story

As a SOB funder, I want to know when a new issue has been approved by developers, so that I can react more quickly without having to search for information.

Description

A better way to address this issue would be to leverage the new GitHub bot (written on the Probot framework) to:

1. listen to issue comment changes;
2. check if issue is labelled with bounty-awaiting-approval;
3. check the reactions summary in the issue and proceed only if the number of +1 reactions is > 2;
4. act on it by notifying the Status Open Bounty funders on Slack through a DM. Location for list of funders TBD in yml config file. List will be filled later on by team.

Documentation for the bot is available in the wiki.

Write script to watch over Status repos for any private keys commited by mistake

User Story

As a developer, I want to be warned if I happen to commit by mistake a private key of an ETH account which contains funds (ETH or ERC20), so that the risk of losing funds is reduced.

Description

The script should watch all the pushes on https://github.com/status-im repos for strings that match the format of either a private key, or a 12 word recovery key. For all the keys that are found, it should look for the balance of the account, and if non-zero, send a warning over Slack to #devops channel.

Documentation for the bot is available in the wiki.

GitHub

Status

A Mobile Ethereum Client.

Test issue

Help create an amazing first contribution experience

Currently the bot greets people with a somewhat impersonal message:

As @PombeirP described to me, the original motivation here was mostly that Status Core gets a notification when new contributors come along. This message however is also the first interaction between Status as an organization and potential contributors. We should invest in this interaction.

Some ideas to improve it:

• 🙋🏽‍♂️ Instead of a message by a bot it could be written by a human, perhaps a contributor to the repo the PR belongs to. I believe being welcomed by an actual human may go a long way in creating a lasting connection between Status and a new contributor. (The role of the bot would then be pinging an appropriate human and maybe supplying him with some additional information about the user, maybe even a varying message template)
• ⭐️ The message could contain additional information such as links to documented best practices for the repo or testing instructions.

Improve script which initializes bot with GitHub->Slack name mappings for active users

Description

The lib/retrieve-slack-github-users.js script retrieves all the Slack users and fetches the respective profile to retrieve the GitHub ID. Currently the initial list fetch operation is not paginated, which poses a risk once Slack will clamp down on non-paginated API calls. The goal is to adapt the script so that it paginates the call using cursors.

Implementation

The Slack API has a https://api.slack.com/methods/users.list method which will return all the members of the status-im Slack organization. That will return a complete list of users (which currently stands at around 2MB, so we should use pagination).

Configuration to scale this bot across Status

This bot could be the foundation for organisation-wide tooling across the various Status repositories.

Tooling concerns the bot can help with:

• process automation (github labels, projects etc.)
• deployment
• monitoring
• various kinds of notifications

An important point of this is to embrace that different repos/teams might have different needs. Thus a somewhat "overarching" configuration scheme could be implemented. This would allow teams to customize the various bot features based on their own needs (editing a config.yml in this repo could be sufficient). Once a user figured out how to configure stuff the conventions would remain the same for each bot making configuration a breeze. (Adequate documentation should be provided.)

Popular features would see increased development & feedback while not so commonly used features might slowly be faded out.

An added benefit of this is that we can move forward with more experimental features (like a nudge to welcome contributors personally which I suggested to @PombeirP) while providing a clear customization path.

---

What follows below is some initial idea for how this could be implemented with minimal effort.

A very first step towards this could be that each bot has a unique identifier $botName and a yaml file at the root of this repository could be filled with something like this:

$botName:
  globallyEnabled: true

status-im/status-react:
  $botName:
    disabled true
status-im/open-bounty:
  $botName:
    - welcome-style: 'contributor-notification'

A simple config.js could then expose this API:

Config.enabled(botName)
Config.enabled(botName, repoName)

Note: the repoName based approach could cause some issues with renaming.

What do you think @PombeirP? I think this could be a really nice way to allow teams to extend the bot stuff while keeping things useful for everybody.

Given that this is pretty straightforward to implement I might just whip up a basic PR extending config.js and writing some documentation.

---

EDIT

I saw that we use probot-config in some places but it spreads bot configuration across repositories and introduces some weird DSL (e.g. _extends) that will break sooner or later.

Colocating configuration in the bot repo itself will allow people working on the bot to refactor without much coordination effort.

Have bot trigger automated tests build on Jenkins when PR is moved to IN TEST on project board

When a PR is moved to the IN TEST column and the build has passed successfully, then the bot should kick a PR test automation build in Jenkins (retrying periodically if the PR build is still running). It needs to provide two parameters (apk and pr_id).

Add script to give tips to kudos recipients

Have bot ping relevant developers regarding issues pending bounty approval

User Story

As a developer, I want to know when a new issue from my team is pending bounty approval, so that I can react more quickly without having to search for information.

Description

A better way to address this issue would be to leverage the new GitHub bot (written on the Probot framework) to listen to issue label changes, specifically when an issue is labeled with bounty-awaiting-approval, and then act on it by notifying the team members from the respective repository on Slack (either through a DM or a mention on a given room).

Since GitHub user names don't necessarily match Slack user names, the bot would need to read the mapping from a file (JSON or YAML). The same list from #4 should be reused. Only the implementation is required, the file itself will be filled later on by the team.

Documentation for the bot is available in the wiki.

Implement script to automate management of PR checklists

User Story

As a developer, I want to be sure that I follow the guidelines for contributions. An easy way to achieve that is for the bot to automatically post a checklist to any new PRs, and only allow merging the PR when the checkboxes in the checklist are all checked.

Description

Whenever a PR is created, Probot could look at the target repo and try to retrieve for a checklist definition (could be located in the .github/github-bot.yml config file). If one is found, it would post a comment to the PR with the checklist.
It would then need to monitor PR comment changes to update the PR state according to the state of the checklist (similar to what the WIP plugin is already doing).

Documentation for the bot is available in the wiki, and several existing scripts already exist in the repo that can serve as reference/inspiration.

Have bot move SOB issue to 'PR submitted' column if a PR references a SOB issue

User Story

As a Status Open Bounty maintainer, I don't want to have to manually manage checking when a pull request is submitted against a SOB issue that is currently in the Bounty Open or Stale Bounties columns and moving the project card that represents the issue to the PR Submitted column.

Description

A better way to address this issue would be to leverage the new GitHub bot (written on the Probot framework) to:

• listen to issue referenced events;
• check if the issue that is referenced is assigned the bounty label;
• check if there is a project card in the Status SOB Swarm organizational project board in the Bounty Open or Stale Bounties.

and if the conditions are met, act on it by:

• moving the project card that represents the bounty issue to the PR Submitted column.
• sending a notification to Slack, saying something like:

  A new PR was submitted for a SOB issue\n${pr.html_url}

Documentation for the bot is available in the wiki.

Acceptance criteria

• The project board, column and label names are fetched from the .github/ github-bot.yml config file;
• The script follows the style of existing scripts and the prescriptions made in the documentation;
• There is a new environment variable (e.g. DRY_RUN_SOB_PR_SUBMITTED) that allows toggling a dry-run for this particular script for easier testing.
• The bot has been tested locally to ensure it works as expected.

Prevent statusbot from moving issues from Contributor back to To Test queue

Description

When a Github issue is put in Contributor, the statusbot is moving it back to To Test queue. This triggers autotests rerun which is useless.

Expected behavior

Issue stays in Contributor. The bot should look at the Tested - Issues label and not move the issue to To Test if that label is present.

Actual behavior

Issue is moved from Contributor to To Test queue:

PR example