GithubHelp home page GithubHelp logo

rust-lang / highfive Goto Github PK

View Code? Open in Web Editor NEW
183.0 20.0 127.0 716 KB

Github hooks to provide an encouraging atmosphere for new contributors

License: Apache License 2.0

Python 97.83% JavaScript 1.64% Dockerfile 0.54%

highfive's Introduction

Highfive

This repository is no longer running on rust-lang infrastructure; instead the relevant code lives in triagebot. This repository has been archived to retain it for the future.


GitHub hooks to provide an encouraging atmosphere for new contributors. Highfive assigns pull requests to users based on rules in configuration files. You can see Highfive in action in several Rust repositories. See the rust-lang/rust pull requests, for example.

This project drives the @rust-highfive bot and was originally a fork of servo/highfive, used by Servo and Servo's @highfive bot. For more history see the comments in #35.

Table of Contents

  1. Installation
  2. Testing
  3. Adding a Project
  4. Enabling a Repository
  5. Local Development
  6. License

Installation

To install highfive, you just need to execute the setup.py script or use pip directly. Both commands have to be executed from the directory where setup.py is located.

$ python setup.py install

or

$ pip install . # the dot is important ;)

Testing

Before running tests, make sure the test-requirements are installed by running the following command:

$ pip install -r test-requirements.txt

Once the dependencies are installed, you can run all tests by executing:

$ pytest

Tests are labeled as "unit", "integration", and "hermetic". All unit tests are hermetic, but only some integration tests are hermetic. A non-hermetic test makes network requests. To run only hermetic tests do:

$ pytest -m hermetic

Hermetic tests are run in PR builds. All tests are run in daily cron builds.

Adding a Project

To make rust-highfive interact with a new repo, add a configuration file in highfive/configs, with a filename of the form reponame.json. The file should look like:

{
    "groups":{
        "all": ["@username", "@otheruser"],
        "subteamname": ["@subteammember", "@username"]
    },
    "dirs":{
        "dirname":  ["subteamname", "@anotheruser"]
    },
    "contributing": "http://project.tld/contributing_guide.html",
    "expected_branch": "develop",
    "mentions": {
        "src/doc": {
            "message": "Documentation was changed.",
            "reviewers": ["@DocumentationReviewPerson"]
        },
        "test.rs": {
            "message": "Some changes occurred in a test file.",
            "reviewers": ["@TestReviewPerson"]
        }
    },
    "new_pr_labels": ["S-waiting-for-review"]
}

The groups section allows you to alias lists of usernames. You should specify at least one user in the group "all". Other keys are optional.

In the dirs section, you map directories of the repository to users or groups who're eligible to review PRs. This section can be left blank.

contributing specifies the contribution guide link in the message which welcomes new contributors to the repository. If contributing is not present, the contributing chapter of the rustc-dev-guide will be linked instead.

If PRs should be filed against a branch other than master, specify the correct destination in the expected_branch field. If expected_branch is left out, highfive will assume that PRs should be filed against master. The bot posts a warning on any PR that targets an unexpected branch.

The mentions section is used by Highfive when new PRs are created. If a PR diff modifies files in the paths configured in the mentions section, a comment is made with the given message that mentions the specified users. Mentions paths have either one or two behaviors.

  • Every path in a diff is checked whether it begins with a path in the mentions list. If there is a match, the mention comment is made.
  • If a path in the diff ends with a mentions path ending in .rs, the mention is a match, and a comment is made.

new_pr_labels contains a list of labels to apply to each new PR. If it's left out or empty, no new labels will be applied.

Enabling a Repository

Once the PR updating the repository configuration has been merged, run the update-webhooks.py script at the root of this repository:

$ python3 update-webhooks.py

The script requires the GITHUB_TOKEN environment variable to be set to a valid GitHub API token, and it will make sure the configuration of all the repositories you have admin access to is correct.

Local Development

You can run Highfive on your machine and configure a repository to use your local instance. Here is one approach for running a local server:

  • Create a virtualenv to isolate the Python environment from the rest of the system, and install highfive in it:
    $ virtualenv -p python2 env
    $ env/bin/pip install -e .
    
  • Run the highfive command to start a development server on port 8000:
    $ env/bin/highfive
    
  • Your Highfive instance will need to be reachable from outside of your machine. One way to do this is to use ngrok to get a temporary domain name that proxies to your Highfive instance. Additionally, you will be able to use ngrok's inspector to easily examine and replay the requests.
  • Set up the webhook by following the instructions in Enabling a Repo, substituting your local Highfive IP address or domain name and port number (if necessary).
  • Obtain an OAuth token. In the account you are creating the token in, go to https://github.com/settings/tokens. Grant access to the repository scope.
  • Put the authorization information obtained in the previous step into a file named .env in the top of the repository (i.e., the directory containing this file). Here is a template of what it should look like:
    HIGHFIVE_GITHUB_TOKEN=your-token
    
    Do not check in this file or commit your OAuth token to a repository in any other way. It is a secret.

Here are some details to be aware of:

  • For Highfive to know how to select reviewers for your repository, you need a configuration file in highfive/configs.
  • Highfive ignores comments from the integration user near the top of new_commment in highfive/newpr.py.

Docker

Alternatively, you can build a Docker image that runs Highfive.

$ docker build -t highfive .

To run a container, you must mount a config file. Assuming you are launching a container from a directory containing a config file, you can do the following.

$ docker run -d --rm --name highfive -p 8000:80 -e HIGHFIVE_GITHUB_TOKEN=token -e HIGHFIVE_WEBHOOK_SECRET=secret highfive

At this point, Highfive is accessible at http://localhost:8080.

License

Highfive is licensed under the terms of both the MIT License and the Apache License (Version 2.0).

See LICENSE-APACHE and LICENSE-MIT for details.

highfive's People

Contributors

alexcrichton avatar davidalber avatar ehuss avatar flaper87 avatar flip1995 avatar guillaumegomez avatar huonw avatar japaric avatar jdm avatar jhwgh1968 avatar johntitor avatar jyn514 avatar kodraus avatar lcnr avatar leseulartichaut avatar m-ou-se avatar manishearth avatar mark-simulacrum avatar nikomatsakis avatar nrc avatar oli-obk avatar petrochenkov avatar pietroalbini avatar pnkfelix avatar steveklabnik avatar therealprof avatar thomcc avatar xampprocky avatar xfrednet avatar yaahc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

highfive's Issues

consider detaching this fork from upstream

there are downsides to having a live fork also be a fork within GitHub because, for instance, it affects your ability to use GitHub search on the code:

https://github.com/rust-lang-nursery/highfive/search?q=reviewer&type=Code

however, it's possible to request that the repo be made a standalone repository:

To detach the fork and turn it into a standalone repository on GitHub, contact GitHub Support. If the fork has forks of its own, let support know if the forks should move with your repository into a new network or remain in the current network. For more information, see "About forks."

(from https://help.github.com/articles/why-are-my-contributions-not-showing-up-on-my-profile/#commit-was-made-in-a-fork)

I was able to follow this process to detach a fork with the following message:

Hi GitHub Support!

As detailed in https://help.github.com/articles/why-are-my-contributions-not-showing-up-on-my-profile/#commit-was-made-in-a-fork, please could you detach https://github.com/sbt/sbt-launcher-package from https://github.com/paulp/sbt-extras, turning it into a standalone repository. Note I co-maintain both, but they're distinct entities at this point.

Also, any fork of sbt/sbt-launcher-package should move with sbt/sbt-launcher-package into a new network, rather than remain in the current network.

Thank you,
Dale

Relabel PR on r?

PRs without S-waiting-on-review can easily fall out of reviewer's task list, and PRs marked as S-waiting-on-author is currently relabeled by hand.

We should add this functionality to highfive: when r? is used, unlabel S-waiting-on-author and relabel S-waiting-on-review.

Recommend better people

Currently we just use a random member of the core team to ping. I would like to have a directory based system: look at the PR and find the directory with the most changes, keep a file mapping directory to people, select a person at random from the most changed directory's pool.

Since the mapping from directories -> people is tied to the Rust repo, it might be better to keep it in-tree. It is easier to have it here, for now.

Fix tests

If I run the tests following the instructions in README.md, I get errors. First I get an indentation error at line 76 in test_cr.py. I tried fixing that by removing the initial spaces on that line. Then I get the error:

  File "/home/ncameron/highfive/test_cr.py", line 71, in <module>
    df = open('diff.diff')
IOError: [Errno 2] No such file or directory: 'diff.diff'

I don't see diff.diff anywhere, so I'm not sure how to fix this.

cc @flaper87

Allow more users to r? PRs

Currently highfive will only respond to r? if it is in a comment by the author of the PR. We should expand this so that more people can do it. At least the currently assigned reviewer, perhaps also the 'all' group of reviewers, some union of reviewer groups, all possible GitHub reviewers (we'll probably want to store this somewhere rather than sending an API request every time), or an explicit group of people.

Contributor gets welcomed twice if existing commits are to non-default branch

Is this correct behavior? It seems incorrect, since landing code in any branch of a repo implies knowing how to contribute.

Example:

rust-www was formerly on GitHub pages, so its default branch is currently gh-pages. The graphs show contributions only to gh-pages. I got my first PR merged into master, and highfive welcomed me correctly. However, I was welcomed again on my second PR despite the fact that the first one landed.

ping about forgotten reviews

highfive should ping reviewers who forget about a review, probably by leaving a GitHub comment and by an irc ping.

I'm not sure exactly how this should work. We could run a cron job, say every hour and do an API request to get all open PRs with assignees, then find any with no comment from the assignee in the last 24 hours. We would also have to make an API request for comments on each commit of the PR to check for comments there. We would have to check for an r+ at any time, not just in the last 24 hours in order to avoid the situation where a patch is reviewed and we're waiting for bors. We should also not ping in the case where an older PR is r+'ed and a newer one has r+ or r=foo comment from someone other than the assignee. But we should ping if there is no comment at all.

Have I missed any requirements there?

Warn if a PR changes Cargo.lock

In rust-lang/rust we will not automatically merge diff from Cargo.lock (see rust-lang/rust#46539 for rationale). This means all PRs modifying Cargo.lock are mutually exclusive with each other, and when one such PR is merged, all other PRs will need to be rebased.

This may bother reviewers and authors when the PR needs to be rebased and r+'ed several times. It is also pretty confusing to new contributors because a rebase often completes without any conflicts locally.

Thus, I think it is the best for rust-highfive to foreshadow such potential extra work to both reviewers and authors.

⚠️ Warning ⚠️

This PR modified Cargo.lock. We will not automatically merge updates to Cargo.lock, since it often leaves the file in a broken state. One unfortunate result is that this often leads to merge conflicts.

You may need to rebase this PR several times after it has been approved. We are sorry for any inconvenience which may arise.

(Note to reviewers: Please also prepare to r+ several times. Consider giving it a p=1 to minimize queuing time and reduce chance of conflict.)

Do something with the contributor list

We currently maintain a list of contributors per repo which supplements the list we pull from GitHub. We could be more efficient (albeit more stateful) by updating this list with contributors as we go along. We could either do this manually, or use a DB instead. See some discussion in #38

Change core team to act as a fallback

Currently the people on core are eligible for all kinds of PRs. But really everybody has their specialty, so it'd be best to use core just as a fallback for PR assignment, and instead also add people to the teams where they have the most knowledge.

Move to nursery/infra team oversight

The infra team is interested in helping maintain this piece of infrastructure, starting with introducing some debugging to figure out why some PRs are continuing to miss assignments. @aidanhs, in particular, has volunteered to help.

Are you open to moving this repo to the nursery?

We should explore the relationship to Servo's fork before making any move.

reset S-waiting-on-author tag when new commits are pushed

I would like to be able to aggressively set the status of PRs to S-waiting-on-author and then filter those out of my display. For me to feel comfortable doing that, though, I need to be sure that I won't miss updates from them -- therefore, I would like highfive to reset the status back to S-waiting-on-review as soon as a new commit or comment is made. (Possibly just a new comment by the PR author? But probably any comment is fine -- or any comment not by bors.)

Latest Highfive failing

PR #136 seems to have broken Highfive on the server. The only relevant info in the logs is:

End of script output before headers: newpr.py

The web server responds with a 500 error

cc @davidalber

Detect r? without @

We currently look for r? @foo in messages, but people often write r? foo. We should check for this and assign a reviewer for it (ignoring r? without a reviewer which is also common).

This will get some false positives, e.g., r? anyone, but I think that is OK, the attempt to set the assignee on GitHub will silently fail. In that case, we should then set a random reviewer (we should close this loophole in any case, I think).

Warn if PR is against non-standard branch

For most repos, there's one branch that most PRs are expected to be filed against.

17:11:54 <@brson> this patch was submitted against 'stable' https://github.com/rust-lang/rust/pull/25739 but thankfully bors merged it into master
17:12:22 <@acrichto> brson: whoa
17:12:45 <@acrichto> brson: I wonder if rust-highfive could give a warning if it's not targeting master

The API contains information on which branch a PR was submitted against, in the form of "base"["label"]. For instance, grep rust-lang:stable in the result of curl -i https://api.github.com/repos/rust-lang/rust/pulls/25739.

I believe the architecture for this would be to add an expected_branch field in each repo's config, then add the check to highfive and default to assuming the expected branch is <reponame>:master if none is specified?

If this is something you'd like to get added, I can implement it.

Give more constructive welcoming to newcomers

Currently the bot says hello and tells you that someone will be on the way:

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @huonw (or someone else) soon.

I think this is great, but it would be even better if it could elaborate on our standard process, and give more detailed advice for how to handle the process.

In particular, I've been burned a lot by people knowing that they should rebase changes for merge, but doing it before someone has given the final sign-off. This makes it really hard to tell if they actually have addressed all your concerns, and how.

I suggest it state the following, or something similar:

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @huonw (or someone else) soon.

If the reviewer has any concerns, you should address them by either making a change, or explaining why you think it's the right thing to do. (Or enter into a more detailed design discussion as necessary.)

If any changes are deemed necessary, please add them as extra commits to the current pull-request. This ensures that the reviewer can see what has changed since they last reviewed the code. The way Github handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Once the reviewer believes that all of their concerns have been addressed, they may ask you to rebase your commits to produce a more useful history for the changelog (for simple changes, this should just be a single commit). Do not do this until you are asked. Doing this too soon will make the reviewer's job much more difficult. Note that you may also be asked to rebase your work if new commits that land during the review process conflict with your changes.

When the change is approved and the history is where it should be, the reviewer will make a comment like r+ or r=huonw on the last commit to signal to our automated system that this commit is approved for inclusion in the project. Your PR will then be inserted into the Bors queue for testing. This queue is based on age; older PRs get priority. Once it reaches the top, your PR will be fully tested on top of the current master. If everything passes, it will be merged! If there is an error, your change will be rejected by the system. At this point you should investigate the logs the bot will post on the PR for what went wrong. It's entirely possible that the failure was spurious. The build might be unstable, or the process might have been interrupted by a system failure. If this is the case, a reviewer can prompt the system to retry.

Set the assignee for a PR

As well as pinging a person with a mention, highfive should set the assignee of the PR to that person. This will require the highfive bot (rust-highfive) to have some extra permissions on the Rust repo.

Consider democratizing access to production logs

I am interested in investigating instances of unexpected Highfive behavior. For example, sometimes, but not consistently, Highfive sends the welcome message to existing contributors. Another example: in rust-lang/rust#49849, Highfive did not take any action at all. It's difficult to know what is going on there without access to the logs. Looking at Highfive's code I can speculate that maybe the diff request failed, but there's several other possibilities.

The first question is: are the organization members comfortable granting access to the logs to a non-member?

If the answer is yes, we should find a reasonable, low-cost-to-@nrc way to accomplish it. A lot of options come to mind, but they generally branch on whether we want to provide access to the host Highfive runs on or whether we want to ship the logs to somewhere else.

  1. Provide access to host running Highfive.
    1. Continue running Highfive where it is now?
    2. Move Highfive to another location?
  2. Ship the logs elsewhere.
    1. Add application logic to handle this?
    2. Use external process to ship logs to somewhere?

I'm happy to talk more about the possibilities, if the answer to the first question is yes.

Respond with a reviewer to all PRs

Highfive currently just responds to PRs from new contributors (it checks for unsafe code in all PRs, but that is separate to this issue). I propose that if a PR has r? @some_user, highfive change the assignee to some_user (note this will silently fail if some_user is not a valid assignee). If there is no proposed reviewer, Highfive will select one, ping them, and set the assignee. If the PR is from a new contributor, Highfive will use the welcome message.

Give some changes back to Servo's highfive

This project was originally a fork of jdm/highfive, used for Servo. I removed a few things, changed a few things, and since then we've added a lot. It would be nice if we could push some of the additions back to Servo land. Perhaps the easiest was to do that is to expand this version to cover the stuff removed or changed and then persuade them to switch (or just push everything back to jdm's original fork). I fear taking our changes and adding them to the original would be a tricky job.

cc @jdm

Work on other repos in the rust-lang org

cc @huonw

I did the first part of this - making choosing a reviewer generic over the repo.

Still todo:

  • make rust-highfive an admin or owner or whatever on the repos we want to cover
  • add the GitHub hooks to these repos
  • change the link to contributing.md in the welcome message (maybe, perhaps the rust one is fine for all repos)
  • create repo.json files for each repo with the list of people to use as reviewers
  • test that it actually works!

Remove unsafe code warning

I'm not actually sure I've ever gotten anything meaningful from this, and it's mostly just another email to have to sift through, perhaps we could just remove this for now?

Automatically label issues

It would be nice if highfive could automatically attach labels to issues that clearly mention what they’re related to in their title. So an issue titled something like ‘rustdoc: foobars create broken links’ would automatically add an ‘A-rustdoc’ label to the issue.

I’m not sure if this is within the scope of highfive, but it’s still a nice thing that could be automated.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.