astrofrog / batchpr

Package in need of a better name to automate opening pull requests :robot:

About

The aim of this package is to provide an easy way to do automated issues or pull requests to a selection of repositories and make specific changes to them. This is currently functional but could be significantly improved, so contributions are welcome!

Installation

To install the latest development version with pip:

pip install git+https://github.com/astrofrog/batchpr.git@master

To install from source:

git clone https://github.com/astrofrog/batchpr.git
cd batchpr
pip install .

Automated Pull Requests

batchpr requires the following packages:

• git
• pygithub
• requests
• termcolor

To use this, you should write a Python script in which you import and subclass the Updater class, and define the following methods and properties:

from batchpr import Updater

class MyUpdater(Updater):

    def process_repo(self):
        # This method should contain any code that you want to run inside
        # the repository to make the changes/updates. You can assume that
        # the current working directory is the repository being processed.
        # This method should return False if it was not able to make the
        # changes, and True if it was. This method should call self.add
        # to git add any files that have changed, but should not commit.

    @property
    def commit_message(self):
        # The commit message to use when making the changes

    @property
    def pull_request_title(self):
        # The title of the pull request

    @property
    def pull_request_body(self)
        # The main body/description of the pull request

    @property
    def branch_name(self):
        # The name of the branch to use

Once you have defined your updater class, you can run it with:

helper = MyUpdater(token=GITHUB_TOKEN)
helper.run('username/repo')

Where GITHUB_TOKEN is a personal access token for GitHub. If you want to customize the author of the commit, you can do this with:

helper = MyUpdater(token=GITHUB_TOKEN, author_name='Foo', author_email='[email protected]')
helper.run('username/repo')

The run method can take a single repository or a list of repositories. If dry_run=True option is passed in to the Updater subclass, pull requests will not be opened but other prior steps (i.e., forking, branching, and committing the changes) are executed. If verbose=True option is passed into the Updater subclass, you will always see output of the commands, not just when a command fails.

When in the Updater class, the following methods are available:

• self.run_command(command): should be used for running shell commands (e.g., git)
• self.warn(message): should be used for warning messages
• self.error(message): should be used for error messages
• self.add(filename): should be used to add files that have changed or are new
• self.copy(filename1, filename2): can be used to copy files

Full Example

The following shows an example of an updater that adds a few sentences from the zen of Python to the README file if present:

import os
from batchpr import Updater

DESCRIPTION = """
This is an automated update made by the ``batchpr`` tool :robot: - feel free to
close if it doesn't look good! You can report issues to @astrofrog.
"""

ADDITION = """
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
"""

class ExampleUpdater(Updater):

    def process_repo(self):

        if os.path.exists('README.md'):
            with open('README.md', 'a') as f:
                f.write(os.linesep + ADDITION)
            self.add('README.md')
            return True
        else:
            return False

    @property
    def commit_message(self):
        return "MNT: Add important text to README.rst"

    @property
    def branch_name(self):
        return 'readme-zen'

    @property
    def pull_request_title(self):
        return self.commit_message

    @property
    def pull_request_body(self):
        return DESCRIPTION.strip()

helper = ExampleUpdater(token=GITHUB_TOKEN)
helper.run('username/repo')

Automated Issues

Opening automated issues is simpler than pull requests as you do not need to create forks or modify any files. You use the same GitHub token as above.

Full Example

The following shows an example of opening a simple issue.

from batchpr import IssueUpdater

ISSUE_TITLE = 'Please fix this and that'

ISSUE_BODY = """
I found this and that with the package. Please fix them.

*This is an automated issue. If this is opened in error, please let me know!*
"""

helper = IssueUpdater(GITHUB_TOKEN, ISSUE_TITLE, ISSUE_BODY)
helper.run('username/repo')

batchpr's Issues

Rename repo

As suggested by @astrofrog -- spamalot

Be more flexible with github authorization

Would be nice to use the same authorizations as in astropy-procedures, e.g. use username and password rather than the token.

Does not work if I also own the affected repo to be updated

Let's say the person running this has GitHub account under username and wants to do some mass PR and username/somerepo happens to be part of the list, the person will encounter some error like this:

Processing repository: username/somerepo
  > Ensuring repository exists
  > Ensuring fork exists (and creating if not)
  > Working in /tmp/...
  > git clone --depth 1 [email protected]:username/somerepo.git
  > git checkout origin/branchname
    error: pathspec 'origin/branchname' did not match any file(s) known to git
  > git remote add upstream [email protected]:username/somerepo.git
  > git fetch upstream
  > git checkout upstream/master
  > git checkout -b branchname
  > git add some_file.txt
  > git commit -m "Some commit"
    On branch branchname
...
Exception: Command 'git commit -m "Some commit"' failed

The return code is 1.

Make a release

Possible release venues:

• Annotated GitHub tag
• PyPI
• Maybe even some conda channel (astropy? conda-forge?)

This would be nice to have before astropy/astropy-tools#106 .

License: Why 2-clause and not 3?

Why BSD 2-clause instead of 3-clause? Just curious.

Enable batch issues?

I can't find the API to batch open issues and not PRs.

DOC: Add requirements

Like pygithub and termcolor.

Is there a way to run command without shell=True?

shell=True should be avoided if possible?

ENH: API for batch call

Would be nice for a helper function that takes the updater class (all nicely set up) to apply run method on a list of repos. This list should be chunked. Within each chunk, maybe time.sleep(0.5) between calls. Between chunk, maybe time.sleep(10) or something, to hopefully not trigger GitHub spam killer. This function will return a list of repos that fail for whatever reason, for manual follow-up (like a second run).

Remove PY2 compat

That would simplify things a little and take advantage of new features in Python 3 only.

Add RTD?

So that the API docstrings that I typed painstakingly in #17 can be rendered in pretty HTML... 😅

Add continuous integration and tests

I think this will have to rely on mock to some extent, but would be good to catch basic issues like #5. I think just Travis or CircleCI would be sufficient.

Does not work when I have existing fork with different name

When this is run and the upstream repo name is different from my fork's repo name, say, upstream renamed it since I last forked it, I get this error: An error occurred when cloning fork - skipping repository

The workaround is to rename the fork's repo name to be the same as upstream.

Is this repo meant to be installable?

As reported by @bsipocz , setup.py does not appear to be functional. If this is not meant as an installable repo, better to remove that file?

batchpr/setup.py

Lines 10 to 11 in 5fbce1d

	url="https://github.com/astrofrog/ancillaries",
	name="ancillaries",

Why git push astrobot?

What is this trying to do? Especially if a different token from astrobot's is used here?

batchpr/batchpr/updater.py

Line 166 in 5fbce1d

self.run_command('git push https://astrobot:{0}@github.com/{1} {2}'.format(self.token, self.fork.full_name, self.branch_name))

Move most of the setup.py config to setup.cfg

See e.g. astropy/sphinx-automodapi#62

Where is --verbose passed in?

        if ('--verbose' in sys.argv or p.returncode != 0) and output:
            print(indent(output, ' ' * 4))

How is --verbose passed into sys.argv here in run_command()? Is it automagically extracted from command?