guyzmo / git-repo Goto Github PK

implement:

git-repo <target> request <user/repo> create [<branch>] <title> --message=<description>

which makes a new pull request with current branch.

As gitlab has been fixed with a patch to support requests sessions, remove the monkey patching hack in the gitlab tests.

to avoid relying on user+password in the gitconfig (and make it more easily revokable just in case…)

This project would be quite useful on Windows machines, and I can't imagine any of the code is inherently tied to OS/X or Linux. If the dependency on https://github.com/amoffat/sh were eliminated it would be possible to install this on Windows as well.

add a command to upload SSH keys to the service, something like:

git <target> ssh push <ssh-public-key-file>
git <target> ssh ls
git <target> ssh delete <ssh-public-key-id>

the whole point being to make it easy to generate a new key (or even rotate keys!) or revoke an old key with the CLI.

As bitbucket-api is proven to be unmaintained, and pybitbucket is still active, and because pybitbucket supports snippets #13 , OAuth2 #14 it's a good idea to switch to it.

I would really like functionality to get a list of all git repos under a given account, preferably in a format easy to parse in scripts and with multiple formatting options (i.e. format as git ssh urls, format as https urls, append default branch or not).

I'm currently using a hand-rolled tool (chegit) to keep my local changes synced to Gitlab, but I don't have tooling for multiple remotes yet (your 'all' remote would solve this), and I don't have a way of getting a list of remote repositories that I might want to keep a local copy of.

Is this functionality already included and did I overlook it in the help page, and if not, would you consider implementing it?

University of Minnesota uses a local GitHub appliance instead of public GitHub. Would be fantastic to get this working against it.

I have 2FA enabled on my GitHub account. When I ran git repo config, this happened:

Do you want to configure the github service?
    [Yn]> y
Please enter your credentials to connect to the service:
username> tcby45
password>
Fatal error: 401 Must specify two-factor authentication OTP code.

Could you please either include a 2FA check on the account (then a prompt for the code), or possibly ask whether to authenticate by personal access token or user/pass?

When creating a git repo for a group at Gitlab, I would want to call git-repo with the arguments lab create <groupname>/<reponame>, but this results in the repo being created for my current user, and not for the group.

Wait for integration of both:

• gitpython-developers/GitPython#445
• gitpython-developers/GitPython#446

to avoid both monkey patching or targeting a development version of gitpython in setup.py.

to be merged and released as new version (either 2.0.5 or 2.0.6 according to the labels).

For git services that supports it (like gitlab or gerrit), implement a simple mechanism to:

• list/fetch comments and votes on current request
• add a comment and vote on it

when doing commands that clones a repository, support a repository_name parameter that mimics git clone <URL> <PATH> behaviour.

create a command to autoconfigure git-repo, which would be:

git repo configure (github|gitlab|bitbucket)

which would then run an authentication loop, to query auth tokens, asking for user/pass for both github and gitlab, and record login:pass for bitbucket (until oauth config is implemented).
Eventually, it could ask for custom gitlab URLs as well.

and it would then populate ~/.gitconfig with:

alias.hub = repo hub
alias.lab = repo lab
alias.bb = repo bb

Here's a mockup of how the interactions could be:

% git repo configure
• Do you want to configure github access? 
[Yn]> Y
Enter your username: 
'guyzmo'> foobar
Enter your password:
>
Please wait while getting authenticated…
Done, cheers! 🍻
• Do you want to configure gitlab access?
[Yn]> Y
Enter your username: 
'$USER'> barfoo
Enter your password:
>
Please wait while getting authenticated…
Done, cheers! 🍻
• Do you want to configure bitbucket access?
[Yn]> Y
Enter your username: 
'$USER'> fubar
Enter your password:
>
Done, cheers! 🍻
• Do you want to add a custom gitlab access?
[yN]> Y
Enter the URL of the gitlab:
> https://my_gitlab.my_domain.tld
Enter the short name of the service:
> mine
Enter your username:
'$USER'> ford.prefect
Enter your password:
>
Please wait while getting authenticated…
Done, cheers! 🍻
• Do you want to add a custom gitlab access?
[yN]> n
Thank you, here's your configuration:
Service   | Name | Username     | Password | Token    | URI
--------- | ---- | ------------ | -------- | -------- | -----
Github    | hub  | foobar       | *        | Acquired | *
Gitlab    | lab  | barfoo       | *        | Acquired | *
Bitbucket | bb   | fubar        | Acquired | *        | *
Gitlab    | mine | ford.prefect | *        | Acquired | https://my_gitlab.my_domain.tld

Getting the above error when running git hub create ndarilek/projectname. Is there some separate authentication step I need to perform? I ran quickstart and expected that would do it.

Thanks.

If the repo name can't be detected _guess_repo_slug is checking to see if .git is anywhere in the repo name instead of checking at the end of the string. This then truncates the repo name by removing the last 4 characters.

Example [email protected]:project/git_repo becomes [email protected]:project/git_

I'm going to submit a pull request with the fix.

I failed to authenticate with a private key.

This is what I have stored in my .gitconfig

[gitrepo "gitlabxxx"]
    alias = xxx
    type = gitlab
    token = ~/.ssh/id_rsa_gitlabxxx
    fqdn = gitlab.xxx.fr

I also defined the alias xxx = repo xxx.

The following instruction works:

$ git clone [email protected]:yyyy/myProject.git

And this is what I tried to do which failed

$ git xxx clone yyyy/myProject
Fatal error: Can't connect to GitLab server ([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645))

I also tried to create a new github repo to test and it failed with the following error

$ git hub create yyyy/test
Fatal error: 'GithubService' object has no attribute 'username'

This is what I have defined in .gitconfig for github

[gitrepo "github"]
    alias = hub
    token = ~/.ssh/id_rsa_github

In my ~/.ssh/config file I have

Host github.com
  User git
  HostName github.com
  IdentityFile ~/.ssh/id_rsa_github
  IdentitiesOnly yes

I didn't need that for the gitlab server.

I keep ~/.gitconfig stored in source control. I can't do so with git-repo storing the github auth token in there...

Thanks.

Thanks for the great tool!

Re-running git repo config has the potential to leak credentials since it seems to randomise which provider it prompts for input for first.

If a lazy user (such as myself) isn't careful and doesn't re-read the prompts on each run due to expecting the order to be the same (as is usually the case for wizards) then it's possible to input the credentials for a provider other than the one you're expecting. The result is that secrets are potentially submitted to the wrong provider which is probably a security concern.

The current command dispatch code has been designed to avoid an over-complex main function. But it was after coding it that I discovered that the author of docopt implemented a similar framework he called docopt_dispatch.

That framework might need some adjustments for custom features built in the current command dispatch code, but it's worth using an existing and common framework than reinventing the wheel.

I followed the installation instructions in the Readme file.
After installation, git repo config is not recognized by git as a valid git command. Same problem with git hub.

There is apparently a missing installation step in the documentation.

Hey,
I would like to install 'git-repo' but I prefer to install in my userspace with pip install --user,
here is the error I get

pip install --user git-repo                                                                                                                                                                       1 
Collecting git-repo
  Downloading git-repo-1.4.tar.gz
    Complete output from command python setup.py egg_info:

    Installed /tmp/pip-build-4e20i_j2/git-repo/.eggs/setuptools_markdown-0.2-py3.4.egg
    Searching for pypandoc
    Reading https://pypi.python.org/simple/pypandoc/
    Best match: pypandoc 1.1.3
    Downloading https://pypi.python.org/packages/source/p/pypandoc/pypandoc-1.1.3.zip#md5=771f376bf9c936a90159cd94235998c2
    Processing pypandoc-1.1.3.zip
    Writing /tmp/easy_install-tobd_snv/pypandoc-1.1.3/setup.cfg
    Running pypandoc-1.1.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-tobd_snv/pypandoc-1.1.3/egg-dist-tmp-g5fpg11a
    zip_safe flag not set; analyzing archive contents...
    pypandoc.__pycache__.__init__.cpython-34: module references __file__
    Patching wheel building...
    creating /tmp/pip-build-4e20i_j2/git-repo/.eggs/pypandoc-1.1.3-py3.4.egg
    Extracting pypandoc-1.1.3-py3.4.egg to /tmp/pip-build-4e20i_j2/git-repo/.eggs

    Installed /tmp/pip-build-4e20i_j2/git-repo/.eggs/pypandoc-1.1.3-py3.4.egg
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "/tmp/pip-build-4e20i_j2/git-repo/setup.py", line 38, in <module>
        zip_safe=False
      File "/usr/lib64/python3.4/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib64/python3.4/site-packages/setuptools/dist.py", line 272, in __init__
        _Distribution.__init__(self,attrs)
      File "/usr/lib64/python3.4/distutils/dist.py", line 280, in __init__
        self.finalize_options()
      File "/usr/lib64/python3.4/site-packages/setuptools/dist.py", line 327, in finalize_options
        ep.load()(self, ep.name, value)
      File "/tmp/pip-build-4e20i_j2/git-repo/.eggs/setuptools_markdown-0.2-py3.4.egg/setuptools_markdown.py", line 22, in long_description_markdown_filename
        output = pypandoc.convert(markdown_filename, 'rst')
      File "/tmp/pip-build-4e20i_j2/git-repo/.eggs/pypandoc-1.1.3-py3.4.egg/pypandoc/__init__.py", line 50, in convert
        outputfile=outputfile, filters=filters)
      File "/tmp/pip-build-4e20i_j2/git-repo/.eggs/pypandoc-1.1.3-py3.4.egg/pypandoc/__init__.py", line 68, in _convert
        raise RuntimeError('Missing format!')
    RuntimeError: Missing format!

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-4e20i_j2/git-repo

I think it should work ( I do the same for other user-managed packages ), so maybe there is a problem in the pypy recipe ?

My usual workflow when creating a pull request is to assign it to a collaborator. Right now, using git hub request create doesn't save me much time because I still have to open up github.com to assign the PR. It would be great if I could pass an -a <username> parameter to git hub request create to automatically assign the PR when it's created.

Hi,

I'm making an archlinux PKGBUILD for git-repo. However the requirements.txt claim for certain version of those dependencies.

Is the version in requirements.txt mandatory, or the requirements can be greater than or equal to (>=)?

Thank you for your time.

Make it possible to "guess" the repo slug of the current repository so the user doesn't have to give full repo slug when listing PR of current directory, i.e. instead of:

% git hub guyzmo/git-repo clone
% cd git-repo
% git hub guyzmo/git-repo request list

make it possible to do:

% git hub guyzmo/git-repo clone 
% cd git-repo
% git hub request list

I guess the strategy would be to look for the fqdn value in URLs of all the remotes and when found, regex it to get the repo-slug!

I wanted to setup a custom gitlab service, so I edited my .gitconfig to include the following section:

[gitrepo "selfhosted"]
        fqdn = mygitlab.example.org
        type = gitlab
        token = SUPER_SECRET

Then when I tried to create a repo I got a "401 Unauthorized" error. I generated a new token just in case it was wrong but that didn't seem to be the problem... I also made sure my instance's GitLab API was working by testing it with curl etc.

Then I thought that maybe git-repo tries to authenticate with another server, so I changed the personal token from my instance to a one generated for my gitlab.com account, and it worked (the repository was created on gitlab.com).

So even when specifying an fqdn in the config file, git-repo still tries to communicate with the default GitLab server, gitlab.com. With the appropriate .gitconfig, gitlab.com and github work fine though.

I should also state that I tried both the latest version from pypi and the master branch and the issue remains.

There is git-repo config that I am trying not to confuse. This issue is about configuring GitHub, GitLab, Bitbucket and not configuring git-repo. Maybe this is out of scope and fits in better with something such as Terraform.io

Problem:
Repos will have a handful of settings, such as enabled services, branch protection, webhooks, merge types, mirroring, etc. Click though web gui introduces inconsistencies and other disadvantages.

If my org were to primarily use GitLab but mirror on GitHub then there are two namespaces to clear and two different configurations to deal with.

Proposed:
Configuration of provider repos in yaml, json (with comments, just minify to strip first), or toml.
A cmd to act on the configuration file such as hub|lab|bit|all create|update|remove

git all create would create whatever repos are configured, in my case, github and gitlab.

The tests are failing since the introduction of the --progress support. The issue being inconsistent failing of the same code between calls. Cf the travis reports.

With a fair bit of debugging, I found out the issue is that because GitPython relies on the kwargs construct to pass around optional arguments, the order of argument will depend on the order of the items within the dict. As when I introduced progress support, opt_arg contains '-v' and '--progress' now, the order depending on both hash('-v') and hash('--progress'). To fix this, making the optional arguments ordered on their value instead on their hash will do the job, but sadly it's not something that can be fixed with a tiny bit of monkey patching…

Thus, I've opened a PR upstream at gitpython-developers/GitPython#431 with a very simple fix for that.

For more convenience, support full URL repo-slugs the same way we support short repo-slugs. Useful for copy/pasting (even if redundant in some cases with canonical git command).

When trying to clone to current working directory git-repo complains that the folder already exists.

git lab clone project/repo .
Fatal error: Cannot clone repository, a folder named ./. already exists!

With git this is allowed assuming that the directory is empty. If the directory is not empty git complains with:

fatal: destination path '.' already exists and is not an empty directory.

Proposal: Use the same logic as git to allow for cloning into empty already existing directories. (PR incoming)

when running bin/py.test betamax is overwriting the cassettes no matter what the recording policy is set to. Need to investigate and solve.

http://i.got.nothing.to/code/on/git-repo:_the_utility_for_services/
For your first time run, I have written a wizard, that you can lanuch using.

I installed git-repo using sudo pip3 install git-repo on a debian sid x64 system. However, upon launching it crashes:

$ git repo config
Traceback (most recent call last):
  File "/usr/local/bin/git-repo", line 7, in <module>
    from git_repo.repo import cli
  File "/usr/local/lib/python3.5/dist-packages/git_repo/repo.py", line 112, in <module>
    __version__ = pkg_resources.require('git-repo')[0].version
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 970, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 861, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (GitPython 2.0.8 (/home/phihag/.local/lib/python3.5/site-packages), Requirement.parse('GitPython==2.0.6'), {'git-repo'})

I have GitPython 2.0.8 installed since I need it for various other projects. The changes in 2.0.8 are all fixes of annoying bugs. Why do I need to downgrade my GitPython installation to a buggy one in order to use git-repo?

The same way github's 2FA support has been recently added (cf #39) implement gitlab's 2FA as well.

Even though implementing 2FA with github was easy thanks to @sigmavirus24's work on sigmavirus24/github3.py where it was already implemented… Doing it on gitlab needs to start with patching gpocentex/python-gitlab to implement a 2FA callback… Looking at the code it looks like there's no mechanism in place for that yet:

• Gitlab.credentials_auth()

That being said, it is unclear how to use 2FA and whether a mechanism is in place for using it with Gitlab's API, as there's no mention of 2FA in the authentication section of the Gitlab API documentation.

Current documentation is way too verbose (29 lines, cf __doc__), and it could be written as follows (16 lines!):

Usage:
    {self} [--path=<path>] [-v...] <target> fork [<user>/<repo>] [<branch>] [--clone]
    {self} [--path=<path>] [-v...] <target> create [<user>/<repo>] [--add]
    {self} [--path=<path>] [-v...] <target> delete [<user>/<repo>] [-f]
    {self} [--path=<path>] [-v...] <target> open [<user>/<repo>]
    {self} [--path=<path>] [-v...] <target> clone <user>/<repo> [<branch>]
    {self} [--path=<path>] [-v...] <target> add <user>/<repo> [<name>] [--tracking=<branch>] [-a]
    {self} [--path=<path>] [-v...] <target> request [<user>/<repo>] (list|ls)
    {self} [--path=<path>] [-v...] <target> request [<user>/<repo>] fetch <request>
    {self} [--path=<path>] [-v...] <target> request [<user>/<repo>] create [<remote_branch> [<local_branch>]] <title> [--branch=<remote>] [--message=<message>]
    {self} [--path=<path>] [-v...] <target> gist (list|ls) [<gist>]
    {self} [--path=<path>] [-v...] <target> gist clone <gist>
    {self} [--path=<path>] [-v...] <target> gist fetch <gist> [<gist_file>]
    {self} [--path=<path>] [-v...] <target> gist create [--secret] <description> [<gist_path> <gist_path>...]
    {self} [--path=<path>] [-v...] <target> gist delete <gist> [-f]
    {self} [--path=<path>] [-v...] [<target>] config [--config=<gitconfig>]
    {self} --help

The issue lies within docopt and I've opened an issue about it. Let's hope we'll find a way.

Another approach would be to consider changing the approach of the help page, by going through a refactoring of the current CLI API design, which could be considered for v2.

I stumbled across this awesome tool while reading hackernews and then a few months later tried to find it took me ages! I had to go through heaps of combinations of search terms before i stumbled across it again. Even now that I have found it, when I searched for it by name git-repo it still doesn't show in google.

Have you thought about changing the name? it would be great if more people could find this!

Add the request subcommand that enables handling of requests:

git-repo [--path=<path>] [-v -v...] <target> request [<user>/<repo>] list
git-repo [--path=<path>] [-v -v...] <target> request [<user>/<repo>] fetch <request>

that makes it possible to list all requests, and fetch a request thanks to its id.

Current code offers a clean and easy to implement API. But as more features gets integrated, the more the help page becomes cluttered. So a redesign of the code would be a good idea to think about.

Also, integrating new services means sensitively increasing the code base. Not everybody wants to use all existing services at once, and implementing new services shouldn't be tied to the main code base: if you want to implement support for Gogs (for example) you shouldn't ask for my permission (by doing fork+PR)!

So the main things I'm thinking to change for a possible V2 redesign are modularizing the CLI API, and extracting the services as external plugins.

Subcommands for the CLI API

So the goal would be to modularize the code for each subcommand, so that each feature comes as an independent module coming with its own help page.

As currently, each service lies in its own module, that would mean implementing a "two axis" modularization: for each service and for each command. Then the main help page would have to be generated and the output would look like:

Usage:
    git-repo [--path=<path>] [-v...] <target> <command> [options]

Options:
    <target>            service to use, using the alias (hub, lab or bb),
    <command>           command to trigger, use --help on that command to see usage.
    --path              repository to use

Commands:
    fork                 Fork a project
    create               Create a new project on the service
    delete               Delete an existing project from the service
    open                 Open the project's page with your browser
    gist                 Manage gists
    request              Manage requests

The best way to do that, would be to have for each feature a module and class following this template:

'''
git-repo : Manage gists from the service 

Usage:
    gist (list|ls) [<gist>]
    gist clone <gist>
    gist fetch <gist> [<gist_file>]
    gist create [--secret] <description> [<gist_path> <gist_path>...]
    gist delete <gist> [-f]

Commands:
    ...
'''

@register_interface
class RepositoryServiceClone:
    def get_service(self, lookup_repository):
        raise NotImplementedError

    def clone(self, user, repo, branch):
        raise NotImplementedError

@register_command('clone', __doc__)
class GitRepoCommandClone:
    def do_clone(self):
        service = self.get_service(lookup_repository=False)
        repo_path = os.path.join(self.path, self.repo_name)
        if os.path.exists(repo_path):
            raise FileExistsError('Cannot clone repository, '
                                  'a folder named {} already exists!'.format(repo_path))
        repository = Repo.init(repo_path)
        service = RepositoryService.get_service(repository, self.target)
        service.clone(self.user_name, self.repo_name, self.branch)
        log.info('Successfully cloned `{}` into `{}`!'.format(
            service.format_path(self.repo_slug),
            repo_path)
        )
        return 0

Then, each service would inherit from each interface offered by a command, raising NotImplementedError exceptions when it does not offer an implementation. All missing implementation for interfaces would be easy to list, though.

Then we could have a change in namespace so it becomes:

git_repo.commands.clone
git_repo.commands.fork
git_repo.commands.create
git_repo.commands.delete
git_repo.commands.open
git_repo.commands.gist
git_repo.commands.request

Use a plugin approach for services

To make it easier to implement new services, they could be offered as independent packages, that would populate in the git_repo.services.ext namespace, and be automagically loaded by git-repo when installed. Then one could only have the code for a single service at any time.

Let's create a new set of commands for gist support, with following usage:

git-repo [--path=<path>] [-v -v...] <target> gist list [<gist>]                                              
git-repo [--path=<path>] [-v -v...] <target> gist clone <gist>                               
git-repo [--path=<path>] [-v -v...] <target> gist fetch <gist> [<gist_file>]
git-repo [--path=<path>] [-v -v...] <target> gist create [--secret] <description> [<gist_path> <gist_path>...]
git-repo [--path=<path>] [-v -v...] <target> gist delete <gist> [-f]

that can list all gists and files within a gist, clone a gist, or fetch a gist's file contents, create a new gist or simply delete it.

[gitrepo "iitserver"]
    type = gitlab
    token = TOP_SECRET
    fqdn = iitserver/gitlab

Results for git repo iitserver clone user/project -vvv
https://gist.github.com/feigaoxyz/94edd2c4844f53403f7cb163109d98e5

I can access the server old styles like git clone http://iitserver/gitlab/user/project.git or git clone git@iitserver:user/project.git.

Is it because I deployed the instance to http not https?

Currently progress bar is not really reporting progress (it's appearing at the end already at 100%). Something's wrong in the implementation between GitPython and my code, need to be investigated.

Improvement of the fork command to have a more natural API.
New API idea (using github as example):

1. git hub fork
   • if current directory is a repository
     • lookup the upstream's URL, make a fork, add the fork as remote
   • otherwise, just fail loudly
2. git hub fork user/repo
   • fork the repository on the server and do not mutate local filesystem
3. git hub fork user/repo path
   • if path exists
     • make the fork and add the fork as remote
   • if path does not exists
     • clone the upstream, make the fork and add the fork as remote

so in the end, you can have a scenario where you've cloned a repository, found a bug and want to fork it, you can:

git hub clone foo/bar
cd bar
# fix a bug, make a commit
git hub fork
git hub push github master # will push to your fork

but if you've cloned a fork, and you want to push to the upstream of that fork, and you know the path to that upstream:

git hub clone foo/bar
cd bar
# fix a bug, make a commit
git hub fork fu/bar . # the path that the fork command shall modify
git hub push github master

if you want to fork a project and clone it elsewhere:

git hub fork foo/bar /path/to/elsewhere

(and if elsewhere is an existing repository, then update it).

or finally, you want to fork a project, but think about cloning it another time:

git hub fork foo/bar

which won't touch the filesystem, just the remote service.