mbitsnbites / git-tools

Various tools for Git

⚠️ This repository has moved to: https://gitlab.com/mbitsnbites/git-tools

git-tools

This is a collection of useful tools for Git.

join-git-repos

The purpose of join-git-repos is to combine several Git repositories into a single repository. It will do the following:

• Import the complete history (including tags and branches) of two or more repositories into a new, single repository.
• Move each source repository into a dedicated sub-folder in the new repository, for all commits in the history (this guarantees that there are no merge conflicts).
• Stitch the histories of the main branches (e.g. master) of all source repositories, in commit date order, while preserving branch and merge points (this makes it possible to check out an old commit and the result should be a reasonable representation of all the source repositories at the given time in history).
• Rename all the refs (branches and tags) in all source repositories except the first (main) repository. For instance, when joining the repositories foo and bar, there will be two branches called master and master-bar in the resulting repository. This minimizes the risk of name collisions.

git-filter-blobs

This tool allows you to modify blobs (file content) for all versions of all files in a repository. The result is written to a new repository that is a complete clone of the source repository (except that file content may have changed).

For example, to run clang-format on all C/C++ files in the history (making it appear as if the files have been correctly formatted from the beginning), you can run:

git-filter-blobs.py -f h,hpp,c,cpp path/to/old-repo path/to/new-repo 'clang-format -style="{BasedOnStyle: Chromium, ColumnLimit: 100}"'

Note that if the command string includes %f, it will be expanded to the filename of the blob that is being processed. That can be useful for creating more advanced file filters (e.g. to filter based on the folder in which the file is located), or to select different tools for different file types, for instance.

make-submodule-repo

make-submodule-repo will create a new (local) repository with one or more submodules. The history of the new repoistory will contain one commit (updating the respective submodule SHA) for each commit in the master branch of the respective source repository.

This can be useful, for instance, for creating a repository for third party dependencies that is to be included in a bigger repo using join-git-repos.

git-tools's Issues

Handle .gitmodules in join-git-repos

When moving repositories into subdirectories (default), .gitmodules also get moved (obviously).

The correct behavior is for all .gitmodules files to be merged into a single file in the root folder. Also all paths in the .gitmodules file need to be prefixed with the correct subdirectory.

This can be tricky...

MemoryError in join-git-repos.py

I'm getting this error shortly after join-git-repos.py starts:

Exporting the main repository (my.git)...
Traceback (most recent call last):
  File "C:\Projects\git-tools\join-git-repos.py", line 463, in <module>
    main_commands = exportrepo(main_spec['path'])
  File "C:\Projects\git-tools\join-git-repos.py", line 102, in exportrepo
    return parseexport(subprocess.check_output(cmd))
  File "C:\Python27\lib\subprocess.py", line 567, in check_output
    output, unused_err = process.communicate()
  File "C:\Python27\lib\subprocess.py", line 791, in communicate
    stdout = _eintr_retry_call(self.stdout.read)
  File "C:\Python27\lib\subprocess.py", line 476, in _eintr_retry_call
    return func(*args)
MemoryError
fatal: Could not write blob 'f8c32f916652446202463db916551abc5513e78f': Broken pipe

Option to use author date instead of commit date for ordering

If the repositories had their histories rewritten before like rebase, the commit dates could be different from the author dates. The current ordering by commit date might not be what one expects.

It would be helpful to add an option to allow using the author date instead of the commit date.