reposync
is a Scala application that maintains local clones of Ellucian's Banner git repositories.
It iterates over each repository in Ellucian's Gitolite instance and creates or updates the corresponding repository in our GitLab instance.
reposync
puts these repositories in the eas/ellucian group in our GitLab instance.
For example, this Ellucian repository:
[email protected]:banner/plugins/banner_student_attendance_tracking.git
is mirrored to this repository in our GitLab instance:
[email protected]:eas/ellucian/banner/plugins/banner_student_attendance_tracking.git
To save time and network bandwidth, reposync
clones repositories into a local directory (specified by -r
) that should be preserved across runs.
reposync
is run daily at 4am by a cron job in our ops
cluster.
The cron job will fail if Ellucian revises history in their repositories. If this happens, add --force
to container arguments. With --force
, reposync
will force push to GitLab.
The cron job depends on secrets that are built from external Gradle properties when reposync
is deployed:
-
A GitLab user's private token used by
reposync
to make changes on behalf of that user. ThegitlabApiKey
property must specify the private token of a GitLab user that has the ability to create repositories and push to protected branches in the Ellucian group (master or owner). -
An ssh key pair (without a passphrase) registered with Ellucian to access the Ellucian Gitolite instance. The
sshPrivateKey
property must specify the base 64 encoded private ssh key. -
Base 64 encoded ssh host keys for local and Ellucian git remotes in the
sshHostKeys
property.
Ellucian's customer facing gitolite instance lists some repositories as available, but git clone
fails with permission errors.
To avoid wasting time attempting to clone them, reposync
will skip repositories in the snubbies
list in src/main/resources/config.yaml
.
Pass --ignore-snubbies
to reposync
to have it try to mirror all repositories that Ellucian's gitolite instance lists.
When --mirror
is set, reposync
runs git push --mirror
to update repositories in our GitLab instance,
insuring that our repositories are identical replicas of Ellucian repositories.
This is useful when we have accidentally pushed changes and wish to restore them.
Unfortunately, some Ellucian repositories have remote branches that GitLab will not accept, which causes git push --mirror
to fail.
As a workaround, reposync
will remove these bad remote branches when run with --mirror --force
options.
If --mirror
is not used, reposync
runs git push
with refspec patterns that avoid pushing bad remote branches.
This will preserve extraneous commits in our repositories that are not in Ellucian repositories.
reposync
is built as a Gradle Scala application. To build and push an image, run:
./gradlew buildImage
Create a property file named ~/.gradle/prod-reposync.properties
that contains secrets as described above:
gitlabApiKey=<your-GitLab-API-key>
sshPrivateKey=<base-64-encoded-ssh-private-key>
sshHostKeys=<base-64-encoded-ssh-host-keys>
You can generate base64 encoded ssh keys with:
echo "sshPrivateKey=$(base64 -w 0 /path/to/ssh/private/key)"
echo "sshHostKeys=$(ssh-keyscan banner-src.ellucian.com 2>/dev/null | base64 -w 0)"
To deploy a reposync
cron job, run:
./gradlew deployProd
reposync 1.3.1
Usage: reposync [options]
-c, --config <file> override built-in configuration file
-d, --debug display debugging output
--debug-gitlab-api enable GitLab API request and response logging
-h, --help display this usage message
-f, --force use --force on push to local repositories
-i, --ignore-snubbies try to mirror repositories on the snubbies list
-l, --list-repo_age list date of last commit in local mirror repos
-m, --mirror use --mirror on push to local repositories
-n, --no-change no changes, display actions only
-r, --repo-dir <directory>
repo cache directory
-t, --token <token> GitLab API token
-v, --verbose display verbose output```