leslieleung / reaper

REpository ArchivER(REAPER) is a tool to archive repositories from any Git servers.

REAPER

English | 简体中文

REpository ArchivER(REAPER) is a tool to archive repositories from any Git servers.

• Features
• Installation
• Usage
  • rip
  • run
  • bury
  • daemon
• Configuration
• Storage
• Run as docker container
  • Docker CLI
  • Docker Compose

Features

• Archive repositories from any Git servers
• Archive repositories of a user/an organization (see Configuration)
• Cron support
• Multiple storage types (see Storage)
• Docker support (see Run as docker container)

Installation

curl -sSfL https://raw.githubusercontent.com/LeslieLeung/reaper/main/install.sh | sh -s -- -b /usr/local/bin

Or get from Release.

Usage

You have to create a configuration file to use REAPER.

repository:
  - name: reaper
    url: github.com/leslieleung/reaper
    cron: "0 * * * *"
    storage:
      - localFile
      - backblaze
    useCache: True

storage:
  - name: localFile
    type: file
    path: ./repo
  - name: backblaze
    type: s3
    endpoint: s3.us-west-000.backblazeb2.com
    region: us-west-000
    bucket: your-bucket-name
    accessKeyID: your-access-key-id
    secretAccessKey: your-secret-access-key

Then you can run REAPER with the configuration file.

reaper -c config.yaml
# or simply call reaper if your configuration file is named config.yaml
reaper

rip

rip archives a single repository defined in configuration.

reaper rip reaper

run

run archives all repositories defined in configuration.

reaper run

Combined with cron, you can archive repositories periodically.

bury

bury archives all release assets of a repository.

reaper bury reaper

daemon

daemon runs REAPER as a daemon. It will archive all repositories defined in configuration periodically.

reaper daemon
# You might want to run it with something like nohup
nohup reaper daemon &

Configuration

For configuration, you can check out this example.

For more details, see Configuration in wiki.

Storage

REAPER supports multiple storage types.

• File
• AWS S3

Run as docker container

Docker CLI

One-off run.

• Change ${pwd}/config/example.config.yaml to your config file path.
• Customize ${pwd}/repo:/repo to be your desired storage path. The in-container path needs to be the same as the path in config file.

docker run --rm \
    -v ${pwd}/config/example.config.yaml:/config.yaml \
    -v ${pwd}/repo:/repo \
    leslieleung/reaper:latest \
    run

Docker Compose

For example compose file, see docker-compose.yml.

docker compose up -d

FAQ

See FAQ.

Stargazers over time

reaper's Issues

new, backup meta info

Is it possible to add a feature that backs up all the meta info in the repository, such as issues/wiki/prs.

Repository config type default value seems not take effect

This is my configuration yaml originally:

repository:
  - name: reaper
    url: github.com/leslieleung/reaper
    cron: "30 * * * *"
    storage:
      - localFile
    useCache: True
  - name: awesome-english-ebooks
    url: xxx
    cron: "30 * * * *"
    storage:
      - localFile
    useCache: True

storage:
  - name: localFile
    type: file
    path: /repo

When I started in deamon mode, it output 2 errors:

Unknown type
Unknown type
Starting daemon

I viewed the source codes, and find that it is because unknown type of repo. So I add type: "repo" to each repo's config, It works correctly now.

I think this is a serious problem that could let this program cant work. So I want to make sure that this is actually a bug or I missed something?

More storage backends

Currently only local file storage is supported. Planning to support more storage backends(s3, webdav etc.).

• S3
• webdav(not planned yet, just listed as an example)
• ...

If there is any storage backends you need, feel free to comment below.

Repository config type default value seems not take effect

This is my configuration yaml originally:

repository:
  - name: reaper
    url: github.com/leslieleung/reaper
    cron: "30 * * * *"
    storage:
      - localFile
    useCache: True
  - name: awesome-english-ebooks
    url: xxx
    cron: "30 * * * *"
    storage:
      - localFile
    useCache: True

storage:
  - name: localFile
    type: file
    path: /repo

When I started in deamon mode, it output 2 errors:

Unknown type
Unknown type
Starting daemon

I viewed the source codes, and find that it is because unknown type of repo. So I add type: "repo" to each repo's config, It works correctly now.

I think this is a serious problem that could let this program cant work. So I want to make sure that this is actually a bug or I missed something?

New, Backup Release

Is it possible to add a feature that backs up all the release files in the repository

Pull repo list from my star/own repos

See GitHub API doc.

However, the API does not support pulling a designated list (see LINK), so pulling all of the repos in star list is the only option available now.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): update module github.com/minio/minio-go/v7 to v7.0.72
• fix(deps): update module github.com/spf13/cobra to v1.8.1
• chore(deps): update alpine docker tag to v3.20.1
• chore(deps): update docker/login-action action to v3.2.0
• fix(deps): update module github.com/go-co-op/gocron/v2 to v2.7.1
• fix(deps): update module github.com/go-git/go-git/v5 to v5.12.0
• fix(deps): update module github.com/spf13/viper to v1.19.0
• chore(deps): update goreleaser/goreleaser-action action to v6
• fix(deps): update module github.com/google/go-github/v56 to v62
• fix(deps): update module github.com/google/go-github/v60 to v62
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

docker-compose

docker-compose.yml

• leslieleung/reaper v0.1.2

dockerfile

Dockerfile

• alpine 3.19.1

github-actions

.github/workflows/release.yml

• actions/checkout v4
• actions/setup-go v5
• docker/login-action v3.1.0
• goreleaser/goreleaser-action v5

gomod

go.mod

• go 1.21
• github.com/go-co-op/gocron/v2 v2.2.6
• github.com/go-git/go-git/v5 v5.11.0
• github.com/google/go-github/v56 v56.0.0
• github.com/google/go-github/v60 v60.0.0
• github.com/google/uuid v1.6.0
• github.com/gookit/color v1.5.4
• github.com/mholt/archiver/v4 v4.0.0-alpha.8
• github.com/minio/minio-go/v7 v7.0.69
• github.com/spf13/cobra v1.8.0
• github.com/spf13/viper v1.18.2

---

• Check this box to trigger a request for Renovate to run again on this repository