unfinishedsentenc / dockup

Docker image to backup/restore your Docker container volumes to S3

Dockup

Docker image to backup your Docker container volumes

This is a fork from wetransform-os/dockup that uses s3cmd instead of awscli.

Why the name? Docker + Backup = Dockup

Usage

You have a container running with one or more volumes:

$ docker run -d --name mysql tutum/mysql

From executing a $ docker inspect mysql we see that this container has two volumes:

"Volumes": {
  "/etc/mysql": {},
  "/var/lib/mysql": {}
}

Backup

Launch dockup container with the following flags:

$ docker run --rm \
--env-file env.txt \
--volumes-from mysql \
--name dockup unfinishedsentenc/dockup:latest

The contents of env.txt being something like:

AWS_ACCESS_KEY_ID=<key_here>
AWS_SECRET_ACCESS_KEY=<secret_here>
AWS_DEFAULT_REGION=us-east-1
S3_HOST=s3.amazonaws.com
S3_HOST_BUCKET=%(bucket)s.s3.amazonaws.com
S3_SSL=true
BACKUP_NAME=mysql
PATHS_TO_BACKUP=/etc/mysql /var/lib/mysql
S3_BUCKET_NAME=container-backup
S3_FOLDER=mybackups/
RESTORE=false

dockup will use your AWS credentials to create a new bucket with name as per the environment variable S3_BUCKET_NAME, or if not defined, using the default name container-backup. The paths in PATHS_TO_BACKUP will be tarballed, gzipped, time-stamped and uploaded to the S3 bucket.

To place backups in a specific folder in the S3 bucket, provide it in the S3_FOLDER variable. It should either be empty or hold a path and end with a slash.

To use a custom S3 endpoint (like RGW or Minio) you can supply s3cmd options via the environment variables S3_HOST for --host, S3_HOST_BUCKET for --host-bucket and S3_SSL=false for --no-ssl. They default to AWS.

For more complex backup tasks as dumping a database, you can optionally define the environment variables BEFORE_BACKUP_CMD and AFTER_BACKUP_CMD.

Detect volumes

Instead of providing paths manually you can set the PATHS_TO_BACKUP to auto. Using this setting the backup script will try to the detect the volumes mounted into the running backup container and include these into the backup archive.

Scheduling

If you want dockup to run the backup as a cron task, you can set the environment variable CRON_TIME to the desired frequency, for example CRON_TIME=0 0 * * * to backup every day at midnight.

Retries

Sometimes creating the TAR archive may fail, often due to modifications to the files while tar is running.

If this happens very often, you should consider using a different option than creating TAR archives for backup. The BEFORE_BACKUP_CMD and AFTER_BACKUP_CMD environment variables can help with that.

If this happens seldomly and you want to avoid a backup failing due to that, you can configure Dockup to retry creating the archive if it fails. For that, use the following environment variables:

• BACKUP_TAR_TRIES - maximum number of tries for the backup (defaults to 5)
• BACKUP_TAR_RETRY_SLEEP - number of seconds to wait between retries (defaults to 30)

Restore

To restore your data simply set the RESTORE environment variable to true - this will restore the latest backup from S3 to your volume. If you want to restore a specific backup instead of the last one, you can also set the environment variable LAST_BACKUP to the desired tarball name.

For more complex restore operations, you can define a command to be run once the tarball has been downloaded and extracted using the environment variable AFTER_RESTORE_CMD.

Encryption

You can use GnuPG to encrypt backup archives and decrpyt them again when you need to restore them. You need a GnuPG public key for encryption and the corresponding private key for decryption. Keep the private key safe (and secret), otherwise you will not be able to restore your backups.

For backup, the following environment variables need to be set:

• GPG_KEYRING - the location of the public keyring containing the public key you want to use for encryption
• GPG_KEYNAME - the user ID identifying the key

For restoring an encrypted file, the following environment variables need to be set:

• GPG_KEYRING - the location of the public keyring
• GPG_SECRING - the location of the secret keyring containing the private key you need for decryption
• GPG_PASSPHRASE - the passphrase needed to access the private key

Notifications

To enable notifications for backups you can use the following environment variables:

• NOTIFY_BACKUP_SUCCESS - set to true to enable notifications on backup success
• NOTIFY_BACKUP_FAILURE - set to true to enable notifications on backup failure

In addition, you need to configure a notification method.

Currently supported are the following notifications methods:

Slack

To configure Slack notifications you need to set at least the NOTIFY_SLACK_WEBHOOK_URL environment variable. Create an Incoming Webhook as a new integration in Slack and put the Webhook URL in here.

Local testing

There is a handy script ./test-backup.sh you can use for local testing. All you need is Docker and configuring your S3 connection. For that purpose, copy test-env.txt.sample to test-env.txt and adapt the variables accordingly.

Optionally generate a GPG key for testing encryption/decryption using ./gen-test-key.sh. It will be automatically used when you execute ./test-backup.sh. If you want to test w/o encryption after generating the key, rn ./test-backup.sh --no-encryption.