borgmatic-collective / docker-borgmatic Goto Github PK

Setting the timezone by setting the TZ environment variable does not work. This could be resolved by installing tzdata into the image.

I'm using this container in unraid to set up backups to an intermittently connected external drive. Following the borgmatic docs, I'm trying to probe for the drive using findmnt, however when I run it in the container I get findmnt: not found.

hooks: before_backup: - findmnt /mnt/disks/Seagate_Backup_Plus_Drive > /dev/null || exit 75 - echo "Starting a backup."

https://torsion.org/borgmatic/docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server/#soft-failure-command-hooks

Could you add findmnt to the docker image?

When looking at the borgmatic output I noticed a borg backup warning message:

Using a pure-python msgpack! This will result in lower performance.

The message can be reproduced with the following borg command:

/ # borg prune --dry-run --keep-within 10y /repository                                                         
Using a pure-python msgpack! This will result in lower performance.                                            
/ #

I am using the b3vis/borgmatic Docker Hub image bhxxxelj3bkkunf6nhu6p5f (2018-01-27T18:20:04.370Z) which contains borg 1.1.4.

I think it would be more beneficial if the image tag follows the borgmatic semver and we state in the readme that we will always follow the latest borgbackup package at time of borgmatic release.

instead of borgmatic:v1.2.0-1.6.1 -> borgmatic:1.6.1

thoughts @witten ?

Hello,

I followed your documentation and now I have a running docker doing backups, everything seems to be good, except that today I noticed that the docker was doing a backup when it wasn't supposed to.
I checked my crontabs.txt file and everything was set correctly there, so I couldn't understand why it was doing a backup.
Then I ran crontab -l and it showed me a different crontab config, it was showing me the first crontab config I did when I was testing the docker.

Following the borgmatic original docs I did a chmod +x to my /etc/borgmatic.d/crontab.txt and then I manually copy the file to /etc/crontabs/root

Can you please update your documentation explaining a little bit more about the permissions on the crontab.txt file and how exactly the file inside /etc/borgmatic.d/ sync with cron from /etc/crontabs/ ?

Maybe after every change I do to the crontab.txt I had to run some commands that I don't know yet, if that's the case, please put it in your documentation.

To be clear, the file that I want to be changing from time to time is the crontab.txt, like your docs says...

Thank you for your time, it is a great docker.

This is more for discussion rather than anything else.

Issue 1:

ntfy hasn't released their latest version to pypi. The latest version it installs is from 2018 (2.7.0 instead of 2.7,1).

A temporary workaround currently is to run:

Step 1:
Install latest version:
python3 -m pip install git+https://github.com/dschep/ntfy.git@master --upgrade

Step 2:
Install additional dependencies:
python3 -m pip install ntfy[pid,emoji,xmpp,telegram,instapush,slack,rocketchat]

Issue 2:

sleekxmpp has been sunset, the replacement is Slixmpp. Unforfunately I don't know if this is a drop-in replacement, but I'm unsure how we'd get ntfy to utilise this.

Note that sleekxmpp will not install on python 3.10, so we'll have issues there.

Would it be easier to drop XMPP given these issues?

Or maybe investigate apprise instead? This would have the added benefit of replacing both ntfy and msmtp containers in favour of a single apprise container leading to less management.

Hi,

Is it possible to update the version of borg for fix this issue ?
I think we need at least this merge PR.
If not, do you have a temporary fix for this ? Because your docker-borgmatic isn't usable now....

Thanks

I want to be able to stop all my services in my docker compose project at once, but borgmatic seems to ignore the SIGTERM and wait for the backup to complete (which might take a while). I think it should be passed down to borg, so it can safely stop the backup without leaving a lock behind or creating a broken backup.

As far as I can tell, Ctrl-C seems to work with borg itself so I imagine there must be some way to make this work.

I use the logging feature in Borgmatic, but currently have to deal with log rotation in a cron script on the host. It would be much simpler and more self-contained if you could add this common utility to the borgmatic docker image.

It's common practice with Docker containers to log to stdout, so container logs can be seen with the "docker logs" command, and if desired, logs can be collected from all the relevant containers and centralized. More on that at https://docs.docker.com/engine/admin/logging/overview/

It looks like right now, the crond invocation is logging to a file in the /config volume:

/usr/sbin/crond -f -L /config/crond.log

My suggestion would be to change that to log to stdout, both so that Docker can collect the logs, but also to keep logs out of the /config directory. For instance, that volume can't be read-only as long as it's doing double-duty as a log volume. Here's the change:

/usr/sbin/crond -f -L /dev/stdout

I tried running that variant manually, and confirmed that logs indeed go to stdout. However, I didn't try to rebuild the container image with this change or confirm that "docker logs" works as expected.

Hi,

First, thank you for this project !

I'm sorry because it's not really an issue but i'm stuck when I try to do a restoration from a backup and I don't really know where to ask for this... (it's probably a silly question coming from a misunderstanding on my part)

For restoring, as said in the README, I run this docker-compose -f docker-compose.yml -f docker-compose.restore.yml run borgmatic, then I can extract or mount my archive for restoring. But in practice, when I configure my docker-compose.yml I mount volumes that I want to backup in "ReadOnly", so I can't modify/restoring file here. So my question is... What is the good practice to restoring files when the volume are in RO ?

Thanks...

It's possible to convert my personal docker hub account into an organisation and possibly rename. This will need to be communicated to users if we were to go down this path. No issues with it being on my docker hub account, I'm just thinking for consistency it should be under the 'borgmatic-collective' moniker than b3vis.

Thoughts @witten

After restarting the container borgmatic can't find the borg cache. When running the borgmatic command it gets stuck with Warning: Attempting to access a previously unknown unencrypted repository.

The cache and repo folder are mounted to the right mount points and I also specified the standard borg cache location(/root/.cache/borg/) in the borgmatic config file.

After manually allowing the use of the unknown unencrypted repository it works on the next run without a problem but after restarting the container the error is coming back at some point.

I also checked the repo id and the id in the cache folder and both match.

docker-compose.yml entrie

  web_file_backup:
    image: b3vis/borgmatic
    env_file: .env
    volumes: 
      - ./borgbackup/config.yml:/etc/borgmatic.d/config.yml
      - ./borgbackup/crontab.txt:/etc/borgmatic.d/crontab.txt
      - /srv/web/public/:/mnt/source/:ro
      - /srv/web/log/:/log/
      - /srv/web/backup/borg_repo:/mnt/borg-repository
      - /srv/web/backup/borg_cache:/root/.cache/borg

config.yml

location:                                                                                                                                                                                                                                                                                                                               
    source_directories:                                                                                                                                                                               
        - /mnt/source/system                                                                                                                                                    
    repositories:
        - /mnt/borg-repository
storage:
    borg_cache_directory: /root/.cache/borg/
    archive_name_format: 'web-{now}'

retention:
    keep_daily: 14
    keep_weekly: 4
    keep_monthly: 12
    keep_yearly: 1
    prefix: 'web-'

consistency:
    checks:
        - repository
        - archives

    prefix: 'web-'
hooks:
    before_backup:
        - echo "`date` - Starting a backup job."
    after_backup:
        - echo "`date` - Backup created."
   on_error:
        - echo "`date` - Error while creating a backup.

crontab.txt

1 0 * * * PATH=$PATH:/usr/bin /usr/bin/borgmatic -c /etc/borgmatic.d/config.yml --stats  -v 1 2>&1 | tee /log/BorgBackup.log

My setup uses a borgmatic container in each stack, taking care of backing up the data for only that stack. Some stack only require a database to be backed up, so I don't need to mount data volumes for borgmatic. The VOLUME statement creates anonymous volumes for each borgmatic container.

According to this , the VOLUME statements are not necessary to be able to mount a folders/volumes in the container. However, they create unnamed volumes. So in my case, running several borgmatic images, I have many unused anonymous volumes created by borgmatic.

I suggest replacing them with mkdir commands. This ensures the locations are available, but does not create unnecessary volumes. Mounting volumes from the host still works as before. The only difference is, in a case where no explicit volume has been mounted, the data is lost after the container stops. But since in the compose file all of these are explicitly mounted, this is not a problem.

Didn't change any config from what I was using before, and now get this message at each startup of the container:
/bin/sh: can't open '/etc/borgmatic.d/msmtprc.sh': No such file or directory

Hi!

I am kinda new with borg. So I was wondering whats the correct procedure with starting and using a borg repository.

in the docker-compose.yml file I changed the volume into
- '$PWD/repository:/mnt/repository'

Is this the correct procedure to create a repo?

docker exec -it borgmatic sh
cd /mnt/repository/
borg init -e test.borg

Also in the config.yml I added
repositories:
- /mnt/repository

does it need to be

- /mnt/repository/test.borg

instead?

Hi, good stuff.

Problem: The base Docker image build doesn't work for Linux Alpine 3.12 on armhf architecture. The py3-wheel package is not available and only python3.6 is installed. Below is a working Dockerfile.

I can make a pull request, just let me know how to organize the project folders per architecture.

FROM alpine:latest as builder
LABEL mainainer='b3vis'
ARG BORG_VERSION=1.1.14
ARG BORGMATIC_VERSION=1.5.10
ARG LLFUSE_VERSION=1.3.6
RUN apk upgrade --no-cache \
    && apk add --no-cache \
    alpine-sdk \
    python3-dev \
    py3-pip \
    openssl-dev \
    lz4-dev \
    acl-dev \
    linux-headers \
    fuse-dev \
    attr-dev
#    py3-wheel \
RUN pip3 install --upgrade pip \
    && pip3 install --upgrade borgbackup==${BORG_VERSION} \
    && pip3 install --upgrade borgmatic==${BORGMATIC_VERSION} \
    && pip3 install --upgrade llfuse==${LLFUSE_VERSION}

FROM alpine:latest
LABEL mainainer='b3vis'
COPY entry.sh /entry.sh
RUN apk upgrade --no-cache \
    && apk add --no-cache \
    tzdata \
    sshfs \
    python3 \
    openssl \
    fuse \
    ca-certificates \
    lz4-libs \
    libacl \
    postgresql-client \
    mariadb-client \
    && rm -rf /var/cache/apk/* \
    && chmod 755 /entry.sh
VOLUME /mnt/source
VOLUME /mnt/borg-repository
VOLUME /etc/borgmatic.d
VOLUME /root/.borgmatic
VOLUME /root/.config/borg
VOLUME /root/.ssh
VOLUME /root/.cache/borg
#COPY --from=builder /usr/lib/python3.8/site-packages /usr/lib/python3.8/
COPY --from=builder /usr/lib/python3.6/site-packages /usr/lib/python3.6/
COPY --from=builder /usr/bin/borg /usr/bin/
COPY --from=builder /usr/bin/borgfs /usr/bin/
COPY --from=builder /usr/bin/borgmatic /usr/bin/
COPY --from=builder /usr/bin/generate-borgmatic-config /usr/bin/
COPY --from=builder /usr/bin/upgrade-borgmatic-config /usr/bin/
CMD ["/entry.sh"]

borgmatic 1.4.0 (just released) includes support for hooks to dump PostgreSQL databases and include those dumps in your backups. I'm not sure if you want to expand scope of your Docker container, but if you do want to support this feature, you could install PostgreSQL's pg_dump/pg_dumpall/pg_restore in your image. In Alpine Linux, that's in the postgresql-client package.

Note that there may be plans to support other databases as well, which would expand scope even further.

Hello,

can you update the docker with the latest version of borg and borgmatic.

It seems that we can't run arbitrary commands with version 1.5.12
https://torsion.org/borgmatic/docs/how-to/run-arbitrary-borg-commands/

Thank you

The MAILTO="" in crontab apparently suppresses output in the docker log. I would therefore suggest to comment it out by default. Any opinions? @pr0totyp

(When running docker logs borgmatic it says sendmail: no recipients found.)

Whenever I do a docker-compose down and then an up again, the docker container gets a random hostname internally.

This is causing the prune command that is run to be: borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prefix {hostname}- --remote-path borg1 --debug --show-rc [email protected]:mailcow

Since it is passing hostname to the prune, it is not catching most previous items that need to be pruned if they were created with a different container of borgmatic.

Attaching image of the different archive names that are being created and then not being pruned

Huge thanks to @B3vis for creating this wonderful container!

I successfully followed the instructions in this repository to set up a Docker borgmatic set up to back up files from a source on the Docker host (e.g. /path/to/files) to a repository also on the host (e.g. /path/to/repo).

The problem is, even if I chown/chmod /path/to/repo to a certain user and group, this container changes the permission of the repository to root with root-exclusive read/write access.

What I want is the ability to at least read this repository from the Docker host, and ideally read and write access.

One thing I tried is use the --user argument with user and group IDs (e.g. 123:456) from the Docker host like this when creating and running this container: docker run --user 123:456 [other arguments go here...] However, this fails because apparently crontab inside the container "requires suid privileges".

So, how can I set up this borgmatic container so that non-root users can access its repository?? As a relative newcomer to Docker I'd appreciate any specific suggestions, thank you!

I've got borgmatic set up as an additional container in a Mailcow containerised setup, running on docker 20.10.5 on an Intel Xeon system running Debian bullseye - along these lines: https://mailcow.github.io/mailcow-dockerized-docs/third_party-borgmatic/

I believe I am seeing borgbackup/borg#5899:

borgmatic-mailcow_1  | crond: USER root pid  64 cmd PATH=$PATH:/usr/bin /usr/bin/borgmatic --stats -v 0 2>&1
borgmatic-mailcow_1  | Fatal Python error: Segmentation fault
borgmatic-mailcow_1  | Current thread 0x00007fa3e65ceb48 (most recent call first):
borgmatic-mailcow_1  |   File "/usr/lib/python3.9/borg/cache.py", line 740 in write_archive_index
borgmatic-mailcow_1  |   File "/usr/lib/python3.9/borg/cache.py", line 736 in fetch_and_build_idx
borgmatic-mailcow_1  |   File "/usr/lib/python3.9/borg/cache.py", line 824 in create_master_idx
borgmatic-mailcow_1  | [email protected]:repo: Error running actions for repository
borgmatic-mailcow_1  | Command 'borg prune --keep-hourly 24 --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prefix {hostname}- --stats [email protected]:repo' died with <Signals.SIGSEGV: 11>.
borgmatic-mailcow_1  | /etc/borgmatic.d/config.yaml: Error running configuration file
borgmatic-mailcow_1  | 
borgmatic-mailcow_1  | summary:
borgmatic-mailcow_1  | /etc/borgmatic.d/config.yaml: Error running configuration file
borgmatic-mailcow_1  | [email protected]:repo: Error running actions for repository
borgmatic-mailcow_1  | Fatal Python error: Segmentation fault
borgmatic-mailcow_1  | Current thread 0x00007fa3e65ceb48 (most recent call first):
borgmatic-mailcow_1  |   File "/usr/lib/python3.9/borg/cache.py", line 740 in write_archive_index
borgmatic-mailcow_1  |   File "/usr/lib/python3.9/borg/cache.py", line 736 in fetch_and_build_idx
borgmatic-mailcow_1  |   File "/usr/lib/python3.9/borg/cache.py", line 824 in create_master_idx
borgmatic-mailcow_1  | Command 'borg prune --keep-hourly 24 --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prefix {hostname}- --stats [email protected]:repo' died with <Signals.SIGSEGV: 11>.
borgmatic-mailcow_1  | 
borgmatic-mailcow_1  | Need some help? https://torsion.org/borgmatic/#issues

Not to repeat the borg bug here, but do you have any suggestions how to instrument the container and Python with debug tools & symbols? I can use gdb, strace, valgrind etc but have not used within a container environment - I'm a relative docker novice and a complete newcomer to Alpine but as I am able to reproduce this relatively rare intermittent issue, it might help to solve the bug if I can get some useful debug information out.

Hi, can we add an example on how to point to a remote repo? Looking at the .env and compose files it looks like the target folder is a mapped volume. Instead, I'd like to configure the source with something like borg create --progress --stats [email protected]:/destination/folder::backup-name /folder/to/back/up

I see ways to map SSH keys and I saw issue #1 , so it seems like I'm just missing something obvious. If there was an example to show where to put [email protected]:/destination/folder::backup-name that would be very helpful.

Thanks!

There are a few improvements in borgmatic v1.5.0 and v1.5.1 that would be ideal:

• #277: Customize Healthchecks log level via borgmatic "--monitoring-verbosity" flag.
• #290: Reduce the default verbosity of borgmatic logs sent to Healthchecks monitoring hook. Now, it's warnings and errors only. You can increase the verbosity via the "--monitoring-verbosity" flag.

The Healthchecks monitoring under 1.4.21 is very chatty, the above quietens this down significantly.

• #284: Backup to a removable drive or intermittent server via "soft failure" feature.

Happy to submit a PR if necessary.

Hi,

First, just wanted to say thanks for your docker image. I've used your docker compose file and your config.yaml and crontab.txt as a starting point. However, I'm unable to SSH into the container itself using sudo docker exec -it borgmatic /bin/bash. Does this mean bash isn't within the container?

If so, I'm unsure how I can verify backups or even restore using the docker container. I'm sure I'm missing something completely obvious but would appreciate your thoughts.

Additionally, does your docker container automatically create a repository? Or is it just the location for said repository (again, without being able to SSH into the container, I'm unsure how this is possible).

Thank you very much

Hello,
I used the following line in borgmatic.d/crontab.txt:

0 6 * * 7 /usr/bin/borgmatic prune --stats -v0 2>&1

Yet it runs every day! Docker logs also show: crond: user root: parse error at 7

crontab -e inside the container does not signal a syntax error and the health-check pings supported by Borgmatic do not (!) report this, as Borgmatic did not even start.
But apparently 7 for Sunday is non-standard, and some cron schedule expression tools online do not mention that, instead allowing for a clean 7 as Sunday.

It does not appear to me that this is an issue of Borgmatic, but rather of the Docker image on which docker-borgmatic is based. Maybe a warning in the docs should make sense?

I have 2 docker containers running this borgmatic image that I'd like to be able to back up to each other. This container has an SSH client but does not appear to have the SSH server component.

My workaround is to install the borg binary to the OS base image but I'd rather just have the 2 docker containers talk directly to each other.

Is there a way to add this without the openssh server being added to the docker image?

Currently, if I restart the host in the middle of a backup, borgmatic can't perform any backups until I manually intervene by removing the lock.

If the host gets restarted or the docker container goes down, the lock stays in place. We can account for at least some of this by removing the lock on SIGTERM if the lock belongs to us.

https://blog.knoldus.com/signals-handling-inside-docker-container/

After checking if the lock is because of us (not sure how to do this yet) run:
borgmatic borg break-lock

This will make it so borgmatic can start working immediately once the container comes back up.

Hey, great work!

Just one small request: please update the readme at dockerhub to the one at github as it seems to be more up to date :)

Hi b3vis! The Dockerfile on dockerhub [1] seems to be not up-to-date with the github repository. In github we have BORG_VERSION=1.1.10, while dockerhub shows BORG_VERSION=1.1.9 and some other changes are not updated. Any reason to hold the latest version back?

[1] https://hub.docker.com/r/b3vis/borgmatic/dockerfile

Hi,
thanks for preparing a ready to go docker image. Unfortunately I have a problem if I try to restore some files. I accessed the docker image via

docker-compose -f docker-compose.yml -f docker-compose.restore.yml run borgmatic

I created a mount point and tried to mount the restore folder but received the following error message:

borg mount not available: loading FUSE support failed [ImportError: Error loading shared library libfuse.so.2: No such file or directory (needed by /usr/lib/python3.7/llfuse.cpython-37m-x86_64-linux-gnu.so)]

Not sure if there is a library mismatch or I just missed something.

Thanks!

Hi,

It will be nice to add a support of Secrets files for environment like the official mysql container. for passwords.

Easy to deploy and more secure than exposing clear password.

Something like this will be perfect:

version: "3.7"

services:
  borgmatic:
    image: b3vis/borgmatic:latest
    container_name: borgmatic
    restart: always
    env_file:
      - "./env/common.env"
      - "./env/borgmatic.env"
    environment:
      - BORG_PASSPHRASE_FILE: /run/secrets/borgmatic-secret1
      - MAIL_PASSWORD_FILE: /run/secrets/borgmatic-secret2
    secrets:
      - borgmatic-secret1
      - borgmatic-secret2
    volumes:
      - "/my/path/source:/mnt/source:ro"
      - "/my/path/borgmatic/borgmatic.d:/etc/borgmatic.d/"
      - "/my/path/borgmatic/sshkeys:/root/.ssh"
      - "/my/path/borgmatic/config:/root/.config/borg"
      - "/my/path/borgmatic/cache:/root/.cache/borg"

secrets:
  borgmatic-secret1:
    external: true
  borgmatic-secret2:
    file: /my/path/secret/MAIL_PASSWORD.txt

Cheers,

Given that recent versions of borgmatic no longer actually write database dumps to file in ~/.borgmatic, and only stream to named pipes in that directory, you probably no longer need to map the directory to the host.

Some related discussion here: https://www.reddit.com/r/BorgBackup/comments/jsklol/borg_backups_hang_and_dont_complete/

When I try using a crontab similar to the example in the README, I get an error that indicates crond is trying to run a command that starts with the name of the user (root), instead of interpreting it as the user to run the command as. Here's the contents of crond.log when that occurs:

/ # cat /config/crond.log
crond: crond (busybox 1.26.2) started, log level 8
crond: USER root pid   7 cmd root PATH=$PATH:/usr/bin /usr/bin/borgmatic --config /config/config.yaml
/bin/sh: root: not found

I believe that's because this crontab is being interpreted as a "personal" crontab instead of a system-wide one, and thus the user is assumed to be the current user.

And here's the content of my crontab.txt:

0 4 * * * root PATH=$PATH:/usr/bin /usr/bin/borgmatic --config /config/config.yaml

However, when I remove "root" from the crontab and try again, cron runs just fine, as evidenced by no more errors in the log file, and a new archive showing up in my repository.

I've deployed the base image and made a test config, to check that everything was fine I entered the container and manually started a backup, however, it hangs on startup and the backup seems to never begin. The source folder contains roughly 1GB of files (pretty small) and the destination is a newly initialized repo from the container itself.

Is this an issue with the image or I should seek support from borg\borgmatic itself?

 # borg init -e none /mnt/borg-repository/
/ # ls /mnt/borg-repository/
README       config       data         hints.1      index.1      integrity.1
/ # PATH=$PATH:/usr/bin /usr/bin/borgmatic --stats -v 0 2>&1
Starting a backup job.
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
Deleted data:                    0 B                  0 B                  0 B
All archives:                    0 B                  0 B                  0 B

                       Unique chunks         Total chunks
Chunk index:                       0                    0
------------------------------------------------------------------------------

location:
    source_directories:
        - /mnt/source
    repositories:
        - /mnt/borg-repository
    one_file_system: true

storage:
    compression: lz4
    archive_name_format: 'backup-{now}'

retention:
    keep_hourly: 2
    keep_daily: 7
    keep_weekly: 4
    keep_monthly: 3
    prefix: 'backup-'

consistency:
    checks:
        - repository
        - archives
    check_last: 3
    prefix: 'backup-'

hooks:
    before_backup:
        - echo "Starting a backup job."
    after_backup:
        - echo "Backup created."
    on_error:
        - echo "Error while creating a backup."

Hey,

I just tried to dump a PostgreSQL server 13 database.
However, the pg_dump binary installed within the latest image is incompatible with PostgreSQL 13.1.
The version of pg_dump is 12.4.
Is there a way to update this?

The package seems to have been updated: https://pkgs.alpinelinux.org/package/edge/main/x86/postgresql-client
So maybe a rebuild would suffice?

I'm using docker-borgmatic to backup volumes of several other containers. To get consistent data during the backup, I would like to stop the containers in the before_backup hook and restart them in the after_backup hook.

To do this I need to run docker in docker which shouldn't be a problem if the docker engine would be installed in the image. Is this something that could be added or are there any objections?

The pip3 is missing from the builder stage in the base image.

(55/56) Installing python3 (3.8.3-r0)
(56/56) Installing python3-dev (3.8.3-r0)
Executing busybox-1.31.1-r19.trigger
Executing ca-certificates-20191127-r4.trigger
OK: 327 MiB in 70 packages
/bin/sh: pip3: not found
The command '/bin/sh -c apk upgrade --no-cache     && apk add --no-cache     alpine-sdk     python3-dev     openssl-dev     lz4-dev     acl-dev     linux-headers     fuse-dev     attr-dev     && pip3 install --upgrade pip     && pip3 install --upgrade borgbackup==${BORG_VERSION}     && pip3 install --upgrade borgmatic==${BORGMATIC_VERSION}     && pip3 install --upgrade llfuse' returned a non-zero code: 127

The py3-pip package is missing from the apk list of packages.`

Could we specify the Borg version as a variable in the Dockerfile? Then install that specific version in the pip install command? The latest Borg/Borgmatic version may not be the most stable or to some, considered the "safest" as shown in this issue: 1.1.8 has BAD gpg signature on pypi.org

This will allow each build to be tagged and pushed to Docker Hub with the same Borg/Borgmatic version as the pip install. With tagged versions should allow an app like watchtower to automatically install the latest version if a user decides to use the latest tag or just get a notification.

Currently, doesn't seem like borgmatic or borg logs are printed on stdout or persisted on disk anywhere. This would be a good to have in case of failed backups.

Hi, I'm trying to initialize a repo and this message is the first line that I can see (before a bunch of info about these repos generating security errors when using older versions of borg and some other stuff) when running the command sh -c "borgmatic --init --encryption repokey-blake2".

I've started the container by overriding the entrypoint to launch an interactive shell session in order to initialize the repo.
I've suspected it might be down to the NTSF target drive connected to my nas, so I've reformatted it to EXT4, but no joy. Another local target also resulted in the same message. I've also tried different users (via PUID and PGID Docker environmental variables) to see if it makes a difference, but it didn't.
At the same time I do see initialization happening at the target with some of the files and directories being present that are referenced in other issues as being important to be there:
config, data, hints.1, index.1, integrity.1, nonce, README

I'm completely new to borg/borgmatic and while I've spent a good amount of time familiarizing myself with the solutions, I could easily be overlooking something.
Any help with this would be appreciated. Thx.

I need the ability to start/top docker containers using the borgmatic pre- and post- hooks.

The plan is to use "HA dockermon" from within the borgmatic container. Would you mind adding curl to the docker container for me? Then, I couls be able to stop any container via:

curl -v -X POST <ha_dockermon_ip>:8126/container/container_name --header 'content-type: application/octet-stream' --data '{"state": "stop"}'

Hi docker-borgmatic users out there!

I've just realised that Docker grants read-write access to a read-only mounted directory when a sub-folder was bind-mounted read-write. My setup looks like follow:

srv/                 --> mounted read-only as $VOLUME_SOURCE
├── guests           --> this disk is a backup source
├── magnetic         --> this disk is a backup source
└── magnetic-backup  --> this disk is a backup target
                         and mounted read-write as $VOLUME_TARGET

[...]
    volumes:
      - ${VOLUME_SOURCE}:/mnt/source:ro           # backup source
      - ${VOLUME_TARGET}:/mnt/borg-repository     # backup target
[...]

In terms of backup this is miserable as the source should be read-only while the target should be read-write. The solution is to bind-mount guestsand magnetic individually which is not possible with the current docker-compose.yml and .env file as only one backup-source can be defined.

I've defined now the bind-mounds in .env in one variable like [/srv/guest:/mnt/source/guest:ro, /srv/magnetic:/mnt/source/magnetic:ro, /srv/magnetic-backup:/mnt/borg-repository:rw, ...] and only refer to this variable in the docker-compose.yml volume declariation. This solution is not very nice in terms of readability. Has someone else more than one backup source as well? How do you deal with it?

If I'm using this with remote repos (https://www.borgbase.com) what should I specify for VOLUME_TARGET?

Also, how would the config.yaml work for multiple source dirs instead of a single bind mount?

Thanks for the code!

Hi! I've modified the Dockerfile to build borg and borgmatic in an intermediate container so the final image will not have all the build dependencies using the multistage-build mechanism.

From the builder I copy the complete site-packages folder and borg related python scripts from /usr/bin.

COPY --from=builder /usr/lib/python3.6/site-packages /usr/lib/python3.6/
COPY --from=builder /usr/bin/*borg* /usr/bin/

Not sure if this is the best approach to move things around or there is a better way in the python world? Any thoughts? May I raise a pull request?

I set borgmatic configuration i docker-compose.override file:
I mount my local storage here in configuration.

borgmatic-mailcow:
    image: b3vis/borgmatic
    restart: always
    dns: ${IPV4_NETWORK:-172.22.1}.254
    volumes:
      - vmail-vol-1:/mnt/source/vmail:ro
      - mysql-socket-vol-1:/var/run/mysqld/:z
      - /backup:/home/user/mnt/backup/backup
      - ./data/conf/borgmatic/etc:/etc/borgmatic.d:Z
      - ./data/conf/borgmatic/state:/root/.config/borg:Z
      - ./data/conf/borgmatic/ssh:/root/.ssh:Z
    environment:
      - TZ=${TZ}
      - BORG_PASSPHRASE=kC3hJ0fC3sV6dJ9w
    networks:
      mailcow-network:
        aliases:
          - borgmatic

I set config file at location data/conf/borgmatic/etc/config.yaml

location:
    source_directories:
        - /mnt/source
    repositories:
        - /home/user/mnt/backup/backup
retention:
    keep_daily: 7
    keep_weekly: 1
    keep_monthly: 1

hooks:
    mysql_databases:
        - name: mailcow_db_name
          username: mailcow_user
          password: mysql_pass
          options: --default-character-set=utf8mb4

I succesfully initiate repository with command:
docker-compose exec borgmatic-mailcow borgmatic init --encryption repokey-blake2

But when I run manually backup with command:

docker-compose exec borgmatic-mailcow borgmatic -v 2

I get error:

mysqldump: Got error: 2002: "Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (111)" when trying to connect
Command 'mysqldump --add-drop-database --user mailcow_user --default-character-set=utf8mb4 --databases mailcow_db_name > /root/.borgmatic/mysql_databases/localhost/mailcow' returned non-zero exit status 2.

I tried to connect to bormgatic container:
docker exec -it mailcowdockerized_borgmatic-mailcow_1 /bin/sh

I can see socket is mounted with this permissions:

-rw-rw----    1 5000     5000             2 Aug  3 11:07 mysqld.pid
srwxrwxrwx    1 5000     5000             0 Aug  3 11:07 mysqld.sock

When I try to login to mysql from borgmatic container I get same error.

Hi! Just realized that the Dockerfile in the README is not the same as the one in the repo. I would suggest to remove it from the README at all.

Author of borgmatic here. This is super cool! I want to use it. However, I create backups with borgmatic and Borg to repositories on a remote host via SSH, instead of a local repository that can be conveniently mounted into a container.

So consider this a feature request for your Docker container to support SSH repositories. I'm not that familiar with Alpine, but I think all you may need to do is apk install "openssh-client", and then document how to mount an "~/.ssh/" volume into a container so that passwordless SSH will work when cron runs borgmatic automatically. I'm happy to beta test this if you like.

For background, here are the Borg docs on remote SSH repositories: https://borgbackup.readthedocs.io/en/stable/usage/general.html#repository-urls

Hi! I would like to use default locations for config and cache and update the readme and Dockerfile accordingly. This way the image would be more consistent with borgmatic documentation.

New set of volumes would be:

  - /mnt/source          # backup source
  - /mnt/repository      # backup destination
  - /root/.config/borg   # config and keyfiles
  - /root/.cache/borg    # checksums used for deduplication
  - /root/.ssh           # ssh key for remotebackup
  - /etc/borgmatic.d     # borgmatic config file(s) + crontab.txt

What do you think about?