On restore there’s no more to choose from different dates. I shows only always the timestamp from the initial backup.

I am trying to restore a VM that was backed up with eve4pve-barc. Whenever I try to restore it ends up with following error:

rbd: error opening image vm-104000031-disk-0: (2) No such file or directory
/usr/sbin/eve4pve-barc: line 1258: local: 0/0: division by 0 (error token is "0")

I did the delete the VM from Proxmox. Does the VM need to be present for me to restore?

Hi,

thanks for this project! We have two pve/ceph cluster on 4.4
After update eve4pve-barc to version 1.2.7 the second on third disks from a VM have errors.

We see this error in debug output:
VM 906 - Ceph last snapshot 'ceph_kvm/vm-906-disk-3' not found!

in the barc directory:

-rw-r--r-- 1 root users 214748364800 Nov 28 10:19 171128094501ceph_kvm.vm-906-disk-1.img
-rw-r--r-- 1 root users 214748364800 Nov 28 10:53 171128094501ceph_kvm.vm-906-disk-3.img
-rw-r----- 1 root users 359 Nov 28 10:53 171128094501.conf
-rw-r--r-- 1 root users 535279487 Nov 28 11:13 171128110001ceph_kvm.vm-906-disk-1.diff
-rw-r--r-- 1 root users 77719858 Nov 28 11:20 171128110001ceph_kvm.vm-906-disk-3.diff
-rw-r----- 1 root users 359 Nov 28 11:20 171128110001.conf
-rw-r--r-- 1 root users 750399017 Nov 28 15:07 171128121001ceph_kvm.vm-906-disk-1.diff
-rw-r----- 1 root users 359 Nov 28 15:21 171128121001.conf
-rw-r--r-- 1 root users 1122160106 Nov 28 21:10 171128183001ceph_kvm.vm-906-disk-1.diff
-rw-r----- 1 root users 359 Nov 28 21:10 171128183001.conf
-rw-r--r-- 1 root users 370376380 Nov 28 23:05 171128230001ceph_kvm.vm-906-disk-1.diff
-rw-r----- 1 root users 359 Nov 28 23:05 171128230001.conf
-rw-r--r-- 1 root users 197851087 Nov 28 23:10 171128230801ceph_kvm.vm-906-disk-1.diff
-rw-r----- 1 root users 359 Nov 28 23:10 171128230801.conf

have anyone the same problem?

Hello,
in my Proxmox now eve4pve-barc is giving me these errors during backup:

VM 119 - ======== Ceph Image 'raid1/vm-119-disk-1' ======== 
ERROR: VM 119 - Ceph snapshot 'raid1/vm-119-disk-1@' not found in backup '/mnt/pve/backupceph/barc/119/daily/raid1.vm-119-disk-1.*'
VM 119 - Copy config
VM 124 - ======== Ceph Image 'raid1/vm-124-disk-1' ======== 
ERROR: VM 124 - Ceph snapshot 'raid1/vm-124-disk-1@' not found in backup '/mnt/pve/backupceph/barc/124/daily/raid1.vm-124-disk-1.*'
VM 124 - Copy config
VM 140 - ======== Ceph Image 'raid1/vm-140-disk-1' ======== 
ERROR: VM 140 - Ceph snapshot 'raid1/vm-140-disk-1@' not found in backup '/mnt/pve/backupceph/barc/140/daily/raid1.vm-140-disk-1.*'
VM 140 - Copy config
VM 140 - Merge diff backup '170418173107raid1.vm-140-disk-1.diff' '170507000001raid1.vm-140-disk-1.diff'
Merging image diff: 32% complete...failed.
rbd: merge-diff error
ERROR: rbd merge-diff '/mnt/pve/backupceph/barc/140/daily/170418173107raid1.vm-140-disk-1.diff' '/mnt/pve/backupceph/barc/140/daily/170507000001raid1.vm-140-disk-1.diff' '/mnt/pve/backupceph/barc/140/daily/170507000001raid1.vm-140-disk-1.diff-merged'
ERROR: VM 140 - Merge diff backup '170418173107raid1.vm-140-disk-1.diff' '170507000001raid1.vm-140-disk-1.diff' to '170507000001raid1.vm-140-disk-1.diff-merged'
VM 143 - ======== Ceph Image 'raid1/vm-143-disk-1' ======== 
ERROR: VM 143 - Ceph snapshot 'raid1/vm-143-disk-1@' not found in backup '/mnt/pve/backupceph/barc/143/daily/raid1.vm-143-disk-1.*'
VM 143 - Copy config

What can I do?
Thanks,
Mario

Not always "ceph" in the name.
Changed

image_spec=$(pvesm path "$disk" | grep 'ceph' | awk '{ split($0,a,":"); print a[2]}')

to

image_spec=$(pvesm path "$disk" | awk '{ split($0,a,"/"); print a[5]}')

Possibility to use the command line to restore an image.

Example

eve4pve-barc restore --vmid = 111 --label = 'daily' --path = / mnt / bckceph
--src_image_name = 'pool-rbd.vm-111-disk-1'
--src_image_timestamp = '17 -01-10_11: 47: 30 '
--dest_pool_name = 'pool-rbd'
--dest_image_name = 'vm-111-disk-1-restored'

I know this is a big change but I ask it anyways.
The standard proxmox backup is full only so it is not very helpful.
Your solution is great because it saves only diffs so I can maintain a long history.
There are two remaining problems:

1. if I have, for example, 40 backups, and I start a restore it is very long because it need to start from first file and then merge all 40 files over it
2. if a vm saves a lot of data I have big diffs and so I can keep short history.

Using borg to save snapshots will solve point 1, point 2 and also deduplicate data saving even more space to keep even more history.

If the barc snapshot is missing from the RBD, it aborts. who would want that functionality?

consider the current code:

#Snapshot Housekeeping..                
                #find last snapshot in ceph
                latest_snap=$(rbd snap ls "$image_spec" | \
                              awk '{print $2}' | \
                              grep "$opt_label" | sort -r | head -n 1)

                #not exist snapshot on rbd
                if [ -z "$latest_snap" ]; then
                    log error "VM $vm_id - Ceph last snapshot '$image_spec' not found!";                    
                    call_hook_script "export-diff-abort" "-" "-"                 
                    rc=30                         
                    break;
                fi

We want to backup this VM/disk... the snapshot is merely a means to an end. If it's missing, forget about it, create a new one, and do a new run with renew=0....?

After backup a VM with 3 disks, 2 of 3 images was deleted by housekeeping.

eve4pve-barc backup --vmid=all --label='daily' --path=/backup --renew=0d --retain=0d

VM 101 - ======== Start backup VMID 101 ========
...
VM 101 - Export initial '/backup/barc/101/backup3daily/20200212071936cold-storage.vm-101-disk-3.img'
VM 101 - Size 8.00 GiB Uncompressed: 8.00 GiB Compressionratio 1:1
VM 101 - Export initial '/backup/barc/101/backup3daily/20200212071936cold-storage.vm-101-disk-1.img'
VM 101 - Size 120.00 GiB Uncompressed: 120.00 GiB Compressionratio 1:1
VM 101 - Export initial '/backup/barc/101/backup3daily/20200212071936cold-storage.vm-101-disk-2.img'
VM 101 - Size 25.00 GiB Uncompressed: 25.00 GiB Compressionratio 1:1
VM 101 - -------- Copy config --------
VM 101 - -------- Housekeeping --------
VM 101 - Houskeeping deleting: /backup/barc/101/backup3daily/20200212071936cold-storage.vm-101-disk-2.img
VM 101 - Houskeeping deleting: /backup/barc/101/backup3daily/20200212071936cold-storage.vm-101-disk-1.img
VM 102 - ======== Start backup VMID 102 ========

I get this error:

VM 103 - Merge diff backup '171102000001rbd.vm-103-disk-1.diff' '171110000001rbd.vm-103-disk-1.diff'
Merging image diff: 76% complete...rbd: failed to read first diff data chunk header
Merging image diff: 76% complete...failed.
rbd: merge-diff error

Thanks in advance for any help.
Mario

Can you please build in compression like
https://github.com/teralytics/ceph-backup/blob/master/cephbackup/ceph_backup.py ?

Send email with log process

Hello, very very nice script! (Complimenti .. sono di Ravenna :-)

I have this issue if i have a previus Snapshot created by Web Interf.:

rbd: failed to create snapshot: (17) File exists
ERROR: rbd snap create ceph-lxc/vm-100-disk-1@barcdaily17032813410

Grazie!

Hello to all,

Now the cron file is in /etc/cron.d.
My idea is to move it to /etc/pce/eve/barc/barc.cron so that it is replicated across the entire cluster (corosync).
In this way you can have a unique configuration and with the --only-local parameter the node will only backup its vm, naturally following the rules of the --vmid.
If vm is moved to another node, there will always be backup without having to reconfigure.

What do you think?

Thank you all

Is it possible to schedule backups by pool? It will give feature to add VM to pool and backup would be made automatically.

Hi guys,
I'm going to add compression support.
The features involved are:

• backup
• restore
• assemble

The points are:

• possibility to choose which compression / decompresisone to use lz4, gz, ..
• implement compression and decompression in pipes
• existing backups how to proceed if one wants to switch to compression?

I accept changes and new ideas.

Thank you all

W Italy 😁

When doing a restore and selecting a time, it always restores all the diffs, no matter which time you selected.
So for example today is the 28th and you want to restore to the 26th it restores all diffs to the 28th anyway.

Hi, can you, please, add new switch .. something like: "do not touch old full images" or "data retention on FS"
VM has really huge performance issues with more snapshots, I'm investigating why.
So idea is to have 1 to 2 snapshots (keep 1) and have stored images for disaster recovery reasons.
Thanks

If I understand soft logic correctly the base image is never updated and all the diffs are merged to the first (oldest) snap: current_image = base_image + oldest_diff + newest_diff

Theoretically oldest_diff could become large than base_image and restore could take for a while.

It would be nice to have an ability to update base image every time the oldest snapshot expired/deleted

Thanks

Hi,

Thank you for the job done on this script.
I can't succeed in backuping towards another rbd pool ?
I would like to parse all VMs in a source pool (seems to be possible since 0.1.9) and backup them towards a destination pool.

Is-it actually possible with eve4pve-barc 0.2.0 ?

Best regards,
Rafael

Hello.
In this function:

function map_vmids_to_host(){
for node in $(/usr/bin/pvecm nodes | tail +5 | tr -s ' ' | cut -d' ' -f 4)
do
for vm in $(ssh root@$node qm list|tail +2|tr -s ' '|cut -f 2 -d' ')
do
pvnode[$vm]=$node
done
done
}

tail +5 gives an error
The right syntax is "tail -n +5"

On PVE 5.x, there is no "qemu guest" command, so this option with default = true:

declare opt_qemu_freeze=true

should be changed to "false" checking pve version to suppress the error.

It would be neat if we could do different schedules on the same, for example to keep 5 daily backup, 3 weekly and 1 monthly, etc. backups.

Currently we can do this by setting up a seperate backup job, but that would mean the other backup jobs are taken at the same time, so for example when the monthly backup is taken we would take a daily, weekly and a monthly backup all at the same time.

A better way would be if the daily backup would be taken and copied as a weekly and a monthly backup rather than having to run three backup procedures.

i am using 5.3.11 proxmox

and tried your latest barc

root@pve00:~# eve4pve-barc backup --vmid=101 --label='daily' --path=/root/temp/barc --keep=2
ACTION: Backup
Start backup 2019-03-16 14:25:23
VM 101 - ======== Start backup ========
VM 101 - -------- Snapshots Disks --------
VM 101 - -------- Export images --------
VM 101 - -------- Copy config --------
VM 101 - -------- Merge diff file --------
End backup 2019-03-16 14:25:25

but there is no image file, as you can see it just takes 2 seconds to get just the config file.
is there somehow i can get more info why it is not working

Hi,

I just tested the new version (0.2.5-renew) on my test Proxmox environment, unfortunately I found some bugs. I might try to fix them myself and make a pull request, but due to the corona situation I'm quite busy now. Just want to report those bugs, so that others might be aware of it. But really nice improvements with fsfreeze, checksum check, etc.!!

Unfortunately I had to notice, that the initial backups in the new version (0.2.5-renew) took way more time and storage space then the version before (0.2.1). It looks like the initial backup is now thick provisioned and not thin like it was in 0.2.1.

Some numbers (Test-Proxmox-Cluster with Test VMs, no changes between those 2 runs):

0.2.5-renew:

Elapsed time: 8h and 23m
Transferred data: 1560GB (pretty much exactly the "provisioned" space)

0.2.1:

Elapsed time: 2h and 28m
Transferred data: 449GB (pretty much exactly the "used" space)

Especially when doing offsite backups, this is kind of a problem due to limited bandwidth and expensive traffic.

Is it a bug or is this behavior needed due to some technical requirement?

Stay healthy!
Greetings from Germany!
Andy

Is there any good reason to store Images without a Checksum? Removing the code would clean a lot.

How does barc compare to backy2? https://github.com/wamdam/backy2

backy2 is obviously not tied to proxmox as barc is, but how about other features?
backy2 documents talk about scrubbing, is this something that's taken care of in barc?

Backup command:

Cron root@pve-node3 eve4pve-barc backup --vmid=storage-rbd --label='daily' --path='/mnt/pve/rbd-backup/' --keep=3 --unprotect-snap --syslog --mail='support@'

Backup log:
`

ACTION: Backup
Start backup 2019-06-04 00:05:03
VM 100 - ======== Start backup ========
VM 100 - -------- Snapshots Disks --------
VM 100 - Creating snapshot rbd/vm-100-disk-0@barcdaily190604000503
VM 100 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-100-disk-0@barcdaily190604000503 <<< !!!!!
VM 100 - -------- Export images --------
VM 100 - Export diff '/mnt/pve/rbd-backup//barc/100/daily/190604000503rbd.vm-100-disk-0.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 5% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 9% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 12% complete...Exporting image: 13% complete...Exporting image: 14% complete...Exporting image: 15% complete...Exporting image: 16% complete...Exporting image: 17% complete...Exporting image: 18% complete...Exporting image: 19% complete...Exporting image: 20% complete...Exporting image: 21% complete...Exporting image: 22% complete...Exporting image: 23% complete...Exporting image: 24% complete...Exporting image: 25% complete...Exporting image: 26% complete...Exporting image: 27% complete...Exporting image: 28% complete...Exporting image: 29% complete...Exporting image: 30% complete...Exporting image:
31% complete...Exporting image: 32% complete...Exporting image: 33% complete...Exporting image: 34% complete...Exporting image: 35% complete...Exporting image: 36% complete...Exporting image: 50% complete...Exporting image: 100% complete...done.
VM 100 - Size 7.0G
VM 100 - Remove snapshot rbd/vm-100-disk-0@barcdaily190603000503
Removing snap: 100% complete...done.
VM 100 - -------- Copy config --------
VM 100 - -------- Merge diff file --------
VM 100 - Merge diff backup '190601000503rbd.vm-100-disk-0.diff' '190602000503rbd.vm-100-disk-0.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 16% complete...Merging image diff: 17% complete...Merging image diff: 18% complete...Merging image diff: 19% complete...Merging image diff: 20% complete...Merging image diff: 21% complete...Merging image diff: 22% complete...Merging image diff: 23% complete...Merging image diff: 24% complete...Merging image diff: 25% complete...Merging image diff: 26% complete...Merging image diff: 27% complete...Merging image diff: 28% complete..
.Merging image diff: 29% complete...Merging image diff: 30% complete...Merging image diff: 31% complete...Merging image diff: 32% complete...Merging image diff: 33% complete...Merging image diff: 34% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 43% complete...Merging image diff: 50% complete...Merging image diff: 75% complete...Merging image diff: 100% complete...done.
VM 100 - Removing backup '/mnt/pve/rbd-backup//barc/100/daily/190601000503rbd.vm-100-disk-0.diff'
VM 100 - Removing backup '/mnt/pve/rbd-backup//barc/100/daily/190602000503rbd.vm-100-disk-0.diff'
VM 101 - ======== Start backup ========
VM 101 - -------- Snapshots Disks --------
VM 101 - Creating snapshot rbd/vm-101-disk-0@barcdaily190604000503
VM 101 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-101-disk-0@barcdaily190604000503 <<< !!!!!
VM 101 - Creating snapshot rbd/vm-101-disk-1@barcdaily190604000503
VM 101 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-101-disk-1@barcdaily190604000503 <<< !!!!!
VM 101 - -------- Export images --------
VM 101 - Export diff '/mnt/pve/rbd-backup//barc/101/daily/190604000503rbd.vm-101-disk-0.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 5% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 9% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 12% complete...Exporting image: 13% complete...Exporting image: 14% complete...Exporting image: 15% complete...Exporting image: 16% complete...Exporting image: 17% complete...Exporting image: 18% complete...Exporting image: 19% complete...Exporting image: 20% complete...Exporting image: 21% complete...Exporting image: 22% complete...Exporting image: 23% complete...Exporting image: 24% complete...Exporting image: 25% complete...Exporting image: 26% complete...Exporting image: 27% complete...Exporting image: 28% complete...Exporting image: 29% complete...Exporting image: 30% complete...Exporting image:
31% complete...Exporting image: 32% complete...Exporting image: 33% complete...Exporting image: 34% complete...Exporting image: 35% complete...Exporting image: 36% complete...Exporting image: 37% complete...Exporting image: 38% complete...Exporting image: 39% complete...Exporting image: 40% complete...Exporting image: 41% complete...Exporting image: 42% complete...Exporting image: 43% complete...Exporting image: 44% complete...Exporting image: 45% complete...Exporting image: 46% complete...Exporting image: 50% complete...Exporting image: 51% complete...Exporting image: 52% complete...Exporting image: 53% complete...Exporting image: 54% complete...Exporting image: 55% complete...Exporting image: 56% complete...Exporting image: 57% complete...Exporting image: 58% complete...Exporting image: 59% complete...Exporting image: 60% complete...Exporting image: 61% complete...Exporting image: 62% complete...Exporting image: 100% complete...done.
VM 101 - Size 24G
VM 101 - Remove snapshot rbd/vm-101-disk-0@barcdaily190603000503
Removing snap: 100% complete...done.
VM 101 - Export diff '/mnt/pve/rbd-backup//barc/101/daily/190604000503rbd.vm-101-disk-1.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 5% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 9% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 12% complete...Exporting image: 13% complete...Exporting image: 14% complete...Exporting image: 15% complete...Exporting image: 16% complete...Exporting image: 17% complete...Exporting image: 18% complete...Exporting image: 19% complete...Exporting image: 20% complete...Exporting image: 21% complete...Exporting image: 22% complete...Exporting image: 23% complete...Exporting image: 24% complete...Exporting image: 25% complete...Exporting image: 26% complete...Exporting image: 27% complete...Exporting image: 28% complete...Exporting image: 29% complete...Exporting image: 30% complete...Exporting image:
31% complete...Exporting image: 32% complete...Exporting image: 33% complete...Exporting image: 34% complete...Exporting image: 35% complete...Exporting image: 36% complete...Exporting image: 37% complete...Exporting image: 38% complete...Exporting image: 39% complete...Exporting image: 40% complete...Exporting image: 41% complete...Exporting image: 42% complete...Exporting image: 43% complete...Exporting image: 44% complete...Exporting image: 45% complete...Exporting image: 46% complete...Exporting image: 47% complete...Exporting image: 48% complete...Exporting image: 49% complete...Exporting image: 50% complete...Exporting image: 51% complete...Exporting image: 52% complete...Exporting image: 53% complete...Exporting image: 54% complete...Exporting image: 55% complete...Exporting image: 56% complete...Exporting image: 57% complete...Exporting image: 58% complete...Exporting image: 59% complete...Exporting image: 60% complete...Exporting image: 61% co
mplete...Exporting image: 62% complete...Exporting image: 63% complete...Exporting image: 64% complete...Exporting image: 65% complete...Exporting image: 66% complete...Exporting image: 67% complete...Exporting image: 68% complete...Exporting image: 100% complete...done.
VM 101 - Size 20G
VM 101 - Remove snapshot rbd/vm-101-disk-1@barcdaily190603000503
Removing snap: 100% complete...done.
VM 101 - -------- Copy config --------
VM 101 - -------- Merge diff file --------
VM 101 - Merge diff backup '190601000503rbd.vm-101-disk-0.diff' '190602000503rbd.vm-101-disk-0.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 16% complete...Merging image diff: 17% complete...Merging image diff: 18% complete...Merging image diff: 19% complete...Merging image diff: 20% complete...Merging image diff: 21% complete...Merging image diff: 22% complete...Merging image diff: 23% complete...Merging image diff: 24% complete...Merging image diff: 25% complete...Merging image diff: 26% complete...Merging image diff: 27% complete...Merging image diff: 28% complete..
.Merging image diff: 29% complete...Merging image diff: 30% complete...Merging image diff: 31% complete...Merging image diff: 32% complete...Merging image diff: 33% complete...Merging image diff: 34% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 37% complete...Merging image diff: 38% complete...Merging image diff: 39% complete...Merging image diff: 40% complete...Merging image diff: 41% complete...Merging image diff: 42% complete...Merging image diff: 43% complete...Merging image diff: 44% complete...Merging image diff: 45% complete...Merging image diff: 46% complete...Merging image diff: 48% complete...Merging image diff: 50% complete...Merging image diff: 51% complete...Merging image diff: 52% complete...Merging image diff: 53% complete...Merging image diff: 54% complete...Merging image diff: 55% complete...Merging image diff: 56% complete...Merging image diff: 57% complete...Merging image diff: 58%
complete...Merging image diff: 59% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 80% complete...Merging image diff: 100% complete...done.
VM 101 - Removing backup '/mnt/pve/rbd-backup//barc/101/daily/190601000503rbd.vm-101-disk-0.diff'
VM 101 - Removing backup '/mnt/pve/rbd-backup//barc/101/daily/190602000503rbd.vm-101-disk-0.diff'
VM 101 - Merge diff backup '190601000503rbd.vm-101-disk-1.diff' '190602000503rbd.vm-101-disk-1.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 16% complete...Merging image diff: 17% complete...Merging image diff: 18% complete...Merging image diff: 19% complete...Merging image diff: 20% complete...Merging image diff: 21% complete...Merging image diff: 22% complete...Merging image diff: 23% complete...Merging image diff: 24% complete...Merging image diff: 25% complete...Merging image diff: 26% complete...Merging image diff: 27% complete...Merging image diff: 28% complete..
.Merging image diff: 29% complete...Merging image diff: 30% complete...Merging image diff: 31% complete...Merging image diff: 32% complete...Merging image diff: 33% complete...Merging image diff: 34% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 37% complete...Merging image diff: 38% complete...Merging image diff: 39% complete...Merging image diff: 40% complete...Merging image diff: 41% complete...Merging image diff: 42% complete...Merging image diff: 43% complete...Merging image diff: 44% complete...Merging image diff: 45% complete...Merging image diff: 46% complete...Merging image diff: 47% complete...Merging image diff: 48% complete...Merging image diff: 49% complete...Merging image diff: 50% complete...Merging image diff: 51% complete...Merging image diff: 52% complete...Merging image diff: 53% complete...Merging image diff: 54% complete...Merging image diff: 55% complete...Merging image diff: 56%
complete...Merging image diff: 57% complete...Merging image diff: 58% complete...Merging image diff: 59% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 62% complete...Merging image diff: 63% complete...Merging image diff: 64% complete...Merging image diff: 65% complete...Merging image diff: 66% complete...Merging image diff: 67% complete...Merging image diff: 83% complete...Merging image diff: 100% complete...done.
VM 101 - Removing backup '/mnt/pve/rbd-backup//barc/101/daily/190601000503rbd.vm-101-disk-1.diff'
VM 101 - Removing backup '/mnt/pve/rbd-backup//barc/101/daily/190602000503rbd.vm-101-disk-1.diff'
VM 101 - ======== Start backup ========
VM 101 - -------- Snapshots Disks --------
VM 101 - Creating snapshot rbd/vm-101-disk-0@barcdaily190604000503
rbd: failed to create snapshot: (17) File exists
ERROR: rbd snap create rbd/vm-101-disk-0@barcdaily190604000503
VM 101 - -------- Export images --------
VM 101 - -------- Copy config --------
VM 101 - -------- Merge diff file --------
VM 102 - ======== Start backup ========
VM 102 - -------- Snapshots Disks --------
VM 102 - Creating snapshot rbd/vm-102-disk-0@barcdaily190604000503
VM 102 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-102-disk-0@barcdaily190604000503 <<< !!!!!
VM 102 - -------- Export images --------
VM 102 - Export diff '/mnt/pve/rbd-backup//barc/102/daily/190604000503rbd.vm-102-disk-0.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 5% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 9% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 12% complete...Exporting image: 14% complete...Exporting image: 15% complete...Exporting image: 16% complete...Exporting image: 17% complete...Exporting image: 18% complete...Exporting image: 19% complete...Exporting image: 20% complete...Exporting image: 21% complete...Exporting image: 22% complete...Exporting image: 23% complete...Exporting image: 24% complete...Exporting image: 25% complete...Exporting image: 26% complete...Exporting image: 27% complete...Exporting image: 28% complete...Exporting image: 29% complete...Exporting image: 30% complete...Exporting image: 31% complete...Exporting image:
32% complete...Exporting image: 33% complete...Exporting image: 34% complete...Exporting image: 35% complete...Exporting image: 36% complete...Exporting image: 37% complete...Exporting image: 38% complete...Exporting image: 39% complete...Exporting image: 40% complete...Exporting image: 41% complete...Exporting image: 42% complete...Exporting image: 43% complete...Exporting image: 45% complete...Exporting image: 46% complete...Exporting image: 47% complete...Exporting image: 48% complete...Exporting image: 49% complete...Exporting image: 50% complete...Exporting image: 51% complete...Exporting image: 52% complete...Exporting image: 53% complete...Exporting image: 54% complete...Exporting image: 55% complete...Exporting image: 56% complete...Exporting image: 57% complete...Exporting image: 58% complete...Exporting image: 59% complete...Exporting image: 60% complete...Exporting image: 61% complete...Exporting image: 62% complete...Exporting image: 63% co
mplete...Exporting image: 64% complete...Exporting image: 65% complete...Exporting image: 66% complete...Exporting image: 67% complete...Exporting image: 68% complete...Exporting image: 69% complete...Exporting image: 70% complete...Exporting image: 71% complete...Exporting image: 72% complete...Exporting image: 73% complete...Exporting image: 74% complete...Exporting image: 75% complete...Exporting image: 76% complete...Exporting image: 77% complete...Exporting image: 78% complete...Exporting image: 79% complete...Exporting image: 80% complete...Exporting image: 81% complete...Exporting image: 82% complete...Exporting image: 83% complete...Exporting image: 84% complete...Exporting image: 85% complete...Exporting image: 86% complete...Exporting image: 87% complete...Exporting image: 88% complete...Exporting image: 89% complete...Exporting image: 90% complete...Exporting image: 91% complete...Exporting image: 92% complete...Exporting image: 93% complete.
..Exporting image: 94% complete...Exporting image: 95% complete...Exporting image: 96% complete...Exporting image: 97% complete...Exporting image: 98% complete...Exporting image: 99% complete...Exporting image: 100% complete...done.
VM 102 - Size 17G
VM 102 - Remove snapshot rbd/vm-102-disk-0@barcdaily190603000503
Removing snap: 100% complete...done.
VM 102 - -------- Copy config --------
VM 102 - -------- Merge diff file --------
VM 102 - Merge diff backup '190601000503rbd.vm-102-disk-0.diff' '190602000503rbd.vm-102-disk-0.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 16% complete...Merging image diff: 17% complete...Merging image diff: 18% complete...Merging image diff: 19% complete...Merging image diff: 20% complete...Merging image diff: 21% complete...Merging image diff: 22% complete...Merging image diff: 23% complete...Merging image diff: 24% complete...Merging image diff: 25% complete...Merging image diff: 26% complete...Merging image diff: 27% complete...Merging image diff: 28% complete..
.Merging image diff: 29% complete...Merging image diff: 30% complete...Merging image diff: 31% complete...Merging image diff: 32% complete...Merging image diff: 33% complete...Merging image diff: 34% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 37% complete...Merging image diff: 38% complete...Merging image diff: 39% complete...Merging image diff: 40% complete...Merging image diff: 41% complete...Merging image diff: 42% complete...Merging image diff: 43% complete...Merging image diff: 44% complete...Merging image diff: 45% complete...Merging image diff: 46% complete...Merging image diff: 47% complete...Merging image diff: 48% complete...Merging image diff: 49% complete...Merging image diff: 50% complete...Merging image diff: 51% complete...Merging image diff: 52% complete...Merging image diff: 53% complete...Merging image diff: 54% complete...Merging image diff: 55% complete...Merging image diff: 56%
complete...Merging image diff: 57% complete...Merging image diff: 58% complete...Merging image diff: 59% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 62% complete...Merging image diff: 63% complete...Merging image diff: 64% complete...Merging image diff: 65% complete...Merging image diff: 66% complete...Merging image diff: 67% complete...Merging image diff: 68% complete...Merging image diff: 69% complete...Merging image diff: 70% complete...Merging image diff: 71% complete...Merging image diff: 72% complete...Merging image diff: 73% complete...Merging image diff: 74% complete...Merging image diff: 75% complete...Merging image diff: 76% complete...Merging image diff: 77% complete...Merging image diff: 78% complete...Merging image diff: 79% complete...Merging image diff: 80% complete...Merging image diff: 81% complete...Merging image diff: 82% complete...Merging image diff: 83% complete...Merging imag
e diff: 84% complete...Merging image diff: 85% complete...Merging image diff: 86% complete...Merging image diff: 87% complete...Merging image diff: 88% complete...Merging image diff: 89% complete...Merging image diff: 90% complete...Merging image diff: 91% complete...Merging image diff: 92% complete...Merging image diff: 93% complete...Merging image diff: 94% complete...Merging image diff: 95% complete...Merging image diff: 96% complete...Merging image diff: 97% complete...Merging image diff: 98% complete...Merging image diff: 99% complete...Merging image diff: 100% complete...done.
VM 102 - Removing backup '/mnt/pve/rbd-backup//barc/102/daily/190601000503rbd.vm-102-disk-0.diff'
VM 102 - Removing backup '/mnt/pve/rbd-backup//barc/102/daily/190602000503rbd.vm-102-disk-0.diff'
VM 103 - ======== Start backup ========
VM 103 - -------- Snapshots Disks --------
VM 103 - Creating snapshot rbd/vm-103-disk-0@barcdaily190604000503
VM 103 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-103-disk-0@barcdaily190604000503 <<< !!!!!
VM 103 - -------- Export images --------
VM 103 - Export diff '/mnt/pve/rbd-backup//barc/103/daily/190604000503rbd.vm-103-disk-0.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 5% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 9% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 12% complete...Exporting image: 13% complete...Exporting image: 14% complete...Exporting image: 15% complete...Exporting image: 16% complete...Exporting image: 17% complete...Exporting image: 18% complete...Exporting image: 19% complete...Exporting image: 20% complete...Exporting image: 21% complete...Exporting image: 22% complete...Exporting image: 23% complete...Exporting image: 24% complete...Exporting image: 25% complete...Exporting image: 26% complete...Exporting image: 27% complete...Exporting image: 28% complete...Exporting image: 29% complete...Exporting image: 30% complete...Exporting image:
31% complete...Exporting image: 32% complete...Exporting image: 34% complete...Exporting image: 35% complete...Exporting image: 36% complete...Exporting image: 37% complete...Exporting image: 38% complete...Exporting image: 50% complete...Exporting image: 100% complete...done.
VM 103 - Size 2.4G
VM 103 - Remove snapshot rbd/vm-103-disk-0@barcdaily190603000503
Removing snap: 100% complete...done.
VM 103 - -------- Copy config --------
VM 103 - -------- Merge diff file --------
VM 103 - Merge diff backup '190601000503rbd.vm-103-disk-0.diff' '190602000503rbd.vm-103-disk-0.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 16% complete...Merging image diff: 17% complete...Merging image diff: 18% complete...Merging image diff: 19% complete...Merging image diff: 20% complete...Merging image diff: 21% complete...Merging image diff: 22% complete...Merging image diff: 23% complete...Merging image diff: 24% complete...Merging image diff: 25% complete...Merging image diff: 26% complete...Merging image diff: 27% complete...Merging image diff: 28% complete..
.Merging image diff: 29% complete...Merging image diff: 30% complete...Merging image diff: 31% complete...Merging image diff: 33% complete...Merging image diff: 34% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 37% complete...Merging image diff: 38% complete...Merging image diff: 44% complete...Merging image diff: 50% complete...Merging image diff: 75% complete...Merging image diff: 81% complete...Merging image diff: 99% complete...Merging image diff: 100% complete...done.
VM 103 - Removing backup '/mnt/pve/rbd-backup//barc/103/daily/190601000503rbd.vm-103-disk-0.diff'
VM 103 - Removing backup '/mnt/pve/rbd-backup//barc/103/daily/190602000503rbd.vm-103-disk-0.diff'
VM 104 - ======== Start backup ========
VM 104 - -------- Snapshots Disks --------
VM 104 - Creating snapshot rbd/vm-104-disk-0@barcdaily190604000503
VM 104 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-104-disk-0@barcdaily190604000503 <<< !!!!!
VM 104 - Creating snapshot rbd/vm-104-disk-1@barcdaily190604000503
VM 104 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-104-disk-1@barcdaily190604000503 <<< !!!!!
VM 104 - Creating snapshot rbd/vm-104-disk-2@barcdaily190604000503
VM 104 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-104-disk-2@barcdaily190604000503 <<< !!!!!
VM 104 - Creating snapshot rbd/vm-104-disk-3@barcdaily190604000503
VM 104 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-104-disk-3@barcdaily190604000503 <<< !!!!!
VM 104 - -------- Export images --------
VM 104 - Export diff '/mnt/pve/rbd-backup//barc/104/daily/190604000503rbd.vm-104-disk-0.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 5% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 9% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 12% complete...Exporting image: 14% complete...Exporting image: 15% complete...Exporting image: 16% complete...Exporting image: 17% complete...Exporting image: 18% complete...Exporting image: 19% complete...Exporting image: 20% complete...Exporting image: 21% complete...Exporting image: 22% complete...Exporting image: 23% complete...Exporting image: 24% complete...Exporting image: 25% complete...Exporting image: 26% complete...Exporting image: 27% complete...Exporting image: 28% complete...Exporting image: 29% complete...Exporting image: 30% complete...Exporting image: 31% complete...Exporting image:
32% complete...Exporting image: 33% complete...Exporting image: 34% complete...Exporting image: 35% complete...Exporting image: 36% complete...Exporting image: 37% complete...Exporting image: 38% complete...Exporting image: 39% complete...Exporting image: 40% complete...Exporting image: 41% complete...Exporting image: 42% complete...Exporting image: 43% complete...Exporting image: 44% complete...Exporting image: 45% complete...Exporting image: 46% complete...Exporting image: 47% complete...Exporting image: 48% complete...Exporting image: 49% complete...Exporting image: 50% complete...Exporting image: 51% complete...Exporting image: 52% complete...Exporting image: 53% complete...Exporting image: 54% complete...Exporting image: 55% complete...Exporting image: 56% complete...Exporting image: 57% complete...Exporting image: 58% complete...Exporting image: 59% complete...Exporting image: 60% complete...Exporting image: 61% complete...Exporting image: 62% co
mplete...Exporting image: 63% complete...Exporting image: 64% complete...Exporting image: 69% complete...Exporting image: 71% complete...Exporting image: 72% complete...Exporting image: 73% complete...Exporting image: 74% complete...Exporting image: 75% complete...Exporting image: 76% complete...Exporting image: 77% complete...Exporting image: 78% complete...Exporting image: 79% complete...Exporting image: 80% complete...Exporting image: 81% complete...Exporting image: 83% complete...Exporting image: 84% complete...Exporting image: 100% complete...done.
VM 104 - Size 1.7G
VM 104 - Remove snapshot rbd/vm-104-disk-0@barcdaily190603000503
Removing snap: 100% complete...done.
VM 104 - Export diff '/mnt/pve/rbd-backup//barc/104/daily/190604000503rbd.vm-104-disk-1.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 4% complete...Exporting image: 35% complete...Exporting image: 36% complete...Exporting image: 51% complete...Exporting image: 52% complete...Exporting image: 55% complete...Exporting image: 60% complete...Exporting image: 61% complete...Exporting image: 62% complete...Exporting image: 63% complete...Exporting image: 66% complete...Exporting image: 68% complete...Exporting image: 72% complete...Exporting image: 100% complete...done.
VM 104 - Size 5.0G
VM 104 - Remove snapshot rbd/vm-104-disk-1@barcdaily190603000503
Removing snap: 100% complete...done.
VM 104 - Export diff '/mnt/pve/rbd-backup//barc/104/daily/190604000503rbd.vm-104-disk-2.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 5% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 9% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 12% complete...Exporting image: 100% complete...done.
VM 104 - Size 3.7G
VM 104 - Remove snapshot rbd/vm-104-disk-2@barcdaily190603000503
Removing snap: 100% complete...done.
VM 104 - Export diff '/mnt/pve/rbd-backup//barc/104/daily/190604000503rbd.vm-104-disk-3.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 9% complete...Exporting image: 10% complete...Exporting image: 83% complete...Exporting image: 84% complete...Exporting image: 87% complete...Exporting image: 88% complete...Exporting image: 89% complete...Exporting image: 90% complete...Exporting image: 91% complete...Exporting image: 95% complete...Exporting image: 99% complete...Exporting image: 100% complete...done.
VM 104 - Size 6.5G
VM 104 - Remove snapshot rbd/vm-104-disk-3@barcdaily190603000503
Removing snap: 100% complete...done.
VM 104 - -------- Copy config --------
VM 104 - -------- Merge diff file --------
VM 104 - Merge diff backup '190601000503rbd.vm-104-disk-0.diff' '190602000503rbd.vm-104-disk-0.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 16% complete...Merging image diff: 17% complete...Merging image diff: 18% complete...Merging image diff: 19% complete...Merging image diff: 20% complete...Merging image diff: 21% complete...Merging image diff: 22% complete...Merging image diff: 23% complete...Merging image diff: 24% complete...Merging image diff: 25% complete...Merging image diff: 26% complete...Merging image diff: 27% complete...Merging image diff: 28% complete..
.Merging image diff: 29% complete...Merging image diff: 30% complete...Merging image diff: 31% complete...Merging image diff: 32% complete...Merging image diff: 33% complete...Merging image diff: 34% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 37% complete...Merging image diff: 38% complete...Merging image diff: 39% complete...Merging image diff: 40% complete...Merging image diff: 41% complete...Merging image diff: 42% complete...Merging image diff: 43% complete...Merging image diff: 44% complete...Merging image diff: 45% complete...Merging image diff: 46% complete...Merging image diff: 47% complete...Merging image diff: 48% complete...Merging image diff: 49% complete...Merging image diff: 50% complete...Merging image diff: 51% complete...Merging image diff: 52% complete...Merging image diff: 53% complete...Merging image diff: 54% complete...Merging image diff: 55% complete...Merging image diff: 56%
complete...Merging image diff: 57% complete...Merging image diff: 58% complete...Merging image diff: 59% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 62% complete...Merging image diff: 63% complete...Merging image diff: 64% complete...Merging image diff: 66% complete...Merging image diff: 67% complete...Merging image diff: 68% complete...Merging image diff: 69% complete...Merging image diff: 70% complete...Merging image diff: 71% complete...Merging image diff: 72% complete...Merging image diff: 73% complete...Merging image diff: 74% complete...Merging image diff: 75% complete...Merging image diff: 76% complete...Merging image diff: 77% complete...Merging image diff: 78% complete...Merging image diff: 79% complete...Merging image diff: 80% complete...Merging image diff: 81% complete...Merging image diff: 82% complete...Merging image diff: 83% complete...Merging image diff: 84% complete...Merging imag
e diff: 92% complete...Merging image diff: 100% complete...done.
VM 104 - Removing backup '/mnt/pve/rbd-backup//barc/104/daily/190601000503rbd.vm-104-disk-0.diff'
VM 104 - Removing backup '/mnt/pve/rbd-backup//barc/104/daily/190602000503rbd.vm-104-disk-0.diff'
VM 104 - Merge diff backup '190601000503rbd.vm-104-disk-1.diff' '190602000503rbd.vm-104-disk-1.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 20% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 44% complete...Merging image diff: 50% complete...Merging image diff: 51% complete...Merging image diff: 52% complete...Merging image diff: 54% complete...Merging image diff: 55% complete...Merging image diff: 57% complete...Merging image diff: 58% complete...Merging image diff: 59% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 64% complete...Merging image diff: 65% complete...Merging image diff: 66% complete...Merging image diff: 67% complete...Merging image diff: 68% complete...Merging image diff: 70% complete...Merging image diff: 71% complete...Merging image diff: 72% complete...Merging image diff: 86% complete...Merging image diff: 100% comp
lete...done.
VM 104 - Removing backup '/mnt/pve/rbd-backup//barc/104/daily/190601000503rbd.vm-104-disk-1.diff'
VM 104 - Removing backup '/mnt/pve/rbd-backup//barc/104/daily/190602000503rbd.vm-104-disk-1.diff'
VM 104 - Merge diff backup '190601000503rbd.vm-104-disk-2.diff' '190602000503rbd.vm-104-disk-2.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 56% complete...Merging image diff: 57% complete...Merging image diff: 74% complete...Merging image diff: 75% complete...Merging image diff: 100% complete...done.
VM 104 - Removing backup '/mnt/pve/rbd-backup//barc/104/daily/190601000503rbd.vm-104-disk-2.diff'
VM 104 - Removing backup '/mnt/pve/rbd-backup//barc/104/daily/190602000503rbd.vm-104-disk-2.diff'
VM 104 - Merge diff backup '190601000503rbd.vm-104-disk-3.diff' '190602000503rbd.vm-104-disk-3.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 6% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 30% complete...Merging image diff: 66% complete...Merging image diff: 67% complete...Merging image diff: 68% complete...Merging image diff: 83% complete...Merging image diff: 84% complete...Merging image diff: 86% complete...Merging image diff: 87% complete...Merging image diff: 88% complete...Merging image diff: 89% complete...Merging image diff: 90% complete...Merging image diff: 91% complete...Merging image diff: 93% complete...Merging image diff: 94% complete...Merging image diff: 95% complete...Merging image diff: 97% complete...Merging image diff: 99% complete...Merging image diff: 100% complete...Merging image diff: 100% complete...done.
VM 104 - Removing backup '/mnt/pve/rbd-backup//barc/104/daily/190601000503rbd.vm-104-disk-3.diff'
VM 104 - Removing backup '/mnt/pve/rbd-backup//barc/104/daily/190602000503rbd.vm-104-disk-3.diff'
VM 104 - ======== Start backup ========
VM 104 - -------- Snapshots Disks --------
VM 104 - Creating snapshot rbd/vm-104-disk-0@barcdaily190604000503
rbd: failed to create snapshot: (17) File exists
ERROR: rbd snap create rbd/vm-104-disk-0@barcdaily190604000503
VM 104 - -------- Export images --------
VM 104 - -------- Copy config --------
VM 104 - -------- Merge diff file --------
VM 104 - ======== Start backup ========
VM 104 - -------- Snapshots Disks --------
VM 104 - Creating snapshot rbd/vm-104-disk-0@barcdaily190604000503
rbd: failed to create snapshot: (17) File exists
ERROR: rbd snap create rbd/vm-104-disk-0@barcdaily190604000503
VM 104 - -------- Export images --------
VM 104 - -------- Copy config --------
VM 104 - -------- Merge diff file --------
VM 104 - ======== Start backup ========
VM 104 - -------- Snapshots Disks --------
VM 104 - Creating snapshot rbd/vm-104-disk-0@barcdaily190604000503
rbd: failed to create snapshot: (17) File exists
ERROR: rbd snap create rbd/vm-104-disk-0@barcdaily190604000503
VM 104 - -------- Export images --------
VM 104 - -------- Copy config --------
VM 104 - -------- Merge diff file --------
VM 105 - ======== Start backup ========
VM 105 - -------- Snapshots Disks --------
VM 105 - Creating snapshot rbd/vm-105-disk-0@barcdaily190604000503
VM 105 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-105-disk-0@barcdaily190604000503 <<< !!!!!
VM 105 - Creating snapshot rbd/vm-105-disk-1@barcdaily190604000503
VM 105 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-105-disk-1@barcdaily190604000503 <<< !!!!!
VM 105 - Creating snapshot rbd/vm-105-disk-2@barcdaily190604000503
VM 105 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-105-disk-2@barcdaily190604000503 <<< !!!!!
VM 105 - Creating snapshot rbd/vm-105-disk-3@barcdaily190604000503
VM 105 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-105-disk-3@barcdaily190604000503 <<< !!!!!
VM 105 - -------- Export images --------
VM 105 - Export diff '/mnt/pve/rbd-backup//barc/105/daily/190604000503rbd.vm-105-disk-0.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 5% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 9% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 12% complete...Exporting image: 13% complete...Exporting image: 14% complete...Exporting image: 15% complete...Exporting image: 16% complete...Exporting image: 17% complete...Exporting image: 18% complete...Exporting image: 19% complete...Exporting image: 20% complete...Exporting image: 21% complete...Exporting image: 22% complete...Exporting image: 23% complete...Exporting image: 24% complete...Exporting image: 25% complete...Exporting image: 26% complete...Exporting image: 27% complete...Exporting image: 28% complete...Exporting image: 29% complete...Exporting image: 30% complete...Exporting image:
31% complete...Exporting image: 32% complete...Exporting image: 33% complete...Exporting image: 34% complete...Exporting image: 35% complete...Exporting image: 36% complete...Exporting image: 37% complete...Exporting image: 38% complete...Exporting image: 39% complete...Exporting image: 40% complete...Exporting image: 43% complete...Exporting image: 44% complete...Exporting image: 46% complete...Exporting image: 48% complete...Exporting image: 49% complete...Exporting image: 50% complete...Exporting image: 51% complete...Exporting image: 53% complete...Exporting image: 54% complete...Exporting image: 57% complete...Exporting image: 58% complete...Exporting image: 60% complete...Exporting image: 62% complete...Exporting image: 63% complete...Exporting image: 64% complete...Exporting image: 65% complete...Exporting image: 66% complete...Exporting image: 68% complete...Exporting image: 69% complete...Exporting image: 71% complete...Exporting image: 73% co
mplete...Exporting image: 75% complete...Exporting image: 76% complete...Exporting image: 77% complete...Exporting image: 79% complete...Exporting image: 80% complete...Exporting image: 81% complete...Exporting image: 84% complete...Exporting image: 85% complete...Exporting image: 86% complete...Exporting image: 87% complete...Exporting image: 88% complete...Exporting image: 89% complete...Exporting image: 90% complete...Exporting image: 91% complete...Exporting image: 92% complete...Exporting image: 93% complete...Exporting image: 94% complete...Exporting image: 95% complete...Exporting image: 96% complete...Exporting image: 97% complete...Exporting image: 98% complete...Exporting image: 99% complete...Exporting image: 100% complete...done.
VM 105 - Size 2.8G
VM 105 - Remove snapshot rbd/vm-105-disk-0@barcdaily190603000503
Removing snap: 100% complete...done.
VM 105 - Export diff '/mnt/pve/rbd-backup//barc/105/daily/190604000503rbd.vm-105-disk-1.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 5% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 9% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 12% complete...Exporting image: 13% complete...Exporting image: 14% complete...Exporting image: 15% complete...Exporting image: 16% complete...Exporting image: 17% complete...Exporting image: 18% complete...Exporting image: 19% complete...Exporting image: 20% complete...Exporting image: 21% complete...Exporting image: 22% complete...Exporting image: 23% complete...Exporting image: 24% complete...Exporting image: 25% complete...Exporting image: 26% complete...Exporting image: 27% complete...Exporting image: 28% complete...Exporting image: 29% complete...Exporting image: 30% complete...Exporting image:
31% complete...Exporting image: 33% complete...Exporting image: 36% complete...Exporting image: 38% complete...Exporting image: 39% complete...Exporting image: 41% complete...Exporting image: 42% complete...Exporting image: 43% complete...Exporting image: 44% complete...Exporting image: 45% complete...Exporting image: 47% complete...Exporting image: 51% complete...Exporting image: 52% complete...Exporting image: 53% complete...Exporting image: 54% complete...Exporting image: 55% complete...Exporting image: 56% complete...Exporting image: 57% complete...Exporting image: 58% complete...Exporting image: 59% complete...Exporting image: 60% complete...Exporting image: 61% complete...Exporting image: 63% complete...Exporting image: 64% complete...Exporting image: 65% complete...Exporting image: 66% complete...Exporting image: 67% complete...Exporting image: 69% complete...Exporting image: 71% complete...Exporting image: 72% complete...Exporting image: 73% co
mplete...Exporting image: 74% complete...Exporting image: 75% complete...Exporting image: 76% complete...Exporting image: 77% complete...Exporting image: 78% complete...Exporting image: 79% complete...Exporting image: 80% complete...Exporting image: 81% complete...Exporting image: 82% complete...Exporting image: 83% complete...Exporting image: 84% complete...Exporting image: 85% complete...Exporting image: 86% complete...Exporting image: 87% complete...Exporting image: 88% complete...Exporting image: 89% complete...Exporting image: 90% complete...Exporting image: 91% complete...Exporting image: 92% complete...Exporting image: 93% complete...Exporting image: 94% complete...Exporting image: 95% complete...Exporting image: 96% complete...Exporting image: 97% complete...Exporting image: 98% complete...Exporting image: 100% complete...done.
VM 105 - Size 3.2G
VM 105 - Remove snapshot rbd/vm-105-disk-1@barcdaily190603000503
Removing snap: 100% complete...done.
VM 105 - Export diff '/mnt/pve/rbd-backup//barc/105/daily/190604000503rbd.vm-105-disk-2.diff'
Exporting image: 1% complete...Exporting image: 16% complete...Exporting image: 17% complete...Exporting image: 28% complete...Exporting image: 29% complete...Exporting image: 30% complete...Exporting image: 32% complete...Exporting image: 34% complete...Exporting image: 35% complete...Exporting image: 36% complete...Exporting image: 37% complete...Exporting image: 38% complete...Exporting image: 45% complete...Exporting image: 46% complete...Exporting image: 100% complete...done.
VM 105 - Size 259M
VM 105 - Remove snapshot rbd/vm-105-disk-2@barcdaily190603000503
Removing snap: 100% complete...done.
VM 105 - Export diff '/mnt/pve/rbd-backup//barc/105/daily/190604000503rbd.vm-105-disk-3.diff'
Exporting image: 47% complete...Exporting image: 50% complete...Exporting image: 56% complete...Exporting image: 60% complete...Exporting image: 61% complete...Exporting image: 100% complete...done.
VM 105 - Size 236M
VM 105 - Remove snapshot rbd/vm-105-disk-3@barcdaily190603000503
Removing snap: 100% complete...done.
VM 105 - -------- Copy config --------
VM 105 - -------- Merge diff file --------
VM 105 - Merge diff backup '190601000503rbd.vm-105-disk-0.diff' '190602000503rbd.vm-105-disk-0.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 16% complete...Merging image diff: 17% complete...Merging image diff: 18% complete...Merging image diff: 19% complete...Merging image diff: 20% complete...Merging image diff: 21% complete...Merging image diff: 22% complete...Merging image diff: 23% complete...Merging image diff: 24% complete...Merging image diff: 25% complete...Merging image diff: 26% complete...Merging image diff: 27% complete...Merging image diff: 28% complete..
.Merging image diff: 29% complete...Merging image diff: 30% complete...Merging image diff: 31% complete...Merging image diff: 32% complete...Merging image diff: 33% complete...Merging image diff: 34% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 37% complete...Merging image diff: 38% complete...Merging image diff: 39% complete...Merging image diff: 40% complete...Merging image diff: 41% complete...Merging image diff: 42% complete...Merging image diff: 43% complete...Merging image diff: 44% complete...Merging image diff: 45% complete...Merging image diff: 46% complete...Merging image diff: 47% complete...Merging image diff: 48% complete...Merging image diff: 49% complete...Merging image diff: 50% complete...Merging image diff: 51% complete...Merging image diff: 52% complete...Merging image diff: 53% complete...Merging image diff: 54% complete...Merging image diff: 55% complete...Merging image diff: 56%
complete...Merging image diff: 57% complete...Merging image diff: 58% complete...Merging image diff: 59% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 62% complete...Merging image diff: 63% complete...Merging image diff: 64% complete...Merging image diff: 65% complete...Merging image diff: 66% complete...Merging image diff: 67% complete...Merging image diff: 68% complete...Merging image diff: 69% complete...Merging image diff: 70% complete...Merging image diff: 71% complete...Merging image diff: 72% complete...Merging image diff: 73% complete...Merging image diff: 74% complete...Merging image diff: 75% complete...Merging image diff: 76% complete...Merging image diff: 77% complete...Merging image diff: 78% complete...Merging image diff: 79% complete...Merging image diff: 80% complete...Merging image diff: 81% complete...Merging image diff: 82% complete...Merging image diff: 83% complete...Merging imag
e diff: 84% complete...Merging image diff: 85% complete...Merging image diff: 86% complete...Merging image diff: 87% complete...Merging image diff: 88% complete...Merging image diff: 89% complete...Merging image diff: 90% complete...Merging image diff: 91% complete...Merging image diff: 92% complete...Merging image diff: 93% complete...Merging image diff: 94% complete...Merging image diff: 95% complete...Merging image diff: 96% complete...Merging image diff: 97% complete...Merging image diff: 98% complete...Merging image diff: 99% complete...Merging image diff: 100% complete...done.
VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190601000503rbd.vm-105-disk-0.diff'
VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190602000503rbd.vm-105-disk-0.diff'
VM 105 - Merge diff backup '190601000503rbd.vm-105-disk-1.diff' '190602000503rbd.vm-105-disk-1.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 16% complete...Merging image diff: 17% complete...Merging image diff: 18% complete...Merging image diff: 19% complete...Merging image diff: 20% complete...Merging image diff: 21% complete...Merging image diff: 22% complete...Merging image diff: 23% complete...Merging image diff: 24% complete...Merging image diff: 25% complete...Merging image diff: 26% complete...Merging image diff: 27% complete...Merging image diff: 28% complete..
.Merging image diff: 29% complete...Merging image diff: 30% complete...Merging image diff: 31% complete...Merging image diff: 32% complete...Merging image diff: 33% complete...Merging image diff: 34% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 37% complete...Merging image diff: 38% complete...Merging image diff: 39% complete...Merging image diff: 40% complete...Merging image diff: 41% complete...Merging image diff: 42% complete...Merging image diff: 43% complete...Merging image diff: 44% complete...Merging image diff: 45% complete...Merging image diff: 46% complete...Merging image diff: 47% complete...Merging image diff: 48% complete...Merging image diff: 49% complete...Merging image diff: 50% complete...Merging image diff: 51% complete...Merging image diff: 52% complete...Merging image diff: 53% complete...Merging image diff: 54% complete...Merging image diff: 55% complete...Merging image diff: 56%
complete...Merging image diff: 57% complete...Merging image diff: 58% complete...Merging image diff: 59% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 62% complete...Merging image diff: 63% complete...Merging image diff: 64% complete...Merging image diff: 65% complete...Merging image diff: 66% complete...Merging image diff: 67% complete...Merging image diff: 68% complete...Merging image diff: 69% complete...Merging image diff: 70% complete...Merging image diff: 71% complete...Merging image diff: 72% complete...Merging image diff: 73% complete...Merging image diff: 74% complete...Merging image diff: 75% complete...Merging image diff: 76% complete...Merging image diff: 77% complete...Merging image diff: 78% complete...Merging image diff: 79% complete...Merging image diff: 80% complete...Merging image diff: 81% complete...Merging image diff: 82% complete...Merging image diff: 83% complete...Merging imag
e diff: 84% complete...Merging image diff: 85% complete...Merging image diff: 86% complete...Merging image diff: 87% complete...Merging image diff: 88% complete...Merging image diff: 89% complete...Merging image diff: 90% complete...Merging image diff: 91% complete...Merging image diff: 92% complete...Merging image diff: 93% complete...Merging image diff: 94% complete...Merging image diff: 95% complete...Merging image diff: 96% complete...Merging image diff: 97% complete...Merging image diff: 98% complete...Merging image diff: 99% complete...Merging image diff: 100% complete...done.
VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190601000503rbd.vm-105-disk-1.diff'
VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190602000503rbd.vm-105-disk-1.diff'
VM 105 - Merge diff backup '190601000503rbd.vm-105-disk-2.diff' '190602000503rbd.vm-105-disk-2.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 22% complete...Merging image diff: 23% complete...Merging image diff: 24% complete...Merging image diff: 25% complete...Merging image diff: 26% complete...Merging image diff: 27% complete...Merging image diff: 28% complete...Merging image diff: 29% complete...Merging image diff: 30% complete...Merging image diff: 31% complete...Merging image diff: 32% complete...Merging image diff: 33% complete...Merging image diff: 34% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 37% complete...Merging image diff: 38% complete...Merging image diff: 42% compl
ete...Merging image diff: 45% complete...Merging image diff: 72% complete...Merging image diff: 73% complete...Merging image diff: 79% complete...Merging image diff: 100% complete...done.
VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190601000503rbd.vm-105-disk-2.diff'
VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190602000503rbd.vm-105-disk-2.diff'
VM 105 - Merge diff backup '190601000503rbd.vm-105-disk-3.diff' '190602000503rbd.vm-105-disk-3.diff'
Merging image diff: 12% complete...Merging image diff: 36% complete...Merging image diff: 37% complete...Merging image diff: 38% complete...Merging image diff: 42% complete...Merging image diff: 43% complete...Merging image diff: 44% complete...Merging image diff: 45% complete...Merging image diff: 46% complete...Merging image diff: 47% complete...Merging image diff: 48% complete...Merging image diff: 50% complete...Merging image diff: 51% complete...Merging image diff: 54% complete...Merging image diff: 55% complete...Merging image diff: 56% complete...Merging image diff: 58% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 63% complete...Merging image diff: 82% complete...Merging image diff: 83% complete...Merging image diff: 100% complete...done.
VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190601000503rbd.vm-105-disk-3.diff'
VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190602000503rbd.vm-105-disk-3.diff'
VM 105 - ======== Start backup ========
VM 105 - -------- Snapshots Disks --------
VM 105 - Creating snapshot rbd/vm-105-disk-0@barcdaily190604000503
rbd: failed to create snapshot: (17) File exists
ERROR: rbd snap create rbd/vm-105-disk-0@barcdaily190604000503
VM 105 - -------- Export images --------
VM 105 - -------- Copy config --------
VM 105 - -------- Merge diff file --------
VM 105 - ======== Start backup ========
VM 105 - -------- Snapshots Disks --------
VM 105 - Creating snapshot rbd/vm-105-disk-0@barcdaily190604000503
rbd: failed to create snapshot: (17) File exists
ERROR: rbd snap create rbd/vm-105-disk-0@barcdaily190604000503
VM 105 - -------- Export images --------
VM 105 - -------- Copy config --------
VM 105 - -------- Merge diff file --------
VM 105 - ======== Start backup ========
VM 105 - -------- Snapshots Disks --------
VM 105 - Creating snapshot rbd/vm-105-disk-0@barcdaily190604000503
rbd: failed to create snapshot: (17) File exists
ERROR: rbd snap create rbd/vm-105-disk-0@barcdaily190604000503
VM 105 - -------- Export images --------
VM 105 - -------- Copy config --------
VM 105 - -------- Merge diff file --------
VM 106 - ======== Start backup ========
VM 106 - -------- Snapshots Disks --------
VM 106 - Creating snapshot rbd/vm-106-disk-0@barcdaily190604000503
VM 106 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-106-disk-0@barcdaily190604000503 <<< !!!!!
VM 106 - -------- Export images --------
VM 106 - Export diff '/mnt/pve/rbd-backup//barc/106/daily/190604000503rbd.vm-106-disk-0.diff'
Exporting image: 4% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 14% complete...Exporting image: 17% complete...Exporting image: 20% complete...Exporting image: 24% complete...Exporting image: 27% complete...Exporting image: 30% complete...Exporting image: 34% complete...Exporting image: 35% complete...Exporting image: 37% complete...Exporting image: 38% complete...Exporting image: 39% complete...Exporting image: 40% complete...Exporting image: 41% complete...Exporting image: 43% complete...Exporting image: 44% complete...Exporting image: 47% complete...Exporting image: 60% complete...Exporting image: 67% complete...Exporting image: 70% complete...Exporting image: 74% complete...Exporting image: 75% complete...Exporting image: 76% complete...Exporting image: 77% complete...Exporting image: 79% complete...Exporting image: 80% complete...Exporting
image: 81% complete...Exporting image: 82% complete...Exporting image: 83% complete...Exporting image: 84% complete...Exporting image: 87% complete...Exporting image: 90% complete...Exporting image: 94% complete...Exporting image: 100% complete...done.
VM 106 - Size 1.1G
VM 106 - Remove snapshot rbd/vm-106-disk-0@barcdaily190603000503
Removing snap: 100% complete...done.
VM 106 - -------- Copy config --------
VM 106 - -------- Merge diff file --------
VM 106 - Merge diff backup '190601000503rbd.vm-106-disk-0.diff' '190602000503rbd.vm-106-disk-0.diff'
Merging image diff: 1% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 12% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 17% complete...Merging image diff: 19% complete...Merging image diff: 20% complete...Merging image diff: 22% complete...Merging image diff: 24% complete...Merging image diff: 25% complete...Merging image diff: 27% complete...Merging image diff: 29% complete...Merging image diff: 30% complete...Merging image diff: 32% complete...Merging image diff: 34% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 37% complete...Merging image diff: 38% complete...Merging image diff: 39% complete...Merging image diff: 40% complete
...Merging image diff: 41% complete...Merging image diff: 42% complete...Merging image diff: 43% complete...Merging image diff: 44% complete...Merging image diff: 46% complete...Merging image diff: 47% complete...Merging image diff: 54% complete...Merging image diff: 60% complete...Merging image diff: 64% complete...Merging image diff: 67% complete...Merging image diff: 69% complete...Merging image diff: 70% complete...Merging image diff: 72% complete...Merging image diff: 74% complete...Merging image diff: 75% complete...Merging image diff: 76% complete...Merging image diff: 77% complete...Merging image diff: 78% complete...Merging image diff: 79% complete...Merging image diff: 80% complete...Merging image diff: 81% complete...Merging image diff: 82% complete...Merging image diff: 83% complete...Merging image diff: 84% complete...Merging image diff: 86% complete...Merging image diff: 87% complete...Merging image diff: 89% complete...Merging image diff: 9
0% complete...Merging image diff: 92% complete...Merging image diff: 94% complete...Merging image diff: 96% complete...Merging image diff: 98% complete...Merging image diff: 99% complete...Merging image diff: 100% complete...done.
VM 106 - Removing backup '/mnt/pve/rbd-backup//barc/106/daily/190601000503rbd.vm-106-disk-0.diff'
VM 106 - Removing backup '/mnt/pve/rbd-backup//barc/106/daily/190602000503rbd.vm-106-disk-0.diff'
VM 107 - ======== Start backup ========
VM 107 - -------- Snapshots Disks --------
VM 107 - Creating snapshot rbd/vm-107-disk-0@barcdaily190604000503
VM 107 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-107-disk-0@barcdaily190604000503 <<< !!!!!
VM 107 - -------- Export images --------
VM 107 - Export diff '/mnt/pve/rbd-backup//barc/107/daily/190604000503rbd.vm-107-disk-0.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 9% complete...Exporting image: 100% complete...done.
VM 107 - Size 1.1G
VM 107 - Remove snapshot rbd/vm-107-disk-0@barcdaily190603000503
Removing snap: 100% complete...done.
VM 107 - -------- Copy config --------
VM 107 - -------- Merge diff file --------
VM 107 - Merge diff backup '190601000503rbd.vm-107-disk-0.diff' '190602000503rbd.vm-107-disk-0.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 54% complete...Merging image diff: 100% complete...done.
VM 107 - Removing backup '/mnt/pve/rbd-backup//barc/107/daily/190601000503rbd.vm-107-disk-0.diff'
VM 107 - Removing backup '/mnt/pve/rbd-backup//barc/107/daily/190602000503rbd.vm-107-disk-0.diff'
VM 108 - ======== Start backup ========
VM 108 - -------- Snapshots Disks --------
VM 108 - Creating snapshot rbd/vm-108-disk-0@barcdaily190604000503
VM 108 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-108-disk-0@barcdaily190604000503 <<< !!!!!
VM 108 - Creating snapshot rbd/vm-108-disk-1@barcdaily190604000503
VM 108 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-108-disk-1@barcdaily190604000503 <<< !!!!!
VM 108 - -------- Export images --------
VM 108 - Export diff '/mnt/pve/rbd-backup//barc/108/daily/190604000503rbd.vm-108-disk-0.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 5% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 12% complete...Exporting image: 13% complete...Exporting image: 14% complete...Exporting image: 15% complete...Exporting image: 16% complete...Exporting image: 17% complete...Exporting image: 18% complete...Exporting image: 19% complete...Exporting image: 20% complete...Exporting image: 21% complete...Exporting image: 22% complete...Exporting image: 23% complete...Exporting image: 24% complete...Exporting image: 25% complete...Exporting image: 26% complete...Exporting image: 27% complete...Exporting image: 28% complete...Exporting image: 29% complete...Exporting image: 30% complete...Exporting image: 31% complete...Exporting image
: 32% complete...Exporting image: 33% complete...Exporting image: 34% complete...Exporting image: 35% complete...Exporting image: 36% complete...Exporting image: 37% complete...Exporting image: 38% complete...Exporting image: 39% complete...Exporting image: 40% complete...Exporting image: 41% complete...Exporting image: 42% complete...Exporting image: 43% complete...Exporting image: 44% complete...Exporting image: 45% complete...Exporting image: 46% complete...Exporting image: 47% complete...Exporting image: 48% complete...Exporting image: 49% complete...Exporting image: 50% complete...Exporting image: 51% complete...Exporting image: 52% complete...Exporting image: 54% complete...Exporting image: 55% complete...Exporting image: 56% complete...Exporting image: 62% complete...Exporting image: 63% complete...Exporting image: 64% complete...Exporting image: 65% complete...Exporting image: 66% complete...Exporting image: 67% complete...Exporting image: 68% c
omplete...Exporting image: 69% complete...Exporting image: 70% complete...Exporting image: 71% complete...Exporting image: 73% complete...Exporting image: 74% complete...Exporting image: 75% complete...Exporting image: 76% complete...Exporting image: 77% complete...Exporting image: 79% complete...Exporting image: 81% complete...Exporting image: 82% complete...Exporting image: 100% complete...done.
VM 108 - Size 160G
VM 108 - Remove snapshot rbd/vm-108-disk-0@barcdaily190603000503
Removing snap: 100% complete...done.
VM 108 - Export diff '/mnt/pve/rbd-backup//barc/108/daily/190604000503rbd.vm-108-disk-1.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 5% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 9% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 12% complete...Exporting image: 13% complete...Exporting image: 14% complete...Exporting image: 15% complete...Exporting image: 16% complete...Exporting image: 17% complete...Exporting image: 18% complete...Exporting image: 19% complete...Exporting image: 20% complete...Exporting image: 21% complete...Exporting image: 22% complete...Exporting image: 23% complete...Exporting image: 27% complete...Exporting image: 29% complete...Exporting image: 30% complete...Exporting image: 31% complete...Exporting image: 32% complete...Exporting image: 33% complete...Exporting image: 34% complete...Exporting image:
35% complete...Exporting image: 36% complete...Exporting image: 37% complete...Exporting image: 38% complete...Exporting image: 39% complete...Exporting image: 40% complete...Exporting image: 41% complete...Exporting image: 42% complete...Exporting image: 43% complete...Exporting image: 44% complete...Exporting image: 46% complete...Exporting image: 47% complete...Exporting image: 48% complete...Exporting image: 49% complete...Exporting image: 50% complete...Exporting image: 51% complete...Exporting image: 53% complete...Exporting image: 54% complete...Exporting image: 55% complete...Exporting image: 57% complete...Exporting image: 58% complete...Exporting image: 59% complete...Exporting image: 60% complete...Exporting image: 61% complete...Exporting image: 62% complete...Exporting image: 63% complete...Exporting image: 64% complete...Exporting image: 65% complete...Exporting image: 66% complete...Exporting image: 67% complete...Exporting image: 68% co
mplete...Exporting image: 69% complete...Exporting image: 70% complete...Exporting image: 71% complete...Exporting image: 72% complete...Exporting image: 73% complete...Exporting image: 74% complete...Exporting image: 75% complete...Exporting image: 76% complete...Exporting image: 77% complete...Exporting image: 78% complete...Exporting image: 79% complete...Exporting image: 80% complete...Exporting image: 81% complete...Exporting image: 82% complete...Exporting image: 84% complete...Exporting image: 85% complete...Exporting image: 86% complete...Exporting image: 87% complete...Exporting image: 88% complete...Exporting image: 89% complete...Exporting image: 90% complete...Exporting image: 91% complete...Exporting image: 100% complete...done.
VM 108 - Size 63G
VM 108 - Remove snapshot rbd/vm-108-disk-1@barcdaily190603000503
Removing snap: 100% complete...done.
VM 108 - -------- Copy config --------
VM 108 - -------- Merge diff file --------
VM 108 - Merge diff backup '190601000503rbd.vm-108-disk-0.diff' '190602000503rbd.vm-108-disk-0.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 16% complete...Merging image diff: 17% complete...Merging image diff: 18% complete...Merging image diff: 19% complete...Merging image diff: 20% complete...Merging image diff: 21% complete...Merging image diff: 22% complete...Merging image diff: 23% complete...Merging image diff: 24% complete...Merging image diff: 25% complete...Merging image diff: 26% complete...Merging image diff: 27% complete...Merging image diff: 28% complete..
.Merging image diff: 29% complete...Merging image diff: 30% complete...Merging image diff: 31% complete...Merging image diff: 32% complete...Merging image diff: 33% complete...Merging image diff: 34% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 37% complete...Merging image diff: 38% complete...Merging image diff: 39% complete...Merging image diff: 40% complete...Merging image diff: 41% complete...Merging image diff: 42% complete...Merging image diff: 43% complete...Merging image diff: 44% complete...Merging image diff: 45% complete...Merging image diff: 46% complete...Merging image diff: 47% complete...Merging image diff: 48% complete...Merging image diff: 49% complete...Merging image diff: 50% complete...Merging image diff: 51% complete...Merging image diff: 52% complete...Merging image diff: 53% complete...Merging image diff: 54% complete...Merging image diff: 55% complete...Merging image diff: 56%
complete...Merging image diff: 57% complete...Merging image diff: 59% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 62% complete...Merging image diff: 63% complete...Merging image diff: 64% complete...Merging image diff: 65% complete...Merging image diff: 66% complete...Merging image diff: 67% complete...Merging image diff: 68% complete...Merging image diff: 69% complete...Merging image diff: 70% complete...Merging image diff: 71% complete...Merging image diff: 72% complete...Merging image diff: 73% complete...Merging image diff: 74% complete...Merging image diff: 75% complete...Merging image diff: 76% complete...Merging image diff: 77% complete...Merging image diff: 78% complete...Merging image diff: 79% complete...Merging image diff: 80% complete...Merging image diff: 81% complete...Merging image diff: 82% complete...Merging image diff: 91% complete...Merging image diff: 100% complete...done.
VM 108 - Removing backup '/mnt/pve/rbd-backup//barc/108/daily/190601000503rbd.vm-108-disk-0.diff'
VM 108 - Removing backup '/mnt/pve/rbd-backup//barc/108/daily/190602000503rbd.vm-108-disk-0.diff'
VM 108 - Merge diff backup '190601000503rbd.vm-108-disk-1.diff' '190602000503rbd.vm-108-disk-1.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 16% complete...Merging image diff: 17% complete...Merging image diff: 18% complete...Merging image diff: 19% complete...Merging image diff: 20% complete...Merging image diff: 21% complete...Merging image diff: 22% complete...Merging image diff: 23% complete...Merging image diff: 25% complete...Merging image diff: 26% complete...Merging image diff: 27% complete...Merging image diff: 29% complete...Merging image diff: 30% complete..
.Merging image diff: 31% complete...Merging image diff: 32% complete...Merging image diff: 33% complete...Merging image diff: 34% complete...Merging image diff: 35% complete...Merging image diff: 36% complete...Merging image diff: 37% complete...Merging image diff: 38% complete...Merging image diff: 39% complete...Merging image diff: 40% complete...Merging image diff: 41% complete...Merging image diff: 42% complete...Merging image diff: 43% complete...Merging image diff: 44% complete...Merging image diff: 45% complete...Merging image diff: 46% complete...Merging image diff: 47% complete...Merging image diff: 48% complete...Merging image diff: 49% complete...Merging image diff: 50% complete...Merging image diff: 51% complete...Merging image diff: 52% complete...Merging image diff: 53% complete...Merging image diff: 54% complete...Merging image diff: 55% complete...Merging image diff: 56% complete...Merging image diff: 57% complete...Merging image diff: 58%
complete...Merging image diff: 59% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 62% complete...Merging image diff: 63% complete...Merging image diff: 64% complete...Merging image diff: 65% complete...Merging image diff: 66% complete...Merging image diff: 67% complete...Merging image diff: 68% complete...Merging image diff: 69% complete...Merging image diff: 70% complete...Merging image diff: 71% complete...Merging image diff: 72% complete...Merging image diff: 73% complete...Merging image diff: 74% complete...Merging image diff: 75% complete...Merging image diff: 76% complete...Merging image diff: 77% complete...Merging image diff: 78% complete...Merging image diff: 79% complete...Merging image diff: 80% complete...Merging image diff: 81% complete...Merging image diff: 82% complete...Merging image diff: 83% complete...Merging image diff: 84% complete...Merging image diff: 85% complete...Merging imag
e diff: 86% complete...Merging image diff: 87% complete...Merging image diff: 88% complete...Merging image diff: 89% complete...Merging image diff: 90% complete...Merging image diff: 91% complete...Merging image diff: 93% complete...Merging image diff: 95% complete...Merging image diff: 97% complete...Merging image diff: 99% complete...Merging image diff: 100% complete...done.
VM 108 - Removing backup '/mnt/pve/rbd-backup//barc/108/daily/190601000503rbd.vm-108-disk-1.diff'
VM 108 - Removing backup '/mnt/pve/rbd-backup//barc/108/daily/190602000503rbd.vm-108-disk-1.diff'
VM 108 - ======== Start backup ========
VM 108 - -------- Snapshots Disks --------
VM 108 - Creating snapshot rbd/vm-108-disk-0@barcdaily190604000503
rbd: failed to create snapshot: (17) File exists
ERROR: rbd snap create rbd/vm-108-disk-0@barcdaily190604000503
VM 108 - -------- Export images --------
VM 108 - -------- Copy config --------
VM 108 - -------- Merge diff file --------
VM 111 - ======== Start backup ========
VM 111 - -------- Snapshots Disks --------
VM 111 - Creating snapshot rbd/vm-111-disk-0@barcdaily190604000503
VM 111 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-111-disk-0@barcdaily190604000503 <<< !!!!!
VM 111 - Creating snapshot rbd/vm-111-disk-1@barcdaily190604000503
VM 111 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-111-disk-1@barcdaily190604000503 <<< !!!!!
VM 111 - -------- Export images --------
VM 111 - Export diff '/mnt/pve/rbd-backup//barc/111/daily/190604000503rbd.vm-111-disk-0.diff'
Exporting image: 1% complete...Exporting image: 2% complete...Exporting image: 3% complete...Exporting image: 4% complete...Exporting image: 5% complete...Exporting image: 6% complete...Exporting image: 7% complete...Exporting image: 8% complete...Exporting image: 9% complete...Exporting image: 10% complete...Exporting image: 11% complete...Exporting image: 12% complete...Exporting image: 13% complete...Exporting image: 14% complete...Exporting image: 15% complete...Exporting image: 16% complete...Exporting image: 17% complete...Exporting image: 18% complete...Exporting image: 19% complete...Exporting image: 20% complete...Exporting image: 21% complete...Exporting image: 22% complete...Exporting image: 23% complete...Exporting image: 24% complete...Exporting image: 26% complete...Exporting image: 27% complete...Exporting image: 28% complete...Exporting image: 29% complete...Exporting image: 30% complete...Exporting image: 67% complete...Exporting image:
68% complete...Exporting image: 69% complete...Exporting image: 70% complete...Exporting image: 71% complete...Exporting image: 72% complete...Exporting image: 73% complete...Exporting image: 74% complete...Exporting image: 75% complete...Exporting image: 76% complete...Exporting image: 77% complete...Exporting image: 78% complete...Exporting image: 79% complete...Exporting image: 80% complete...Exporting image: 81% complete...Exporting image: 82% complete...Exporting image: 83% complete...Exporting image: 84% complete...Exporting image: 100% complete...done.
VM 111 - Size 2.3G
VM 111 - Remove snapshot rbd/vm-111-disk-0@barcdaily190603000503
Removing snap: 100% complete...done.
VM 111 - Export diff '/mnt/pve/rbd-backup//barc/111/daily/190604000503rbd.vm-111-disk-1.diff'
Exporting image: 1% complete...Exporting image: 12% complete...Exporting image: 13% complete...Exporting image: 14% complete...Exporting image: 15% complete...Exporting image: 16% complete...Exporting image: 78% complete...Exporting image: 98% complete...Exporting image: 100% complete...done.
VM 111 - Size 1.3G
VM 111 - Remove snapshot rbd/vm-111-disk-1@barcdaily190603000503
Removing snap: 100% complete...done.
VM 111 - -------- Copy config --------
VM 111 - -------- Merge diff file --------
VM 111 - Merge diff backup '190601000503rbd.vm-111-disk-0.diff' '190602000503rbd.vm-111-disk-0.diff'
Merging image diff: 1% complete...Merging image diff: 2% complete...Merging image diff: 3% complete...Merging image diff: 4% complete...Merging image diff: 5% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 10% complete...Merging image diff: 11% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 16% complete...Merging image diff: 17% complete...Merging image diff: 18% complete...Merging image diff: 19% complete...Merging image diff: 20% complete...Merging image diff: 21% complete...Merging image diff: 22% complete...Merging image diff: 23% complete...Merging image diff: 24% complete...Merging image diff: 25% complete...Merging image diff: 26% complete...Merging image diff: 27% complete...Merging image diff: 28% complete..
.Merging image diff: 29% complete...Merging image diff: 30% complete...Merging image diff: 48% complete...Merging image diff: 57% complete...Merging image diff: 58% complete...Merging image diff: 59% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 62% complete...Merging image diff: 63% complete...Merging image diff: 64% complete...Merging image diff: 65% complete...Merging image diff: 66% complete...Merging image diff: 67% complete...Merging image diff: 68% complete...Merging image diff: 69% complete...Merging image diff: 70% complete...Merging image diff: 71% complete...Merging image diff: 72% complete...Merging image diff: 73% complete...Merging image diff: 74% complete...Merging image diff: 75% complete...Merging image diff: 76% complete...Merging image diff: 77% complete...Merging image diff: 78% complete...Merging image diff: 79% complete...Merging image diff: 80% complete...Merging image diff: 81%
complete...Merging image diff: 82% complete...Merging image diff: 83% complete...Merging image diff: 84% complete...Merging image diff: 92% complete...Merging image diff: 93% complete...Merging image diff: 94% complete...Merging image diff: 95% complete...Merging image diff: 96% complete...Merging image diff: 100% complete...done.
VM 111 - Removing backup '/mnt/pve/rbd-backup//barc/111/daily/190601000503rbd.vm-111-disk-0.diff'
VM 111 - Removing backup '/mnt/pve/rbd-backup//barc/111/daily/190602000503rbd.vm-111-disk-0.diff'
VM 111 - Merge diff backup '190601000503rbd.vm-111-disk-1.diff' '190602000503rbd.vm-111-disk-1.diff'
Merging image diff: 1% complete...Merging image diff: 6% complete...Merging image diff: 7% complete...Merging image diff: 8% complete...Merging image diff: 9% complete...Merging image diff: 12% complete...Merging image diff: 13% complete...Merging image diff: 14% complete...Merging image diff: 15% complete...Merging image diff: 16% complete...Merging image diff: 47% complete...Merging image diff: 48% complete...Merging image diff: 49% complete...Merging image diff: 50% complete...Merging image diff: 51% complete...Merging image diff: 52% complete...Merging image diff: 53% complete...Merging image diff: 54% complete...Merging image diff: 55% complete...Merging image diff: 56% complete...Merging image diff: 57% complete...Merging image diff: 58% complete...Merging image diff: 59% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 62% complete...Merging image diff: 63% complete...Merging image diff: 64% comple
te...Merging image diff: 65% complete...Merging image diff: 66% complete...Merging image diff: 67% complete...Merging image diff: 68% complete...Merging image diff: 69% complete...Merging image diff: 70% complete...Merging image diff: 71% complete...Merging image diff: 72% complete...Merging image diff: 73% complete...Merging image diff: 74% complete...Merging image diff: 75% complete...Merging image diff: 76% complete...Merging image diff: 77% complete...Merging image diff: 78% complete...Merging image diff: 88% complete...Merging image diff: 89% complete...Merging image diff: 90% complete...Merging image diff: 91% complete...Merging image diff: 92% complete...Merging image diff: 93% complete...Merging image diff: 94% complete...Merging image diff: 95% complete...Merging image diff: 96% complete...Merging image diff: 97% complete...Merging image diff: 98% complete...Merging image diff: 99% complete...Merging image diff: 100% complete...done.
VM 111 - Removing backup '/mnt/pve/rbd-backup//barc/111/daily/190601000503rbd.vm-111-disk-1.diff'
VM 111 - Removing backup '/mnt/pve/rbd-backup//barc/111/daily/190602000503rbd.vm-111-disk-1.diff'
VM 111 - ======== Start backup ========
VM 111 - -------- Snapshots Disks --------
VM 111 - Creating snapshot rbd/vm-111-disk-0@barcdaily190604000503
rbd: failed to create snapshot: (17) File exists
ERROR: rbd snap create rbd/vm-111-disk-0@barcdaily190604000503
VM 111 - -------- Export images --------
VM 111 - -------- Copy config --------
VM 111 - -------- Merge diff file --------
VM 200 - ======== Start backup ========
VM 200 - -------- Snapshots Disks --------
VM 200 - Creating snapshot rbd/vm-200-disk-0@barcdaily190604000503
VM 200 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-200-disk-0@barcdaily190604000503 <<< !!!!!
VM 200 - -------- Export images --------
VM 200 - Export diff '/mnt/pve/rbd-backup//barc/200/daily/190604000503rbd.vm-200-disk-0.diff'
Exporting image: 100% complete...done.
VM 200 - Size 74
VM 200 - Remove snapshot rbd/vm-200-disk-0@barcdaily190603000503
Removing snap: 100% complete...done.
VM 200 - -------- Copy config --------
VM 200 - -------- Merge diff file --------
VM 200 - Merge diff backup '190601000503rbd.vm-200-disk-0.diff' '190602000503rbd.vm-200-disk-0.diff'
Merging image diff: 50% complete...Merging image diff: 51% complete...Merging image diff: 52% complete...Merging image diff: 53% complete...Merging image diff: 54% complete...Merging image diff: 55% complete...Merging image diff: 56% complete...Merging image diff: 57% complete...Merging image diff: 58% complete...Merging image diff: 59% complete...Merging image diff: 60% complete...Merging image diff: 61% complete...Merging image diff: 62% complete...Merging image diff: 63% complete...Merging image diff: 64% complete...Merging image diff: 65% complete...Merging image diff: 100% complete...done.
VM 200 - Removing backup '/mnt/pve/rbd-backup//barc/200/daily/190601000503rbd.vm-200-disk-0.diff'
VM 200 - Removing backup '/mnt/pve/rbd-backup//barc/200/daily/190602000503rbd.vm-200-disk-0.diff'
End backup 2019-06-04 12:50:20
`

As it can bee seen VM 104 si processed twice what generates Error (file already exists)

Som info:
`root@pve-node2:~# cat /etc/pve/qemu-server/104.conf
agent: 1
boot: c
bootdisk: scsi0
cores: 2
cpu: host,flags=+pcid;+spec-ctrl
hotplug: disk,network,usb,memory
ide2: none,media=cdrom
memory: 47104
name: OLAP
net0: virtio=E2:B3:85:5A:85:E0,bridge=vmbr0,tag=100
numa: 1
ostype: win10
scsi0: rbd:vm-104-disk-0,size=80G
scsi1: rbd:vm-104-disk-1,size=300G
scsi2: rbd:vm-104-disk-2,size=50G
scsi3: rbd:vm-104-disk-3,size=200G
scsihw: virtio-scsi-pci
smbios1: uuid=0d487f5d-9b11-4852-81d0-55935d04f303
sockets: 2

root@pve-node2:# rados lspools
rbd
ssd-cache
cephfs-backup_data
cephfs-backup_metadata
root@pve-node2:# rbd -p rbd ls
vm-100-disk-0
vm-101-disk-0
vm-101-disk-1
vm-102-disk-0
vm-103-disk-0
vm-104-disk-0
vm-104-disk-1
vm-104-disk-2
vm-104-disk-3
vm-105-disk-0
vm-105-disk-1
vm-105-disk-2
vm-105-disk-3
vm-106-disk-0
vm-107-disk-0
vm-108-disk-0
vm-108-disk-1
vm-111-disk-0
vm-111-disk-1
vm-200-disk-0
root@pve-node2:~#

root@pve-node2:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 63G 0 63G 0% /dev
tmpfs 13G 1.1G 12G 9% /run
rpool/ROOT/pve-1 25G 3.2G 22G 13% /
tmpfs 63G 54M 63G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 63G 0 63G 0% /sys/fs/cgroup
rpool/ROOT 22G 128K 22G 1% /rpool/ROOT
rpool/data 22G 128K 22G 1% /rpool/data
/dev/fuse 30M 44K 30M 1% /etc/pve
/dev/sdi1 97M 5.5M 92M 6% /var/lib/ceph/osd/ceph-22
/dev/sdg1 97M 5.5M 92M 6% /var/lib/ceph/osd/ceph-16
/dev/sdd1 94M 5.5M 89M 6% /var/lib/ceph/osd/ceph-4
/dev/sde1 94M 5.5M 89M 6% /var/lib/ceph/osd/ceph-10
/dev/sdh1 97M 5.5M 92M 6% /var/lib/ceph/osd/ceph-19
/dev/sdb1 94M 5.5M 89M 6% /var/lib/ceph/osd/ceph-25
/dev/sda1 94M 5.5M 89M 6% /var/lib/ceph/osd/ceph-7
/dev/sdf1 94M 5.5M 89M 6% /var/lib/ceph/osd/ceph-13
/dev/sdc1 94M 5.5M 89M 6% /var/lib/ceph/osd/ceph-1
10.10.10.100:/mnt/datastore/rbd-backup 22T 17T 4.5T 79% /mnt/pve/rbd-backup
10.10.10.101:6789,10.10.10.102:6789,10.10.10.103:6789:/ 7.3T 4.2T 3.1T 58% /mnt/pve/cephfs-backup
tmpfs 13G 0 13G 0% /run/user/0

`

I really like you backup solution.
It would be really nice if your program supports compression of images with lzo.

Hi, thank you for a great tool.

I have a macOS system that I want to make sure is backed up with barc.

It has a SATA disk, an IDE disk, and an EFI disk. All of them are Ceph RBD images, but barc always skips the EFI disk with no message at all.

The EFI disk is vm-102-disk-1. It will only backup disk-0 and disk-2.

Thanks for looking. I will include the barc log below, and the QEMU conf below that:

ACTION: Backup
Start backup 2020-05-06 17:55:43
VM 102 - ======== Start backup VMID 102 (macOS) ========
VM 102 - -------- Snapshots Disks VMID 102--------
VM 102 - Issuing fsfreeze-freeze to 102 on virtual37
QEMU guest agent is not running
ERROR: ssh root@virtual37 /usr/sbin/qm guest cmd 102 fsfreeze-freeze
VM 102 - No previous backup: issueing full run for cephRBD:vm-102-disk-2
VM 102 - Creating snapshot cephRBD/vm-102-disk-2@barcdaily20200506175543
VM 102 - !!!!! >>> UNPROTECTED SNAPSHOT cephRBD/vm-102-disk-2@barcdaily20200506175543 <<< !!!!!
VM 102 - No previous backup: issueing full run for cephRBD:vm-102-disk-0
VM 102 - Creating snapshot cephRBD/vm-102-disk-0@barcdaily20200506175543
VM 102 - !!!!! >>> UNPROTECTED SNAPSHOT cephRBD/vm-102-disk-0@barcdaily20200506175543 <<< !!!!!
VM 102 - Issuing fsfreeze-thaw to 102 on virtual37
QEMU guest agent is not running
ERROR: ssh root@virtual37 /usr/sbin/qm guest cmd 102 fsfreeze-thaw
VM 102 - -------- Export images --------
VM 102 - Export initial '/mnt/pve/B2.C4-J/guest_backups//barc/102/daily/20200506175543cephRBD.vm-102-disk-2.img.zz'
VM 102 - excmd: rbd export --rbd-concurrent-management-ops 10 cephRBD/vm-102-disk-2@barcdaily20200506175543 - | tee >(pigz -3 -p 2 > '/mnt/pve/B2.C4-J/guest_backups//barc/102/daily/20200506175543cephRBD.vm-102-disk-2.img.zz') | tee >({ wc -c; } > /mnt/pve/B2.C4-J/guest_backups//barc/102/daily/20200506175543cephRBD.vm-102-disk-2.img.zz.sha1.size) | sha1sum > /mnt/pve/B2.C4-J/guest_backups//barc/102/daily/20200506175543cephRBD.vm-102-disk-2.img.zz.sha1

Exporting image: 2% complete...
Exporting image: 5% complete...
Exporting image: 8% complete...
Exporting image: 10% complete...
Exporting image: 13% complete...
Exporting image: 16% complete...
Exporting image: 18% complete...
Exporting image: 21% complete...
Exporting image: 24% complete...
Exporting image: 26% complete...
Exporting image: 29% complete...
Exporting image: 32% complete...
Exporting image: 34% complete...
Exporting image: 37% complete...
Exporting image: 40% complete...
Exporting image: 42% complete...
Exporting image: 45% complete...
Exporting image: 48% complete...
Exporting image: 50% complete...
Exporting image: 53% complete...
Exporting image: 56% complete...
Exporting image: 58% complete...
Exporting image: 61% complete...
Exporting image: 64% complete...
Exporting image: 66% complete...
Exporting image: 69% complete...
Exporting image: 72% complete...
Exporting image: 74% complete...
Exporting image: 77% complete...
Exporting image: 80% complete...
Exporting image: 82% complete...
Exporting image: 85% complete...
Exporting image: 88% complete...
Exporting image: 90% complete...
Exporting image: 93% complete...
Exporting image: 96% complete...
Exporting image: 98% complete...
Exporting image: 100% complete...done.
VM 102 - Size 58.58 MiB Uncompressed: 150.00 MiB Compressionratio 1:2
VM 102 - Export initial '/mnt/pve/B2.C4-J/guest_backups//barc/102/daily/20200506175543cephRBD.vm-102-disk-0.img.zz'
VM 102 - excmd: rbd export --rbd-concurrent-management-ops 10 cephRBD/vm-102-disk-0@barcdaily20200506175543 - | tee >(pigz -3 -p 2 > '/mnt/pve/B2.C4-J/guest_backups//barc/102/daily/20200506175543cephRBD.vm-102-disk-0.img.zz') | tee >({ wc -c; } > /mnt/pve/B2.C4-J/guest_backups//barc/102/daily/20200506175543cephRBD.vm-102-disk-0.img.zz.sha1.size) | sha1sum > /mnt/pve/B2.C4-J/guest_backups//barc/102/daily/20200506175543cephRBD.vm-102-disk-0.img.zz.sha1

Exporting image: 1% complete...
Exporting image: 2% complete...
Exporting image: 3% complete...
Exporting image: 4% complete...
Exporting image: 5% complete...
Exporting image: 6% complete...
Exporting image: 7% complete...
Exporting image: 8% complete...
Exporting image: 9% complete...
Exporting image: 10% complete...
Exporting image: 11% complete...
Exporting image: 12% complete...
Exporting image: 13% complete...
Exporting image: 14% complete...
Exporting image: 15% complete...
Exporting image: 16% complete...
Exporting image: 17% complete...
Exporting image: 18% complete...
Exporting image: 19% complete...
Exporting image: 20% complete...
Exporting image: 21% complete...
Exporting image: 22% complete...
Exporting image: 23% complete...
Exporting image: 24% complete...
Exporting image: 25% complete...
Exporting image: 26% complete...
Exporting image: 27% complete...
Exporting image: 28% complete...
Exporting image: 29% complete...
Exporting image: 30% complete...
Exporting image: 31% complete...
Exporting image: 32% complete...
Exporting image: 33% complete...
Exporting image: 34% complete...
Exporting image: 35% complete...
Exporting image: 36% complete...
Exporting image: 37% complete...
Exporting image: 38% complete...
Exporting image: 39% complete...
Exporting image: 40% complete...
Exporting image: 41% complete...
Exporting image: 42% complete...
Exporting image: 43% complete...
Exporting image: 44% complete...
Exporting image: 45% complete...
Exporting image: 46% complete...
Exporting image: 47% complete...
Exporting image: 48% complete...
Exporting image: 49% complete...
Exporting image: 50% complete...
Exporting image: 51% complete...
Exporting image: 52% complete...
Exporting image: 53% complete...
Exporting image: 54% complete...
Exporting image: 55% complete...
Exporting image: 56% complete...
Exporting image: 57% complete...
Exporting image: 58% complete...
Exporting image: 59% complete...
Exporting image: 60% complete...
Exporting image: 61% complete...
Exporting image: 62% complete...
Exporting image: 63% complete...
Exporting image: 64% complete...
Exporting image: 65% complete...
Exporting image: 66% complete...
Exporting image: 67% complete...
Exporting image: 68% complete...
Exporting image: 69% complete...
Exporting image: 70% complete...
Exporting image: 71% complete...
Exporting image: 72% complete...
Exporting image: 73% complete...
Exporting image: 74% complete...
Exporting image: 75% complete...
Exporting image: 76% complete...
Exporting image: 77% complete...
Exporting image: 78% complete...
Exporting image: 79% complete...
Exporting image: 80% complete...
Exporting image: 81% complete...
Exporting image: 82% complete...
Exporting image: 83% complete...
Exporting image: 84% complete...
Exporting image: 85% complete...
Exporting image: 86% complete...
Exporting image: 87% complete...
Exporting image: 88% complete...
Exporting image: 89% complete...
Exporting image: 90% complete...
Exporting image: 91% complete...
Exporting image: 92% complete...
Exporting image: 93% complete...
Exporting image: 94% complete...
Exporting image: 95% complete...
Exporting image: 96% complete...
Exporting image: 97% complete...
Exporting image: 98% complete...
Exporting image: 99% complete...
Exporting image: 100% complete...done.
VM 102 - Size 16.77 GiB Uncompressed: 60.00 GiB Compressionratio 1:3
VM 102 - -------- Copy config --------
VM 102 - -------- Housekeeping --------
End backup 2020-05-06 18:05:01

agent: 1,fstrim_cloned_disks=1,type=virtio
args: -device isa-applesmc,osk="redacted" -smbios type=2 -cpu Penryn,kvm=on,vendor=GenuineIntel,+kvm_pv_unhalt,+kvm_pv_eoi,+invtsc,vmware-cpuid-freq=on,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check -device usb-kbd,bus=ehci.0,port=2
balloon: 0
bios: ovmf
boot: c
bootdisk: ide2
cores: 4
cpu: Penryn
efidisk0: cephRBD:vm-102-disk-1,size=1M
ide2: cephRBD:vm-102-disk-2,cache=unsafe,size=150M
machine: q35
memory: 8192
name: macOS
net0: vmxnet3=5A:E6:B3:33:62:46,bridge=vmbr0
numa: 0
ostype: other
sata0: cephRBD:vm-102-disk-0,cache=unsafe,discard=on,size=60G
scsihw: virtio-scsi-pci
smbios1: uuid=293106dc-369e-4d20-86f4-5404bab56f07
sockets: 1
vga: vmware
vmgenid: a5d1e941-bbca-4336-b2fa-3ced02e06e1c

wrong project :) pls delete

Hi,

I just tested the new version (0.2.5-renew) on my test Proxmox environment, unfortunately I found some bugs. I might try to fix them myself and make a pull request, but due to the corona situation I'm quite busy now. Just want to report those bugs, so that others might be aware of it. But really nice improvements with fsfreeze, checksum check, etc.!!

This bug is just a cosmetic issue, this does not have any functional impact! At least I didn't recognized any.

As far as I saw, the date format for the year was changed from "20" to "2020" in the new version. As you can see the year is splitted "20-20" and the seconds moved into the name of the image.

root@pve03:~# eve4pve-barc status --vmid=102 --label='daily' --path=/mnt/pve/bak2hetznerCeph
VM  TYPE SIZE  BACKUP            IMAGE
102 img    15G 20-20-04 05:11:22 37RBD01.vm-102-disk-0 
102 sha1    44 20-20-04 05:11:22 37RBD01.vm-102-disk-0.img 
102 size    12 20-20-04 05:11:22 37RBD01.vm-102-disk-0.img.sha1 

Stay healthy!
Greetings from Germany!
Andy

Hi,

I just tested the new version (0.2.5-renew) on my test Proxmox environment, unfortunately I found some bugs. I might try to fix them myself and make a pull request, but due to the corona situation I'm quite busy now. Just want to report those bugs, so that others might be aware of it. But really nice improvements with fsfreeze, checksum check, etc.!!

I'm not able to run a restore with the new version. This backup/diff was already made with the new version. A restore with the version before (0.2.1) is working without any issue. Output:

root@pve03:/mnt/pve/bak2hetznerCeph/barc# eve4pve-barc restore --vmid=104 --label='daily' --path='/mnt/pve/bak2hetznerCeph/'
ACTION: Restore image
Start restore 2020-04-06 09:12:08 20200406023001RBD01.vm-104-disk-0.diff
Remove all snapshots
rbd: error opening image vm-114-disk-0: (2) No such file or directory
ERROR: rbd snap purge RBD01/vm-114-disk-0
Backup 20200406023001RBD01.vm-104-disk-0.diff restored in RBD01/vm-114-disk-0 with success!
Consider to manually create VM/CT and change config file from backup adapting restored image.
End restore 2020-04-06 09:12:20
0 Bytes restored in 12 Seconds. [0 Bytes/s] [Compressed Size 0 Bytes]
Issuing qm-rescan
rescan volumes...

Then I tried it with a new VM, which was never backed up by this tool. Those diffs were made manually within just several minutes in a row in comparison to the output above. With the version before, again, no problem. I got this output:

root@pve02:~# eve4pve-barc restore --vmid=114 --label='daily' --path='/mnt/pve/bak2hetznerCeph/'
ACTION: Restore image
Start restore 2020-04-06 09:54:29 20200406095123RBD01.vm-114-disk-0.diff
Remove all snapshots
rbd: error opening image vm-114-disk-1: (2) No such file or directory
ERROR: rbd snap purge RBD01/vm-114-disk-1
Backup 20200406095123RBD01.vm-114-disk-0.diff restored in RBD01/vm-114-disk-1 with success!
Consider to manually create VM/CT and change config file from backup adapting restored image.
End restore 2020-04-06 09:54:29
/usr/sbin/eve4pve-barc: line 1258: local: 0/0: division by 0 (error token is "0")

Stay healthy!
Greetings from Germany,
Andy

Hey. Does not backup. Searches for Ceph settings on the wrong path. Need some kind of setup?

eve4pve-barc backup --vmid=287 --label='daily' --path=/mnt/zfs/barc --renew=7d --retain=30d
ACTION: Backup
Start backup 2020-04-21 13:18:04
VM 287 - ======== Start backup VMID 287 (sql-sharepoint) ========
VM 287 - -------- Snapshots Disks VMID 287--------
VM 287 - Issuing fsfreeze-freeze to 287 on node3
2
VM 287 - No previous backup: issueing full run for big_pool:vm-287-disk-0
VM 287 - Creating snapshot big_pool/vm-287-disk-0@barcdaily20200421131804
did not load config file, using default settings.
2020-04-21 13:18:28.417 7fd429a63f40 -1 Errors while parsing config file!
2020-04-21 13:18:28.417 7fd429a63f40 -1 parse_file: cannot open /etc/ceph/ceph.conf: (2) No such file or directory
2020-04-21 13:18:28.417 7fd429a63f40 -1 parse_file: cannot open /root/.ceph/ceph.conf: (2) No such file or directory
2020-04-21 13:18:28.417 7fd429a63f40 -1 parse_file: cannot open ceph.conf: (2) No such file or directory
2020-04-21 13:18:28.417 7fd429a63f40 -1 Errors while parsing config file!
2020-04-21 13:18:28.417 7fd429a63f40 -1 parse_file: cannot open /etc/ceph/ceph.conf: (2) No such file or directory
2020-04-21 13:18:28.417 7fd429a63f40 -1 parse_file: cannot open /root/.ceph/ceph.conf: (2) No such file or directory
2020-04-21 13:18:28.417 7fd429a63f40 -1 parse_file: cannot open ceph.conf: (2) No such file or directory
unable to get monitor info from DNS SRV with service name: ceph-mon
2020-04-21 13:18:28.429 7fd429a63f40 -1 failed for service _ceph-mon._tcp
rbd: couldn't connect to the cluster!
2020-04-21 13:18:28.429 7fd429a63f40 -1 monclient: get_monmap_and_config cannot identify monitors to contact
ERROR: rbd snap create big_pool/vm-287-disk-0@barcdaily20200421131804
VM 287 - Issuing fsfreeze-thaw to 287 on node3
2
VM 287 - Issuing fsfreeze-thaw to 287 on node3
0
VM 287 - -------- Export images --------
VM 287 - -------- Copy config --------
VM 287 - -------- Housekeeping --------
End backup 2020-04-21 13:18:37

Restored image does not work for LXC containers.

Getting error,
Mar 27 12:15:04 nod1 kernel: rbd: image vm-104-disk-1-new: image uses unsupported features: 0x38

Hi,

currently rbd commands don't specify specifc monitors or specific ceph.conf.

That mean that it's only work with the cluster defined in /etc/pve/conf

This could be great to be able to add

1. another ceph.conf (rbd -c /path/ceph.conf)
   or
2. pass monitor address to command line (rbd -m ip1;ip2;ip3)

and cephx auth + keyring if supported

proxmox code is

/usr/share/perl5/PVE/Storage/RBDPlugin.pm

if ($pveceph_managed) {
    push @$cmd, '-c', $pveceph_config;
} else {
    push @$cmd, '-m', $hostlist->($scfg->{monhost}, ',');
    push @$cmd, '--auth_supported', -e $keyring ? 'cephx' : 'none';
}

if (-e $keyring) {
    push @$cmd, '-n', "client.$username";
    push @$cmd, '--keyring', $keyring;
}

Hello,
I have this problem: if the backup medium became almost full you cannot recover the situation.
I think: I reduce the number of copies so it merges the old ones and I save space
Unfortunately if the backup space is almost full the merge operation cannot complete (because it needs some free space) and so I have no ways to recover.

It would be cool to have an ability to perform integrity check on stored images

To reduce memory usage checksum should be calculated for stream instead of file thus we have only double read: compare( read(base_image +diffs) vs read(current_rbd_snapshot))

Yes this function will be resource consuming but if it applies once per week/month I think it would be enough.

However in my understanding this feature could only be implemented after adding:
#36

Hi,to have consistency, we need to freeze/unfreeze the filesystem if qemu-agent is available.
(Proxomox implemented it with backup and snapshots)

#qm agent fsfreeze-freeze
do the snasphot of all drives
#qm agent fsfreeze-thaw

Hey there,

I decided to try using your backup program, and so far it looks pretty good. I am however having an issue after backing up a couple of containers. I can't delete them from the Proxmox GUI, I get an error that says "rbd snap purge 'vm-100-disk-1' error: Removing all snapshots: 0% complete...failed"

I've tried migrating the container to a different node in the cluster but that didn't help. At this point I'm not sure what else to try.

Hi,

I just tested the new version (0.2.5-renew) on my test Proxmox environment, unfortunately I found some bugs. I might try to fix them myself and make a pull request, but due to the corona situation I'm quite busy now. Just want to report those bugs, so that others might be aware of it. But really nice improvements with fsfreeze, checksum check, etc.!!

I tried to switch from the "keep" behavior to "renew" in combination with "retain". Unfortunately, whatever I've set "renew" or "retain" to, I never got a new initial backup. I manually made multiple backups in a row to test this, unfortunately just normal diffs were made without any limits. No new initial backup, even that the numbers defined in "renew" and "retain" were clearly exceeded. Not sure if I've done something wrong or maybe I've misunderstood something, does someone else have the same behavior?

Stay healthy!
Greetings from Germany!
Andy

eve4pve backups are great but are of limited use for restoration if the vm config is not present; can you add an option to backup/restore the configuration along with the disk image?

Add possibility to assemble image with diff files

Since there are currently two approaches in the Codebase, I want to discuss if you want to keep the ability to merge snapshots into each other. I personally wouldn't dare to go on and take one Base Snapshot in a year, and then only merge onto it. And: There's no way to merge into the Basecopy, so the volatile data would accumulate in the first Differential file after the Basecopy.

eve4barc is running since 2 years without any problem. Out of the sudden, I can not recover a backup (keep-mode) at all. None of the diff parts can be used.

I get a flood of "unrecognized tag byte 0 in stream; skipping" messages

and at the end
7fcae69b4f40 -1 librbd::Operations: New size not compatible with object map

After some further operations (successfull) I get the message "Successfully recovered".

But the image is unusable.
Any ideas whats going wrong?

How to reproduce:

forward all messages to remote syslog
. @syslog.srv:514

All messages from barc4pve are marked as ERROR;
2019-04-12 01:35:52 Daemon Error 10.71.200.103 Apr 12 01:35:52 pve-node3 eve4pve-barc: rbd snap create rbd/vm-111-disk-0@barcdaily190412000002
2019-04-12 01:28:59 Daemon Error 10.71.200.103 Apr 12 01:28:59 pve-node3 eve4pve-barc: rbd snap create rbd/vm-105-disk-0@barcdaily190412000002
2019-04-12 01:28:58 Daemon Error 10.71.200.103 Apr 12 01:28:58 pve-node3 eve4pve-barc: rbd snap create rbd/vm-105-disk-0@barcdaily190412000002
2019-04-12 01:28:57 Daemon Error 10.71.200.103 Apr 12 01:28:57 pve-node3 eve4pve-barc: rbd snap create rbd/vm-105-disk-0@barcdaily190412000002

#cat /var/log/syslog.1
Apr 12 01:28:40 pve-node3 eve4pve-barc: VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190409000003rbd.vm-105-disk-1.diff'
Apr 12 01:28:47 pve-node3 eve4pve-barc: VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190410000002rbd.vm-105-disk-1.diff'
Apr 12 01:28:47 pve-node3 eve4pve-barc: VM 105 - Merge diff backup '190409000003rbd.vm-105-disk-2.diff' '190410000002rbd.vm-105-disk-2.diff'
Apr 12 01:28:51 pve-node3 eve4pve-barc: VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190409000003rbd.vm-105-disk-2.diff'
Apr 12 01:28:51 pve-node3 eve4pve-barc: VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190410000002rbd.vm-105-disk-2.diff'
Apr 12 01:28:51 pve-node3 eve4pve-barc: VM 105 - Merge diff backup '190409000003rbd.vm-105-disk-3.diff' '190410000002rbd.vm-105-disk-3.diff'
Apr 12 01:28:55 pve-node3 eve4pve-barc: VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190409000003rbd.vm-105-disk-3.diff'
Apr 12 01:28:56 pve-node3 eve4pve-barc: VM 105 - Removing backup '/mnt/pve/rbd-backup//barc/105/daily/190410000002rbd.vm-105-disk-3.diff'
Apr 12 01:28:56 pve-node3 eve4pve-barc: VM 105 - ======== Start backup ========
Apr 12 01:28:56 pve-node3 eve4pve-barc: VM 105 - -------- Snapshots Disks --------
Apr 12 01:28:57 pve-node3 eve4pve-barc: VM 105 - Creating snapshot rbd/vm-105-disk-0@barcdaily190412000002
Apr 12 01:28:57 pve-node3 eve4pve-barc: rbd snap create rbd/vm-105-disk-0@barcdaily190412000002
Apr 12 01:28:57 pve-node3 eve4pve-barc: VM 105 - -------- Export images --------
Apr 12 01:28:57 pve-node3 eve4pve-barc: VM 105 - -------- Copy config --------
Apr 12 01:28:57 pve-node3 eve4pve-barc: VM 105 - -------- Merge diff file --------
Apr 12 01:28:57 pve-node3 eve4pve-barc: VM 105 - ======== Start backup ========
Apr 12 01:28:57 pve-node3 eve4pve-barc: VM 105 - -------- Snapshots Disks --------
Apr 12 01:28:58 pve-node3 eve4pve-barc: VM 105 - Creating snapshot rbd/vm-105-disk-0@barcdaily190412000002
Apr 12 01:28:58 pve-node3 eve4pve-barc: rbd snap create rbd/vm-105-disk-0@barcdaily190412000002
Apr 12 01:28:58 pve-node3 eve4pve-barc: VM 105 - -------- Export images --------
Apr 12 01:28:58 pve-node3 eve4pve-barc: VM 105 - -------- Copy config --------
Apr 12 01:28:58 pve-node3 eve4pve-barc: VM 105 - -------- Merge diff file --------
Apr 12 01:28:58 pve-node3 eve4pve-barc: VM 105 - ======== Start backup ========
Apr 12 01:28:58 pve-node3 eve4pve-barc: VM 105 - -------- Snapshots Disks --------
Apr 12 01:28:59 pve-node3 eve4pve-barc: VM 105 - Creating snapshot rbd/vm-105-disk-0@barcdaily190412000002
Apr 12 01:28:59 pve-node3 eve4pve-barc: rbd snap create rbd/vm-105-disk-0@barcdaily190412000002
Apr 12 01:28:59 pve-node3 eve4pve-barc: VM 105 - -------- Export images --------
Apr 12 01:28:59 pve-node3 eve4pve-barc: VM 105 - -------- Copy config --------
Apr 12 01:28:59 pve-node3 eve4pve-barc: VM 105 - -------- Merge diff file --------
Apr 12 01:28:59 pve-node3 eve4pve-barc: VM 106 - ======== Start backup ========
Apr 12 01:28:59 pve-node3 eve4pve-barc: VM 106 - -------- Snapshots Disks --------
Apr 12 01:29:00 pve-node3 eve4pve-barc: VM 106 - Creating snapshot rbd/vm-106-disk-0@barcdaily190412000002
Apr 12 01:29:01 pve-node3 eve4pve-barc: VM 106 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-106-disk-0@barcdaily190412000002 <<< !!!!!
Apr 12 01:29:01 pve-node3 eve4pve-barc: VM 106 - -------- Export images --------
Apr 12 01:29:01 pve-node3 eve4pve-barc: VM 106 - Export diff '/mnt/pve/rbd-backup//barc/106/daily/190412000002rbd.vm-106-disk-0.diff'
Apr 12 01:29:55 pve-node3 eve4pve-barc: VM 106 - Size 1.1G
Apr 12 01:29:55 pve-node3 eve4pve-barc: VM 106 - Remove snapshot rbd/vm-106-disk-0@barcdaily190411000003
Apr 12 01:29:57 pve-node3 eve4pve-barc: VM 106 - -------- Copy config --------
Apr 12 01:29:57 pve-node3 eve4pve-barc: VM 106 - -------- Merge diff file --------
Apr 12 01:29:57 pve-node3 eve4pve-barc: VM 106 - Merge diff backup '190409000003rbd.vm-106-disk-0.diff' '190410000002rbd.vm-106-disk-0.diff'
Apr 12 01:30:31 pve-node3 eve4pve-barc: VM 106 - Removing backup '/mnt/pve/rbd-backup//barc/106/daily/190409000003rbd.vm-106-disk-0.diff'
Apr 12 01:30:31 pve-node3 eve4pve-barc: VM 106 - Removing backup '/mnt/pve/rbd-backup//barc/106/daily/190410000002rbd.vm-106-disk-0.diff'
Apr 12 01:30:32 pve-node3 eve4pve-barc: VM 111 - ======== Start backup ========
Apr 12 01:30:32 pve-node3 eve4pve-barc: VM 111 - -------- Snapshots Disks --------
Apr 12 01:30:33 pve-node3 eve4pve-barc: VM 111 - Creating snapshot rbd/vm-111-disk-0@barcdaily190412000002
Apr 12 01:30:35 pve-node3 eve4pve-barc: VM 111 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-111-disk-0@barcdaily190412000002 <<< !!!!!
Apr 12 01:30:36 pve-node3 eve4pve-barc: VM 111 - Creating snapshot rbd/vm-111-disk-1@barcdaily190412000002
Apr 12 01:30:38 pve-node3 eve4pve-barc: VM 111 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-111-disk-1@barcdaily190412000002 <<< !!!!!
Apr 12 01:30:38 pve-node3 eve4pve-barc: VM 111 - -------- Export images --------
Apr 12 01:30:38 pve-node3 eve4pve-barc: VM 111 - Export diff '/mnt/pve/rbd-backup//barc/111/daily/190412000002rbd.vm-111-disk-0.diff'
Apr 12 01:31:49 pve-node3 eve4pve-barc: VM 111 - Size 1.2G
Apr 12 01:31:49 pve-node3 eve4pve-barc: VM 111 - Remove snapshot rbd/vm-111-disk-0@barcdaily190411000003
Apr 12 01:31:51 pve-node3 eve4pve-barc: VM 111 - Export diff '/mnt/pve/rbd-backup//barc/111/daily/190412000002rbd.vm-111-disk-1.diff'
Apr 12 01:34:56 pve-node3 eve4pve-barc: VM 111 - Size 962M
Apr 12 01:34:56 pve-node3 eve4pve-barc: VM 111 - Remove snapshot rbd/vm-111-disk-1@barcdaily190411000003
Apr 12 01:34:58 pve-node3 eve4pve-barc: VM 111 - -------- Copy config --------
Apr 12 01:34:58 pve-node3 eve4pve-barc: VM 111 - -------- Merge diff file --------
Apr 12 01:34:58 pve-node3 eve4pve-barc: VM 111 - Merge diff backup '190409000003rbd.vm-111-disk-0.diff' '190410000002rbd.vm-111-disk-0.diff'
Apr 12 01:35:32 pve-node3 eve4pve-barc: VM 111 - Removing backup '/mnt/pve/rbd-backup//barc/111/daily/190409000003rbd.vm-111-disk-0.diff'
Apr 12 01:35:33 pve-node3 eve4pve-barc: VM 111 - Removing backup '/mnt/pve/rbd-backup//barc/111/daily/190410000002rbd.vm-111-disk-0.diff'
Apr 12 01:35:33 pve-node3 eve4pve-barc: VM 111 - Merge diff backup '190409000003rbd.vm-111-disk-1.diff' '190410000002rbd.vm-111-disk-1.diff'
Apr 12 01:35:50 pve-node3 eve4pve-barc: VM 111 - Removing backup '/mnt/pve/rbd-backup//barc/111/daily/190409000003rbd.vm-111-disk-1.diff'
Apr 12 01:35:51 pve-node3 eve4pve-barc: VM 111 - Removing backup '/mnt/pve/rbd-backup//barc/111/daily/190410000002rbd.vm-111-disk-1.diff'
Apr 12 01:35:51 pve-node3 eve4pve-barc: VM 111 - ======== Start backup ========
Apr 12 01:35:51 pve-node3 eve4pve-barc: VM 111 - -------- Snapshots Disks --------
Apr 12 01:35:52 pve-node3 eve4pve-barc: VM 111 - Creating snapshot rbd/vm-111-disk-0@barcdaily190412000002
Apr 12 01:35:52 pve-node3 eve4pve-barc: rbd snap create rbd/vm-111-disk-0@barcdaily190412000002
Apr 12 01:35:52 pve-node3 eve4pve-barc: VM 111 - -------- Export images --------
Apr 12 01:35:52 pve-node3 eve4pve-barc: VM 111 - -------- Copy config --------
Apr 12 01:35:52 pve-node3 eve4pve-barc: VM 111 - -------- Merge diff file --------
Apr 12 01:35:52 pve-node3 eve4pve-barc: VM 200 - ======== Start backup ========
Apr 12 01:35:52 pve-node3 eve4pve-barc: VM 200 - -------- Snapshots Disks --------
Apr 12 01:35:53 pve-node3 eve4pve-barc: VM 200 - Creating snapshot rbd/vm-200-disk-0@barcdaily190412000002
Apr 12 01:35:54 pve-node3 eve4pve-barc: VM 200 - !!!!! >>> UNPROTECTED SNAPSHOT rbd/vm-200-disk-0@barcdaily190412000002 <<< !!!!!
Apr 12 01:35:54 pve-node3 eve4pve-barc: VM 200 - -------- Export images --------
Apr 12 01:35:54 pve-node3 eve4pve-barc: VM 200 - Export diff '/mnt/pve/rbd-backup//barc/200/daily/190412000002rbd.vm-200-disk-0.diff'
Apr 12 01:36:47 pve-node3 eve4pve-barc: VM 200 - Size 1.5G
Apr 12 01:36:47 pve-node3 eve4pve-barc: VM 200 - Remove snapshot rbd/vm-200-disk-0@barcdaily190411000003
Apr 12 01:36:47 pve-node3 eve4pve-barc: VM 200 - -------- Copy config --------
Apr 12 01:36:47 pve-node3 eve4pve-barc: VM 200 - -------- Merge diff file --------
Apr 12 01:36:47 pve-node3 eve4pve-barc: VM 200 - Merge diff backup '190409000003rbd.vm-200-disk-0.diff' '190410000002rbd.vm-200-disk-0.diff'
Apr 12 01:37:03 pve-node3 eve4pve-barc: VM 200 - Removing backup '/mnt/pve/rbd-backup//barc/200/daily/190409000003rbd.vm-200-disk-0.diff'
Apr 12 01:37:04 pve-node3 eve4pve-barc: VM 200 - Removing backup '/mnt/pve/rbd-backup//barc/200/daily/190410000002rbd.vm-200-disk-0.diff'
Apr 12 01:37:04 pve-node3 eve4pve-barc: End backup 2019-04-12 01:37:04

It would be nice if there is the possibility to define that your program only backup the VMs/CTs that are located on that node the program is started.

This is usefull for a clustered solution, imagine that you have a cluster of 5 nodes. So i want to run the backup job on every node to make sure we have no single point of failure. But, we need a logic to not concurrently backup the images. So the easiest would be that each backup job would only backup the VMs/CTs that are located on the node where the backup job is running.

If we run clustered and a node fails, the VMs/CTs get moved to another node by HA. Therefore we always backup all VMs/CTS.

I think this would be a great addition to your product.

Best regards from Austria ;)
Andreas

Hi, really great work, thanks for it.
Please consider deleting old snapshots after successful export of all images, not between exporting.
And there is a bug: second disk isn't unprotected before deleting.
another a little bug with: "rbd: failed to create snapshot: (17) File exists" (please see log))
another bug: loop will destroy barc snapshot of first disk, cause IMHO keep=2 and VM has 2 disks/snapshots or etc.

because i ran barc at screen, i lost output from it.
i've deleted all snapshots and re-ran it again, all errors have appeared again, please see log below

#i tried it even without symbol '=' .. same results

eve4pve-barc backup --vmid=135 --label='daily' --path=/snapshoty-sshfs --keep=2

Start backup 2017-09-25 17:11:13
VM 135 - ======== Ceph Image 'rbd/vm-135-disk-1' ========
VM 135 - Creating snapshot rbd/vm-135-disk-1@barcdaily170925171113
VM 135 - ======== Ceph Image 'rbd/vm-135-disk-2' ========
VM 135 - Creating snapshot rbd/vm-135-disk-2@barcdaily170925171113
VM 135 - ======== Ceph Image 'rbd/vm-135-disk-1' ========
VM 135 - Creating snapshot rbd/vm-135-disk-1@barcdaily170925171113
rbd: failed to create snapshot: (17) File exists
ERROR: rbd snap create rbd/vm-135-disk-1@barcdaily170925171113
VM 135 - Export initial '/snapshoty-sshfs/barc/135/daily/170925171113rbd.vm-135-disk-1.img'
Exporting image: 100% complete...done.
VM 135 - Size 1.0G
VM 135 - Export initial '/snapshoty-sshfs/barc/135/daily/170925171113rbd.vm-135-disk-2.img'
Exporting image: 100% complete...done.
VM 135 - Size 50G
/usr/sbin/eve4pve-barc: line 574: [: -eq: unary operator expected
VM 135 - Export diff ''
rbd: image name was not specified
ERROR: rbd export-diff --from-snap rbd/vm-135-disk-1@barcdaily170925171113 ''
VM 135 - Remove snapshot rbd/vm-135-disk-1@barcdaily170925171113
VM 135 - Remove file ''
VM 135 - Copy config
End backup 2017-09-25 17:12:45

echo $?

0

SNAPID NAME SIZE
159 testcopy 1024 MB
SNAPID NAME SIZE
160 testcopy 51200 MB
188 barcdaily170925172359 51200 MB

thanks for your time and work
appreciate it
best regards
Tomas

Hello, please make snapshot of all disk for specific VM at same time.

I have sshfs mouted storage, with 72 MBps upload.
VM has 4x 200 GB disk, who is PV for one VG in LVM
There is a lot of minutes gap between disk snapshosts of this VM, so data will be inconsistent
Thanks for your work.