Making backups have several options so its setup is manual as of now. But as this toolset is intended to make these kind of operations as simple as possible, we should provide a simple make backup-sync target that will run ./dcs-tools/make-backup script with predefined parameters, where the appropriate setup would be previously performed by ./dcs-tools/setup script.

Hi,
I couldn't stop myself from going ahead and trying to get it working.

I set up my project as proposed in the advanced setup:

home/
remote-machines/
raspberrypiB/
dcs-tools/

But when I try to run the command
./setup

I get this error
$ ./setup
ln: failed to create symbolic link '/c/Users/The Producer/remote-machines/raspberrypiB/dcs-tools/../Makefile': No such file or directory
There is a configuration file already

I did find a file called config.sh, which looked as though it might be the intended result of the setup, so I added the IP address of my 'target', the RaspberryPi

However.. when I go on to run
$ ./dcs-tools/make-target-settings

I get this error:
bash: ./dcs-tools/make-target-settings: No such file or directory

In researching the ./setup error I find that symbolic links may be the root of the problem:

microsoft/WSL#353

Does this seem likely to you?
Is there a workaround that you can suggest?
Other than me buying a linux workstation!

Thanks

Recent changes introduce fuse: unknown option 'HashKnownHosts=no' error while using make mount-root action.

I want to backup a Raspberry Pi running Jessie from the RPI to a NAS, mounted as an sshfs mounted volume.

I used the following command to mount the volume:

...
sudo sshfs admin@diskstation:/RPI/rpi-full-backups /mnt/nas -o IdentityFile=~/.ssh/id_rsa,idmap=user,reconnect,uid=1000,gid=1000,allow_other
...

This works fine; I can access the volume without providing a password. I then installed the dcs-tools - following the provided instructions - on the NAS from the RPI. When I run the command I get the following output:

pi@araminta:/mnt/nas $ make set-local-session
make: Makefile: No such file or directory
make: *** No rule to make target 'Makefile'.  Stop.

It looks like the symbolic link to the sshfs mounted volume does not work and therefore the Makefile is not found.

Any suggestions?

use --info=progress2 instead of --progress option.

When using connections over the link up server (with conn-proxy option), large files can not be transferred due to breaking connections due to double encryption overhead. (see https://serverfault.com/q/26358/261445)

Hi Cerem,

When I run the backup command as follows I get an error

Enter Comment : test
Creating backup with the following name: 20171126T1509-test
Using hardlinks method
ERROR:
ERROR:
ERROR: FIXME: invalid cross-device link error on btrfs filesystem. Below code is still tested well.
ERROR:
ERROR:```

I looked inside the make-backup script and can see that it's an error message from your system, not a system error, so I hope you will know what it means.

When I try to Google it, this error seems to be related to separate hard drives 

https://unix.stackexchange.com/questions/79132/invalid-cross-device-link-while-hardlinking-in-the-same-file-system

My understanding of 'hard linking' is that it's an ACTUAL copy of data, not a pointer to a single data set, am I wrong?

N

Hi Cerem,
I've succeded with both an initial and incremental backup of my RaspberryPiZeroW and now I'm looking to make that backup into a bootable SD card image.

Here's what I get when I try to make the backup

root@raspberrypi:/media/pi/PiStore/remote-machines/zero1# ./dcs-tools/make-backu p --source sync-root/ --backups bootable_backups/ --method hardlinks
Enter Comment : First_attempt_Zero_01
Creating backup with the following name: 20171119T0844-First_attempt_Zero_01
Using hardlinks method
ERROR:
ERROR:
ERROR: FIXME: invalid cross-device link error on btrfs filesystem. Below code is still tested well.
ERROR:
ERROR:

This is on a Raspberry Pi B+ running the latest version of Raspbian
PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"
NAME="Raspbian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"

Thanks in advance for your consideration
Nigel

Using BTRFS snapshots in method-btrfs mode creates btrfs snapshots in the backup directory. This prevents whole operating system backups containing those btrfs snapshots, which would cause requiring additional attention to those who wants to use dcs-tools to keep multiple backups of target systems and use btrfs snapshot to keep whole operating system backups.

Proposal

Use cp --reflink=always in place of btrfs sub snap. This will be still

1. Fast enough
2. Space efficient

default should be at least 50, instead of 5.

Whole system slows down while performing backup. Videos (played by browser) start glitching, browsing performance dramatically reduces, thus system becomes less usable. This makes user stay back from performing backups in daily usage.

convert-to-fresh-install should reset the sshd keys in order to create the distinction between controllers.

If there are many controllers/machines in the field, they share the same username (NODE_USER) and they either have the same password or are configured to let the same RSA key for passwordless login (where ./make-target-settings.sh performs); then the following scenario may pose from trivial to catastrophic consequences:

If there are more than one controller in the field and their IP address is assigned by the DHCP server, when DHCP server accidentally switches their IP addresses, developer may perfectly connect to the wrong controller (Linux machine) with the right IP address.

Proposal

A NODE_HOSTNAME parameter should be required so that this name can be used to check if the developer is connected the right target or not.

See: https://blog.no-panic.at/2016/08/11/a-clean-start-how-to-prepare-a-minimal-debian-template-for-lxc-containers/

1. make a dry run
2. check for deleted/new files
3. test if deleted and new files are matched

make backup-root exits with error due to LXCFS permission error. That might be related with: https://bugs.launchpad.net/ubuntu/+source/lxcfs/+bug/1656309

Commands should not vary that much. There are connection strategies, such as:

• direct
• over proxy
• local

There are commands:

• make init
• make ssh
• make backup-root
• make mount-root

So, there might be a flag file which will indicate connection strategy, and above commands will work according to that strategy automatically.

Session changes might be done with these commands:

• make session-[direct, proxy, local]

Hi!

I see this on my Raspbian system:

make backup-root
/no/ssh/id/file/specified: No such file or directory
/no/ssh/id/file/specified: No such file or directory
/no/ssh/id/file/specified: No such file or directory
project root is correct
/no/ssh/id/file/specified: No such file or directory
/no/ssh/id/file/specified: No such file or directory
/no/ssh/id/file/specified: No such file or directory
tor  9 mar 2017 18:38:24 CET

But the script seems to work. How do I get rid of that annoying message?

In order to fix #24 , multiplexed connections are disabled for sshfs commands. Re-enable multiplexed connection file usages for sshfs commands.

Hi again,
I feel I've made some good progress but here's where I'm stuck now. I've put a freshly formatted MicroSD card in a USB adaptor, into my RaspberryPi and done the following

pi@raspberrypi:/media/pi/PiStore/remote-machines/zero2 $ lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    1 28.9G  0 disk
└─sda1        8:1    1 28.9G  0 part /media/pi/PiStore
sdc           8:32   1 14.9G  0 disk
└─sdc1        8:33   1 14.9G  0 part
mmcblk0     179:0    0   29G  0 disk
├─mmcblk0p1 179:1    0  1.6G  0 part
├─mmcblk0p2 179:2    0    1K  0 part
├─mmcblk0p5 179:5    0   32M  0 part /media/pi/SETTINGS
├─mmcblk0p6 179:6    0   69M  0 part /boot
└─mmcblk0p7 179:7    0 13.4G  0 part /
pi@raspberrypi:/media/pi/PiStore/remote-machines/zero2 $ dcs-tools/produce-bootable-disk  --backup backups --device /dev/sdc
Using backup directory: /media/pi/PiStore/remote-machines/zero2/backups
Using device: /dev/sdc
Using default ext3 type.
----------------------  YES / NO  ----------------------
Should we really continue? (yes/no) yes
Bootable device /dev/sdc will be built by using /media/pi/PiStore/remote-machines/zero2/backups
ERROR:
ERROR:
ERROR: No such partition/device can be found: /dev/sdcp1
ERROR:
ERROR:

It seems to be looking for sdcp1 but the SD card is unformatted, so, of course, there are no partitions.

Hi,

Another thing I think I may be doing wrong is executing commands from the wrong directory.
If it's important that I'm in a certain place, please add that information to the guide

Some more simple examples would be really helpful
IE.
cd to your backup destination folder then run the command make sync-root

Please excuse such simple questions but when I run the command:
./make-target-settings what does the 'target' refer to.. the destination machine where the backup is to be copied to, or the raspberry pi which I'm trying to back up?

If you could add step by step instructions, with details of which machine the command is being performed on, that would help. I have managed to set up a Putty ssh connection from my workstation to the Raspberry Pi but I have been getting confused whether it makes any difference when I'm configuring the target and setup files.
thanks

btrfs-utils should be optional, so following error shouldn't be thrown if it's missing:

./dcs-tools/setup: line 31: btrfs: command not found

#Direct connection settings:

NODE_IP Is the IP address of the Raspberry Pi or the machine connecting to it?
MOUNT_DIR=$(mktemp -d) what is this?