welaika / webackup

A simple script that simplify use of rdiff-backup to perform backup operations on multiple hosts

weBackup

Wraps rdiff-backup and manage multiple host backup to add and remove them easily.

HOW TO - INSTALL

1. Untar package or

    git clone git://github.com/welaika/weBackup.git
   

2. Install rdiff-backup on local machine

3. Install an heirloom-mailx (debian family: apt-get install heirloom-mailx). Then go to the "configure" file in the script's root folder and set up your external SMTP.

4. Install rdiff-backup on every remote machine you need to backup or... (ehi wait! Are you using a MediaTemple Grid-Service? Take a look here)

5. If you cannot install rdiff-backup on your remote machines, we can work it out using sshfs. Please install it on your server system (this one!) with "apt-get install sshfs" for debian-based OS or equivalent command.

6. Have a "configure" file

    cp configure.tpl configure
   

read and set it up

HOW TO - HOW IT WORKS

Script structure dissection and explanation

├── addhost.sh

Invoke this to add a host to backups. More follows in this readme

├── backup.sh

This is the main mackup script. Nothing to be configured here. Read just if your interest. Usage follows in this readme

├── CHANGELOG.md

A changelog bad emulation

├── conf
│   ├── example.com
│   │   ├── globbing.conf
│   │   ├── host.conf
│   └── template.tpl
│       ├── globbing.conf
│       └── host.conf

The addhost.sh produces host configuration in this directory. Each subdir is a host. You can rename a dir hiding it (e.g.: .example.com) to disable backups for that host. For each subdir we have two configuration files. We'll tell more about them in a while.

├── configure.tpl

Main configuration file. SO MUCH IMPORTANT! We mentioned it before in HOW TO INSTALL

├── lib
│   ├── function.backup.lib
│   ├── function.log.lib
│   ├── settings.sh
│   ├── sysconfig (deprecated)
│   └── transitionals.sh

These are libraries of bash functions.

├── log
│   └── weBackup.log

Logs. The script will try to automatically configure logrotate to handle log rotation inside this folder

└── README.md

Just what are you reading ;)

HOW TO - CONFIGURE A HOST

addhost.sh

Go in weBackup's base directory and type

./addhost.sh

An interactive script will guide you through the creation of the host configuration. At the end of this process it will remind you to set up pubkey ssh autentication (not yet implemented an automatic process) and to set up the globbing-file-list in globbing.conf. Take a look also at conf/hostname/host.conf to see configured host and comments about options.

globbing.conf

This is an rdiff internal, so you should have to study a bit of rdiff-backup doc. See below for references. Any way a couple of tips follows.

Overall if you wont specify nothing here, the whole filesystem starting from the specified basedir (rpath in host.conf) will be backed up. If you like to backup just some directories, assuming my basedir is /home/pioneerskies, write something like this:

/home/pioneerskies/dev
/home/pioneerskies/web
/home/pioneerskies/git
- **

this will backup dev, web and git into your $BACKUP_DIR, ecluding all other path.

REMEBER that if you want to exclude a specific dir inside a dir you want to backup you have, e.g.:

/home/pioneerskies/dev
/home/pioneerskies/web
- /home/pioneerskies/git/weBackup
/home/pioneerskies/git
- **

in the right order, with the exclusion of a deepest path before the inclusione of a higher one. That's ok?

ssh-copy-id

To let rdiff-backup perform backups of remote host, a ssh access to the remote host must be provided. rdiff-backup and SSHFS use ssh to connect to remote host, so is requested enabling pubkey auth for your user accont on server, disabling access with password. The command is:

ssh-copy-id [email protected]

then follow on screen istructions.

For further references:

• http://bit.ly/qdECva
• http://bit.ly/qMbbVG

HOW TO - DO A BACKUP

From the scripts basedir run script

./backup.sh [OPTIONS]

See ./backup.sh -h for list of options. Common options are ./backup.sh -v -b

You can now setup a single cronjob to manage all your backups:

# m h  dom mon dow   command
0 21 * * * bash -l -c "cd /root/weBackup && ./backup.sh -v -b"

bash -l -c is not mandatory: I use it usually, so is in the example ;)

REFERENCES

http://www.nongnu.org/rdiff-backup/

http://wiki.rdiff-backup.org/wiki/index.php/BackupSomeDirectoriesOnly

Author

made with ❤️ and ☕️ by weLaika

License

(The MIT License)

Copyright © 2017-2019 weLaika

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ‘Software’), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED ‘AS IS’, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

webackup's Issues

logdate() not printing

The function remains as "logdate" in the resulting log; investigate in function.log.lib

Wrong submission of backup's log via mail

Design FAIL! The entire logfile is submitted via mail at the end of each backup!
Wake up the brain and kill this issue, guy!

ssh-copy-id

The whole script is made to run without user interactivity, so we have to guide the user to automate ssh remote login against the added hosts using pubkeys. Don't know if in addhost.sh automagically or just with an advise or just in the docs.

Let's think about it!

BACKUP_DIR can't be blank

Implement a conf parser for

BACKUP_DIR=""

in configure, so that it cannot be left blank

TAGlines in logs

Seems that they aren't where thay have to. Bad one guys...

Doubt in create_log()

Don't know why in the create_log function in lib/function.log.lib line 12 there is a

echo $LOG

instead of a simple return.

Ooold doc and readme

No one implementation done by me is documented! Holy shit!

Issue in unmounting sshfs dir

Fuck! After all those attentions the backup ends but the remote dir mounted with fuse+sshfs remains mounted.
Improve logging statements in mount_sshfs() and find out the fucking problem!

Check if sshfs is installed

Check deactivated at the moment in function.backup.lib:196 cuz doesn't work.
Fix it and reactivate and handle eventual errors

Activate test_logrotate()

Function created but not implemented in the code. Find a good place in the flow and let it rock'n'roll!

bug in addhost interactive print

We will connect with: example.com@y, all right? :) after specifying the SSH user

@y sounds like a bug...

Retention per host

Would be nice to have a custom retention per host.

Debug of local backup not handled!

In function.backup.lib +73 the debug of "not remote" starts. But it handle only the test for "locally mounted" filesystems (sshfs) and not the "local direcotires" one:
This is ok...

if ${servconf[3]}; then #if a remotely mounted sshfs filesystem
    mount_sshfs ${user}${host}::/ ${servconf[2]}
fi

but if it is not a remotely mounted sshfs filesystem we check anyway the mountpoint...

log "TESTING mountpoint /mnt/$host" "" 1
        mountpoint ${servconf[2]} > /dev/null
        if [[ $? != 0 ]]; then
          log "ERROR: mount of the remote filesystem went bad; problem probably genereted remotly. Aborting" "" 1
fi

it should be smt like

if ${servconf[3]}; then #if a remotely mounted sshfs filesystem
    mount_sshfs ${user}${host}::/ ${servconf[2]}

    log "TESTING mountpoint /mnt/$host" "" 1
    mountpoint ${servconf[2]} > /dev/null
    if [[ $? != 0 ]]; then
        log "ERROR: mount of the remote filesystem went bad; problem probably genereted remotly. Aborting" "" 1
    else #this is a local dir
        log "TESTING directory ${servconf[2]}" "" 1
        log "rdiff-backup --print-statistics --verbosity $RDIFF_VERBOSITY --include-globbing-filelist ${CONF_DIR}/${host}/globbing.conf --exclude / ${servconf[2]} ${BACKUP_DIR}/${host}" 1 1
fi

Let's try and test

Log fancyness

Work on log readibility

logrotate test fails

I don't know exaxctly why, but the test

if [[ ! `which logrotate` ]];

fails. I've reproduced the bug doind the test as unprivileged user in a debian 5 box (in ubuntu works even if executable takes place in /usr/sbin). But I assume that the script is invoked by the root's crontab so executed by root. Is this incorrect? Have I to load some environment related stuff?

split backup function from the debug function

The code of the main function is unreadable! Split theme is the first step to do

Make interaction of addhost.sh polite

Text output isn't clear for the user. Let's play a bit with \n and \r ;)