vanzhiganov / backup Goto Github PK

View Code? Open in Web Editor NEW

1.0 1.0 2.0 42 KB

Yet another backup tool

Home Page: https://github.com/vanzhiganov/backup

License: The Unlicense

Python 99.49% Dockerfile 0.51%

backup python gpg-key files mysql ftp webdav

backup's Introduction

backup

Current version: v1.1.6

collection backup and recovery scripts

Features

backup files
backup mysql databases
backup pgsql databases
compress archives
upload archives to webdav storage
upload archives to ftp server

Tested

CentOS 6.6 Server
Fedora Linux 21 Desktop
Ubuntu 14.04 LTS Server

Install

sudo apt-get install gnupg python-gnupg python-configparser python-easywebdav rng-tools

sudo pip install ftptool

For Postgresql

createuser <backup_username>

alter user <backup_username> password '<password>';

GRANT SELECT ON ALL TABLES IN SCHEMA public TO <backup_usermame>;

GRANT CONNECT ON DATABASE <database_name> to <backup_usermame>;

GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO <backup_usermame>;

for pg_basebackup (EXPERIMENTAL)

pg_hba.conf local replication postgres md5

postgresql.conf max_wal_senders = 1 wal_level = hot_standby

Using

Examples

Show help

python backup.py --help

Specify config file

python backup.py -c config.ini

Specify log file

python backup.py -l backup.log

Configuring

config file example

'default' section example

[DEFAULT]
InstanceName = WorkPCUbuntuLinux
StorageLocal = /tmp/storage
Compression = yes
CompressionLevel = 9

example files backup

[File:test]
Enabled = yes
Directory = /etc
Compression = yes
CompressionLevel = 9
SavedDaily = 7
SavedWeekly = 4
SavedMonthly = 12
Remote = webdav_yandex
SaveLocal = yes

example mysql database backup

[Database:mysql_testdatabase]
Enabled = yes
Engine = mysql
Host = localhost
User = root
Password = rootpassword
Database = test_database
Compression = yes
CompressionLevel = 9
SavedDaily = 7
SavedWeekly = 4
SavedMonthly = 12
SaveLocal = no

example postgresql database backup

[Database:test_pgsql]
Enabled = yes
Engine = pgsql
Host = localhost
User = backup_user
Password = BacK8OLpU@r
Database = myappdatabase
Remote = webdav_yandex

example webdav config

[REMOTE:webdav_yandex]
Type = webdav
Host = webdav.yandex.ru
Port = 443
Protocol = https
Login = user
Password = password

GPG keys and crypt

Prepare GPG keys

gpg --gen-key

Make few steps and gpg-key will be generated

In the next step we need to change trust level for user,

gpg --sign-key [email protected]

After that you fully trust the key.

Trusting levels

1 = I don't know or won't say
2 = I do NOT trust
3 = I trust marginally
4 = I trust fully
5 = I trust ultimately

Decrypt archive

gpg -r [email protected] --decrypt-files WorkPCUbuntuLinux_test_20141107.tar.gpg

backup's People

Contributors

Stargazers

Watchers

Forkers

shubham-gupta3197 nehak275

backup's Issues

Database backup

Mysql
PostgreSQL

make structure and data separated if possible

backup installed software packages

Debian / ubuntu

dpkg --get-selections

RHEL/Centos/Fedora

rpm -qa

yum list installed

remove old archives

daily: max 7 days
weekly: max 4 weeks
month: max 12 month

naming format in DEFAULT section

ArchiveName = %(date)s-%(instance)s-%(task)s

DateFormat = %Y%m%d
or
DateFormat = %A %d.%m.%Y

parameters

%(date)s
%(instance)s
%(task)s

union sources in one file

Like this

[Union:site-and-db-anzhiganov.com]
File = site-anzhiganov.com
Database = mysql-anzhiganov.com
Remote = webdav_yandex

[File:site-anzhiganov.com]
Enabled = yes
Directory = /var/www/anzhiganov.com/
Compression = yes
CompressionLevel = 9

[Database:mysql-anzhiganov.com]
Enabled = yes
Engine = mysql
Host = localhost
User = dbuser
Password = dbpassword
Database = anzhiganovcom
Compression = yes
CompressionLevel = 9

Restore from webdav

import easywebdav

webdav = easywebdav.connect(remote['host'], port=remote['port'], protocol=remote['protocol'], username=remote['login'], password=remote['password'])

webdav.download("remote source file", "destination file")

Dockerize

create dockerfile
add crudini for config management with environment variables
crontab in docker image

Чтение .my.cnf

Ini файл с имеющимся паролем для подключения к серверу mysql

Пример:

[client]
password=PasswordStrin1356

Email notifications

Email notifications after finish with errors and short stats.

Errors
Archive size
Total files archived

backup management tool

make restore tool

with features:

list stored backups
remove backup from storage
restore backup

Traceback (most recent call last):
File "backupfiles.py", line 75, in
backup(FilesToBackup)
File "backupfiles.py", line 65, in backup
logging.error(u'Ошибка добавления файла: %s в архив %s' % (l, Backup.filename))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 16: ordinal not in range(128)