epfl-enac / enacrestic Goto Github PK

Since this can now be launched on a server, document !

As requested in #1
Give the end-user an easy way to adjust the interval between automatic backups.

As said, this can be set by editing ~/.enacrestic/state.json file while ENACrestic is off and changing the backup_every_n_minutes value.

But it could be even an easy select from a drop-down values : 30min, 1h, 2h for instance.

When launching ENACrestic for the 1st time, would be great that everything could be configured with default values + wizzard for S3 setup.

Steps that would/could be automated:

• mkdir ~/.enacrestic
• vi ~/.enacrestic/env.sh
• vi ~/.enacrestic/.pw
  note: not sure this is safe to be automated ... user has to write it down somewhere else to be sure to be able to recover his files if he loses his laptop!
• vi ~/.enacrestic/bkp_include
• vi ~/.enacrestic/bkp_exclude
• vi ~/.bashrc ~/.zshrc
• restic init --password-file ~/.enacrestic/.pw #96

Package enacrestic 0.5.1 is missing 2 dependencies:

• requests
• dynaconf

In the gui, we get a "last backup failed at ..." also when it was a cleanup. It would be more helpful to see "backup" and "cleanup" distinction in the error message.

Apparently last forget was on "2022-05-20 14:17:02". This is confirmed by state.json.
But log says :

---

2022-09-05 14:40:38.078373
Running restic forget!
Applying Policy: keep 3 latest, 24 hourly, 7 daily, 4 weekly, 12 monthly, 5 yearly snapshots
[...]
done

Process finished in 35.106 seconds.

What is going wrong ?

Does it impact last backup also in state.json ?

Testing ENACrestic raise the following error

Nov 30 17:17:56 enacvm0086 poetry[5002]: Traceback (most recent call last):
Nov 30 17:17:56 enacvm0086 poetry[5002]: File "/root/ENACrestic/src/enacrestic/restic_backup.py", line 179, in _handle_stderr
Nov 30 17:17:56 enacvm0086 poetry[5002]: if self.state.current_operation in (
Nov 30 17:17:56 enacvm0086 poetry[5002]: AttributeError: 'ResticBackup' object has no attribute 'state'

All following strings from stderr should result to Status.NO_NETWORK:

dial tcp 128.178.32.7:443: i/o timeout

dial tcp: lookup s3.epfl.ch: Temporary failure in name resolution

• Migrate from pipenv to poetry
• https://github.com/googleapis/release-please
• #20 - is delayed so that we won't trigger many new releases in current phase (to prevent users to be notified many times)

I would like to backup important configuration files present in the folder /etc, but restic complains that

! error: Open: open /etc/filename: permission denied

Any ways of achieving this with ENACrestic?

Today, ENACrestic does the check for a new version online and notify the user with a tiny red dot in the icon.

It would be more visible if a popup would come to draw the attention.

Things are note done at the right place. This is annoying for other issues to be fixed -> refactor !

• State need to be split into State + Conf
• current_state to be split into Operation + Status
• we need a queue for the commands to run

It might happen that a file disappear during the backup. This gives:

! error: lstat /home/username/.thunderbird/bla.default-release/global-messages-db.sqlite-journal: no such file or directory
!

Files: 82 new, 117 changed, 1345252 unmodified
Dirs: 20 new, 135 changed, 181152 unmodified
Added to the repo: 50.099 MiB

processed 1345451 files, 75.801 GiB in 1:16

snapshot 12345678 saved

! Warning: at least one source file could not be read
!
Process finished (3) in 76.79 seconds with status: 'last_operation_failed'

Would it be possible to have something less alarming than the red cross icon for such scenario !?

Using the old restic version 0.9.6, the feature that auto unlock stale locks doesn't work.

In case of an error in last backup, don't display a red icon if the last successful backup is within a certain period (let's say 24h).

Maybe a gray version of the backup_success would be great.

Note: we need to think about recurring failed cleanup !

Running restic backup!

! unable to create lock in backend: repository is already locked exclusively by PID 40541 on dell-2020 by sbancal (UID 1000, GID 1000)
! lock was created at 2022-02-01 15:02:55 (18h7m38.888008415s ago)
! storage ID 2c0cfe22
! the unlock command can be used to remove stale locks

potential solution :

• check if a process is still running
• unlock with :

restic unlock --password-file ~/.enacrestic/.pw

First of all, that's really helpful to have such an automated backup tool.
Installation is straightforward for someone who knows how to use linux.

Now some points of possible improvements:

1. Include the exclude_files list or document it if it is already included
2. Add some more functionality to the button:
   • Frequency of backups
   • Pause restic (pause backups when not on EPFL network)

restic has the --json flag that makes string output parsing no longer required.

This will make it cleaner and simpler to parse restic command result.

It might be useful to combine it with --quiet flag, so that we don't get all the progression status in the json.

We'd love to run it on server without the Qt stack.

Suggested command :

enacrestic --no-gui

Would be useful & user-friendly to have access to the log file just with 1-click from the system tray.

Log is going to accumulate over years ... ▶️ too heavy !

Need to implement an automatic log rotation.

Thanks for the great work! Seems to work pretty seamlessly! One question: is it possible to exclude some folders by editing a file like ~/.enacrestic/bkp_exclude for instance? As an example, one may want to include /home/user but exclude /home/user/.local/Trash. Is this possible?

ENACrestic doesn't recognize repo locked or network timeout anymore:

---

2022-12-02 11:21:49.377499
Running restic backup!
! unable to create lock in backend: repository is already locked exclusively by PID 273246 on
dell2020 by sbancal (UID 1000, GID 1000)
! lock was created at 2022-12-01 11:30:48 (23h51m1.929575232s ago)
! storage ID ace9387e
! the unlock command can be used to remove stale locks
!
Process finished (1) in 0.61 seconds with status: 'last_operation_failed'

---

2022-12-04 21:12:35.214405
Running restic backup!
! Fatal: unable to open config file: Stat: Get "https://s3.epfl.ch/something/?location=": dial tcp 128.178.32.7:443: i/o timeout
! Is there a repository at the following location?
! s3:s3.epfl.ch/10208-7796cdead8c3d3f2115c040c7a7b1589/restic_dell-2020
!
Process finished (1) in 90.08 seconds with status: 'last_operation_failed'

Both of them finish in a 'last_operation_failed' which is not OK !

CI-CD to deploy new release directly to PyPI.

Sometime we get the following error :

Running restic forget!
! unable to create lock in backend: repository is already locked by PID 300029 on dell-2020 by sbancal (UID 1000, GID 1000)
! lock was created at 2021-08-26 11:16:14 (22h47m52.477653081s ago)
! storage ID 732f566b
! the unlock command can be used to remove stale locks
!

That means that previous restic forget didn't complete and left the repo locked. This is unexpected for the end-user ... Strategy is to be found !

Launching enacrestic on a server with no GUI displays this error:

enacrestic
--------------------------------------------------
2022-11-07 07:56:25.784803
Started ENACrestic 0.1.8

--------------------------------------------------
2022-11-07 07:56:25.785084
Checking for latest release
ok
qt.qpa.xcb: could not connect to display
Segmentation fault (core dumped)

Tested on Ubuntu srv 22.04

Documentation should be adapted for servers with no GUI?

When user session takes much more time to launch and ENACrestic is autostarted, the QT ENACrestic icon is not visible.
Application works in the background, but end-user can't see it and use it directly.

Still some users running Python 3.7 which doesn't support f-string with equal specifier.

Replace it with legacy syntax.

On a VM, enacrestic.pid lock is still here after reboot. This prevents enacrestic to start !

We can see it in /root/.enacrestic/last_backups.log:

--------------------------------------------------
2023-03-24 10:18:23.798925
Started ENACrestic (noGUI) 0.7.1

Already running -> quit

We'd love to be able to run some commands before the backup is run.
This could be defined in a pre-backup hook !?

/run/user/0/enacrestic.pid path is created on user login.
Because no user logins on a server, the backup won't start.

We need a fix for server scenario. 3 possibilities:

• choose an other location, which doesn't depend on user session, but that the current user has a write access
• make it different for user root than the other users and expect that servers are backed up by root user
• put it in ~/.enacrestic/enacrestic.pid

release-please config is missing to update into src/ python scripts.

• src/enacrestic/__init__.py
• pyproject.toml
• .release-please-manifest.json line 3