ryanscottlewis / cutback Goto Github PK

Cutback only works on POSIX-compliant operating systems, but once there is option interpolation (#5) and the reliance of piping/redirection is broken (possibly based on Cutback determining if the operating system is POSIX or not), then we should be able to have Cutback be entirely cross-platform. This also obviously depends on valid Crystal targets (windows??).

Since the business end of Cutback really only runs system commands, a backup configuration should be able to generate a runnable script which runs the same commands.

Generated script could even re-implement the identifier generation functionality of cutback to continuously generate backups with incremental indicies.

• Either a --uuid option or combine with #5 and allow an interpolation option --identifier; replacing --index and --date

Right now, only Cutback errors are handled (naively). All errors should be handled and the verbosity of the configuration should denote whether to show a simple message or a full error message + backtrace (for issue reporting)

Possibly related to #1

When collecting records, Cutback should be able to optionally gather metadata about each record item including file size, *nix permissions/times, exif/filetype specific metadata.

Show a warning when this gathered metadata exceeds size of the actual data, causing the act of gathering the record of that file useless (probably a rare instance but totally possible).

If this process is also applied to manifest items, the FileListCalculator class becomes near-obsolete as we would already have that data in most cases. The only time it would be used is when a user chooses both to view progress and not generate the manifest item attributes. In this case, the user should be warned that "hey, we're already generating this data, you might as well save it)

So far, this application creates backups, but it should also list, read, update, and destroy backups as well (CRUD).

Features:

• Automatically remove backups using combinational logic
  • Older than a certain date
  • When a backup count threshold has passed
  • Backups of the same date of a lower index than the highest for that date
• Listing backups with metadata
• Viewing a backup's metadata, log, configuration, output shell script, etc..

Allow a user to output to multiple locations. for example I might want a copy of the final backup to be in both Google Drive as well as a RAID 1 NAS.

Use Process to run the tool to get the process id and report during/after build?

• /proc/PID/io
• /proc/PID/status

Auto-remove oldest after certain backup count threshold

Warn if threshold is larger than the output directory can hold?
For example if the last archive was 30GB and the output dir can hold 1.2TB then it can hold 40 backups. If the backup threshold is at 50 then the output directory will run out of space before that threshold is ever met.

Right now, files are matched to be listed in Manifest or Records by filename patterns only. It would be nice to filter by file size or combine rules such as all files less than 100MB that matches the given pattern.

Filters:

• Path pattern
• Within given path (equivalent to a wildcard path pattern such as /foo/bar/**/*
• File extension(s)
• Media/MIME type
• File/directory size
• File type: file, dir, socket, special, etc.

An option should allow the user to be user that they will not be prompted for any warning/questions when generating a backup.

Search order:

• CLI Option (--config)
• ($XDG_CONFIG_HOME|~/.config)/cutback/config.{yaml,yml,json,js}
• /etc/cutback/config.{yaml,yml,json,js}

Note that this is semi-implemented within options_parser.cr however it is not documented anywhere.

Right now, we use predetermined tools and we assume any tool passed to the option works like
the default tool for that option. Users should be able to set entire commands, although some commands are quite complicated (see find or compress).

These would require interpolations within themselves, for example xz needs -T #{@paths.manifest} so it could do something like -T %{manifest} or -T $(manifest).

This would enable users to use fzf instead of find for the search step, for example, which at the moment would not work at all since fzf's options are wildly different.

This causes Command and it's subclasses to be obsolete, allowing us to rename Action to Command to align more with the command pattern as laid out in #20.

An optional step for encrypting the archive.
Should default to being performed after compression with the option be perform before compression.

• embed/i18n.yml
• Man pages
• Website
• Error messages

This would take a whole lot of community effort, possibly through a 3rd party translation service.

Even if only a manifest or records file is generated, it's nice to see the file size totals for the files within them. Same goes for most other information within the manifest.

Validate tools before usage and either raise error or allow for continuation (for optional steps, if allowed by the configuration).

• Statically generated (using app.yml, written in Markdown)
• Host releases (cloned from GitHub releases)
• Domain name
• Examples
  • Show examples for desktop and server situations

When extracting the package (built from make package), it's contents are dumped into pwd/output path but it would be nicer (and is conventional) to have a single entry within a package, named after the version.

This would change the manual install steps:

tar xf cutback-linux-x86_64-0.0.1-pre.tar.gz
cd cutback-0.0.1-pre # The new bit
make install

Several actions/commands are strung together which form a logical "group" of actions.

Perhaps Cutback could handle both of these cases by splitting actions/commands into stages (in practice rather than theory which is currently the case) and allowing users to move actions to wherever they want within these stages.

Some options such as date and index don't really make sense for inclusion within the config files. Add a config boolean for options within app.yml which indicates if the option is also a config file variable.

At the moment, there is only a placeholder for this file.

Similar to #4, the resulting backup should be optionally signed.

Order of operations between encryption and signing needs to be considered:
https://crypto.stackexchange.com/questions/5458/should-we-sign-then-encrypt-or-encrypt-then-sign

Some steps such as compression and encryption can be completely optional.
Since they are optional, they could also be optionally skippable. For example a build configuration might have the compression step enabled, but also skippable if the platform the build is being run on does not have a compressor available.

All internal dependencies are required within setup.cr. Require at the beginning of files using the dependency? Is this even worth working on?

Instead of incrementing index on each build, it should be incremented with a new option or possibly by passing + to --index. This would mean that cutback would operate like a build automation system. Once the build for that index is created, it doesn't rebuild it.

At the moment, the only system package is for pacman on the Arch Linux AUR.

All system packages should be generated via templates and any extra commands needed, for example after generating the PKGBUILD for the pacman package the following command would need to be called in order to publish to the AUR:

makepkg --printsrcinfo > .SRCINFO

A user might want to try fzf for finding before falling back to find when fzf is not installed.

Use an array for tool options. Might depend on #5.

When a controller delete action is run, ask for a confirmation of that action, unless the --force option is given.

To avoid the excessive app. prefixes within templates.

<%= name %> instead of <%= app.name %> .

Tar the final collection of backup files into a single file:

• Optionally compress?
  • Choose to compress only the final tar rather than the archive? Or both? Is there any benefit to compressing the final tar?
• Encryption stage comes before or after this stage? Same questions as compression on this final tar.

On a side note, all compression and encryption should be optional, regardless of order of operations.

Action subclasses are simply wrappers to allow a large method to be refactored into multiple smaller methods.
Controller subclasses contain methods, conventionally called actions and called such within documentation.

This is confusing from a developer standpoint, if someone mentions an "action", are they talking about an Action? Or a controller action?

Overview graphic is semi-finished, however it references the backup threshold (#10) and doesn't mention compression or encryption (#4).

Once it is finished/up-to-date, add it to README or website. Or both?

When generating manifest (and possibly records) file(s), a checksum and/or cryptographic hash could be generated for each file entry to be used for checking authenticity and integrity of each file in the backup files archive.