candide-guevara / btrfs_to_glacier Goto Github PK
View Code? Open in Web Editor NEWMakes periodic snapshots of my btrfs volumes and upload them to Glacier.
License: GNU General Public License v2.0
Makes periodic snapshots of my btrfs volumes and upload them to Glacier.
License: GNU General Public License v2.0
I will forget how to do stuff after a while not working on this project.
Document the following procedures.
We use a custom scheme to decrypt/encrypt the backup keyring.
We could do it using gpg
, that way the password would be cached and will not be stored in the application memory.
encryption
package with shell calls to gpg
.
gpg
I want to have a "quick" way to check I can restore chunks from deep glacier into a btrfs subvolume.
Put a static subvolume (aka the same subvol for any backup) in a canary storage infrastructure.
A conscious user can run this at least once a year and check all is ok. The only issue is that it will cost a few cents and take a few hours.
The keyring is stored encrypted in the configuration. If it gets corrupted or the config file is lost then all backups are unusable.
If an attacker gets control of backup role account, they should NOT be able to render the backup data unusable.
types.BackupRestoreCanary
should provide an API for validating large multichunk snapshots.
The new API should be compatible with small snapshots for that the canary can create an arbitrary chain of large/small snapshots.
Add a large snapshot test in workflow/backup_restore_canary/canary_integration
for in memory.
Does compressing stream (before encryption) bring any benefit?
~/.aws/credentials
).~/.aws/config
or the application config should be standalone?~/.aws/config
)
gpg-agent
.gpg
to decrypt the credentials using https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/credentials/processcreds#NewProviderIn cases where we need to re-encrypt sensitive material the password prompt should ask for the new password twice to prevent typos.
SessionTokenKeyring.EncryptAwsCreds
AesZlibCodecGlobalState.OutputEncryptedKeyring
btrfs-send
?Implement a "canary" backup/restore that runs before the real backup.
If there is an error when restoring ALL snapshots from the canary, then there is a compatibility issue. Probably the best is to stop and start a new snapshot chain from scratch.
btrfs-send
can encode streams using different protocols. Which one should I choose for ensuring longevity?
--proto=0
to use the latest version (depends on running linux and btrfs-progs versions).
btrfs-receive
should understand old protocols but the best strategy is to create a new snapshot sequence every few years to remove dependence on old protocols?Note: option
--compressed-data
is useless since it applies only if the btrfs filesystem uses transparent compression of files. It does not influence whetherbtrfs-send
will compress the stream.
If I call twice btrfs-send to get differential data on a same snapshot, do I get the exact same bytes ?
This is important if I want to enable resumable uploads of snapshots.
Snapshot data is broken into chunks. If an upload fails but some chunks were uploaded successfully, can I just resume the upload of the missing chunks ? Will the result be the same byte per byte as a fully successful upload ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.