iamthemovie / legacy Goto Github PK

SStable activity and compaction progress can amount to hundreds of unnecessary S3 uploads.

Either:

• we pass a --inc-system-keyspaces
• or selecting keyspaces solves this

Log file name format for some reason is:

YYYYDDMM when it should be YYYYMMDD.

New feature suggestion: Backing up the source node's token assignment can be useful if total node failure occurs. Just did some quick searching and didn't see anything about it here. Great project!

http://docs.datastax.com/en/cassandra/2.0/cassandra/operations/ops_snapshot_restore_new_cluster.html

Currently legacy stores it's paths in S3 to mimic the file system:

/{s3-bucket}/{node-hostname}/{data-directory}/{keyspace}/{table}/{snapshot}/...

From a management perspective this makes it much harder to clear up an entire snapshot in S3 without the help of legacy (a feature which does not exist yet).

Change the storage path to:
/{s3-bucket}/{node-hostname}/{data-directory}/{snapshot}/{keyspace}/{table}/...

a) Manual restores would be much easier.
b) Manual cleanups would be much easier.
c) Compile a size of the entire path would be easier (not that it would be difficult to aggregate some directories together but it's still more work).
d) We could very easily store legacy meta-data within each snapshot / per node.

There are many other benefits to this too.

Because we store the table snapshot meta by hostname (this file contains a reference to the initial snapshot name), if you rename the node and re-run legacy it will resnapshot the cluster and upload the entire node again.

Potentially we can use the UUID rather than the hostname to prevent this.

Whether or not this is an issue or not it's something that could present an issue to some users.