goware / s3up Goto Github PK

It's possible hashes can be generated that are just case-different, like
zZyEhTGi
ZZyEhTGi

Should we be concerned?

During a quick inspection with cyberduck, the root bucket took a long time to list (with warnings):

It might be prudent to chop our prefixes into 2 character chunks, inspired by how git organizes objects:

it would be useful to have a -manifest=FILEPATH flag which would write a JSON file of s3.FileData in an array of the files that it had worked on during its operation.

For example.. s3up -config=s3up.conf -sync -manifest=list.json -- as this command would run, the sync operation checks existing hashes of files against the Etag (very nice), and it will only reupload files that have changed (nice nice). Upon completion, the list.json file should be written and include all existing files and all new files. This list.json would represent the state of the remote s3 bucket upon completion the operation and we can be assured that the list.json contains files on the remote server.

Finally, I would add a LastModified time.Time json:"modified"`` field on FileData.. or something similar which would give the timestamp the file was uploaded on the s3 bucket, so existing files would have an older data, and new files of course would be current date.