mrworf / iceshelf Goto Github PK
View Code? Open in Web Editor NEWA simple tool to allow storage of signed, encrypted, incremental backups using Amazon's Glacier storage
License: GNU General Public License v2.0
iceshelf's People
Contributors
Stargazers
Watchers
iceshelf's Issues
stable release?
Do you plan to release a stable release anytime soon?
Use AWS client instead of glacier-cmd
The glacier-cmd is getting too poorly maintained and limiting. Iceshelf should make use of the AWS client which is up to date and allows us to support progress reports and better handling of resume if something fails. It's also a standard app in ubuntu which makes it easier to install.
Should solve current issues seen with 32GB archives.
Encrypt failed
Hi,
I'm trying to set up iceshelf and get the following exception:
Encrypting archive
Traceback (most recent call last):
File "/home/osmc/build/iceshelf/iceshelf", line 426
files = gatherData()
File "/home/osmc/build/iceshelf/iceshelf", line 217
gpg.encrypt_file(
AttributeError: 'GPG' object has no attribute 'encrypt_file'
I'm running Jessie and python 2.7. pip says I've got gnupg version 2.0.2
Can you help me?
UTF-8 filenames
Hi,
my test file was uploaded successfully 👍
Now I changed to real data and stumbled upon the umlauts in my filenames (I guess that's the problem):
2016-03-13 15:40:46,663 - iceshelf@140 - INFO - Processing "music" (/raid/Multimedia/Audio/)
2016-03-13 15:44:20,814 - iceshelf@398 - DEBUG - Processing file structure changes
2016-03-13 15:44:20,851 - iceshelf@173 - INFO - Content is not likely to compress (0% chance), skipping compression.
2016-03-13 15:49:01,309 - iceshelf@202 - INFO - Creating archive
2016-03-13 15:49:01,311 - shutil.py@524 - DEBUG - changing into '/raid/Temp/iceshelf/20160313-144046-80c9f'
2016-03-13 15:49:01,327 - shutil.py@376 - INFO - Creating tar archive
2016-03-13 15:53:54,889 - shutil.py@552 - DEBUG - changing back to '/home/osmc/build/iceshelf'
2016-03-13 15:53:54,891 - iceshelf@207 - INFO - Removing temporary copies of files
Traceback (most recent call last):
File "./iceshelf", line 452, in <module>
files = gatherData()
File "./iceshelf", line 227, in gatherData
json.dump(manifest, fp)
File "/usr/lib/python2.7/json/__init__.py", line 189, in dump
for chunk in iterable:
File "/usr/lib/python2.7/json/encoder.py", line 434, in _iterencode
for chunk in _iterencode_dict(o, _current_indent_level):
File "/usr/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
for chunk in chunks:
File "/usr/lib/python2.7/json/encoder.py", line 387, in _iterencode_dict
yield _encoder(key)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xfc in position 77: invalid start byte
These are some of my files:
$ ls -la /raid/Multimedia/Audio/
drwxrwxr-x 30 osmc users 4096 Dec 11 15:55 Hörbücher
drwxrwxr-x 507 osmc users 20480 Mar 2 19:02 Musik
$ ls -la /raid/Multimedia/Audio/Hörbücher/
drwxrwxr-x 3 osmc users 4096 Nov 11 13:40 Jo Nesbø
JSON file will become too big
The way data is stored and processed today is using JSON. This works pretty well until you eventually have too many changes and JSON file gets too big. Since it's loaded into memory as one big block, this will become a scaling issue down the road.
Need to investigate other methods of storing the metadata or simply splitting it into multiple files to keep it simple (would prefer not to use sqlite or similar).
This issue is created so I can track thoughts and ideas as I experiment with different ways. Needless to say it must be backward compatible and not break existing backups.
Done dir cannot be empty
The comments in iceshelf.sample.conf and README.md say that the done dir can be left empty to disable the copying after a successful operation. Still, I get the warning Done dir cannot be empty and the path points to the default backup/done/.
I suggest to replace line 256 in modules/configuration.py by setting["donedir"] = None
How to run the tool itself?
Do I just need to run the command ./iceshelf?
Use file name as archive description
I think it would be nice to use the file name (date, time, hash/rand, e.g. 20170309-140558-78a43) as archive description for all subordinated files.
These two files belong together and could be grouped by the description:
... | Archive Description | Uploaded | SHA256 tree hash | ...
... | /temp/iceshelf/20170309-140558-78a43.tar.gpg.sig.vol01+2.par2.sig | 2017-03-09T14:21:32Z | 294e1fca1... | ...
... | /temp/iceshelf/20170309-140558-78a43.tar.gpg.sig.vol03+4.par2.sig | 2017-03-09T14:21:48Z | 6dd92125e... | ...
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.