ethall / syphon Goto Github PK
View Code? Open in Web Editor NEWA storage and management engine for CSV data. Moved to:
Home Page: https://github.com/tektronix/syphon
License: MIT License
A storage and management engine for CSV data. Moved to:
Home Page: https://github.com/tektronix/syphon
License: MIT License
Not sure what all is required or if the package list is exclusive. Additional research is required.
Something similar to the following would be preferred:
pandas.errors
In the linked module, none of the custom exceptions do anything special, so they're defined in the __init__.py
and contain only a docstring.
Create unit tests which covers attempting to archive an empty data file.
Currently Syphon just raises an IndexError and dumps the trace.
Useful for testing purposes.
Requirements:
Not sure how the tool will figure out how many data (and optional metadata) files to add.
Keep it simple!
Create a unit test which attempts to archive a metadata file that contains multiple values under a single column header.
A ValueError should be raised.
For example, when a FileExistsError
is thrown, Syphon's exit code is 0 when it should be 1
Delusions of grandeur made the command line argument system a bit unwieldy.
Did you know?
This syphon call will complain about not having metadata files:
py -3 syphon.py import --data /some/import/path
Better add a -q
flag to force it to try and find those metadata files! This call will search the import directory for metadata files:
py -3 syphon.py -q import --data /some/import/path
Also, the quiet flag is dumb. Its default state should be quiet. Who cares about super awesome percentage indicators? Dump them both. Use verbosity flags instead.
Look for additional 'features' to dump. Maybe the switch
subcommand?
Adding a verbose flag does not explain what the program is doing.
If you make a typo in the directory name, it is difficult to understand what went wrong.
Here's the output when I accidentally passed a path that didn't exist:
Traceback (most recent call last):
File "C:\Python3\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "C:\Python3\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Source\syphon\syphon\__main__.py", line 79, in <module>
exit(_main(argv))
File "C:\Source\syphon\syphon\__main__.py", line 63, in _main
archive(this_context)
File "C:\Source\syphon\syphon\archive\archive.py", line 34, in archive
lock_list.append(lock_manager.lock(split(data_list[0])[0]))
File "C:\Python3\lib\site-packages\sortedcontainers\sortedlist.py", line 641, in __getitem__
raise IndexError('list index out of range')
IndexError: list index out of range
Probably best to raise an exception before we try to create a lock file.
Plug-in system shouldn't be too complex. Maybe check the contents of a directory for plugin files?
Plugin system should allow for things like data format verification, format conversion, value mapping, etc.
API documentation would need to be created.
To allow for continuous integration
Store default settings in a Python module rather than a JSON file.
Command line arguments (#1) should overwrite default settings.
Installing Syphon from PIP fails if sortedcontainers is not installed.
Fix setup.py to detect this missing dependency and install it before importing context in line 9 of __init__.py
Tests should include:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.