pingali / dgit Goto Github PK
View Code? Open in Web Editor NEWGit Wrapper for Dataset Management
License: ISC License
Git Wrapper for Dataset Management
License: ISC License
dgit capture --schema --datatypes --samples or
Database backends:
Transformations:
Add checkers
dgit add --validator balance-check.py
Now validate
dgit validate
Some possible integration points:
Instead of having a separate metadata server, how about converting it into an RST file that can be searched/viewed in github itself. This will reduce one service depenedency.
Support a generator script - accessing the dataset's content
requires running the generator script (bash?)
The name is conflicting with multiple projects
(a) debian project
(b) github's own distributed data git
Some alternative names:
(a) dmgit (data management git)
(b) mgit (metadata git)
(c) git-data-manage
... Anything else?
Use libraries like s3git-go instead of current approach (simulate a git server and sync with s3)
Call the instrumentation module.
Has to show the following (ideally in a single tutorial):
Check for 2/3 compatibility
dgit extract
Right now we only have a generic shell command to use.
dgit sh cp -r <filename or pattern ~/target
This extract should
(a) leave a log action record
(b) incorporate git commit information
~/target/username/reponame//filename
Especially the API-based usecases
circle.yml
travis.yml etc
When repos are cloned from s3, the username is being replaced.
It should not be.
[Wrong] DrDrake/regression.git -> pingali/regression.git
Create a new repository from existing one using
the result of a transformation command
Implement assertr like assertions as a validation module
dgit add --snippet
dgit add --link s3://....
This resource is, for some reason, outside the git (e.g., too big)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.