phdata / sdc-api-tool Goto Github PK
View Code? Open in Web Editor NEWA set of utilities to help with management of Streamsets pipelines.
Home Page: https://www.phdata.io/
License: Other
A set of utilities to help with management of Streamsets pipelines.
Home Page: https://www.phdata.io/
License: Other
There should be no dashes in module name
Travis might be good and is free for open source projects. This could be a template: https://github.com/AndreLouisCaron/pytest-docker/blob/master/.travis.yml. This example also uses coveralls for reporting
There's the python file and script that use 'util'
using the long flag format --command-here all lowercase and dashes separating words
sdc-tool pipeline delete --pipelineId ...
verb DELETE to /vi/pipeline/{pipelineid}
Check return values for import/export start/stop etc
When executing
--dest=dev \
--pipelineJson ss-producer.json \
--pipelineId <pipeline-id> \
--overwrite
The tool should first check to see if that pipeline is already in a state of RUNNING
before trying to import. Otherwise, the import process will fail.
I'm not huge on a lot of unit tests for an application like this because it doesn't have a lot of branching and most of the work it does is with an external tool. Mocking is mostly a waste when dealing with an external api so Docker seems the good choice for most testing. The current docker 'test' is just a temporary hack, tests written in python that do real assertion and container cleanup are needed.
I think the root init.py could/should be removed. I don't believe it serves a purpose now that the code is inside the src/sdc-tool
subdirectory.
Can be enabled as part of travis build
I'm going to throw out 'streamliner'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.