View Code? Open in Web Editor
NEW
A complete development environment for Azure DataBricks workflows. This solution includes a development container with all the required tooling to connect, test, and deploy python based DataBrick workflows.
License: MIT License
Dockerfile 37.03%
Batchfile 1.07%
Shell 1.45%
Makefile 4.79%
Python 41.66%
Roff 14.00%
masonry's People
Contributors
Watchers
masonry's Issues
Need a new python library creation steps or script that creates a new library "project" that is connected with make file and other requirements.
Add a simple example to demonstrate the workflow/usage of Masonry
Add the existing code base to the repo, stripping away any project specifics
To bring the repo up to date and make it easier to navigate, the following needs to be completed:
The stub project(s) are currently implemented via a Git clone call. As a backup, a local (included) archive could be added.
To close this issue, the following must be completed:
Add documentation about workflow:
We need a way to populate the cluster variables without manually setting them in the configuration file.
Currently, we do not have a path for connection to clusters behind private VNETs or VPNs.
This issue should be closed when the following is completed.
Optional tasks to be completed based on the findings above:
A simple script to create or if possible, retrieve information about Azure Databricks cluster(s)
To round out the end-to-end testing scenario, Nutter should be included as part of the base, and an example of usage should be supplied.
To close this issue, the following must be completed:
The codebase should have an initial README file with basic information, any code check badges should be added.
The codebase should have linter set up before the first code pushes